Hey everyone,
I just stumbled across this binary: python3.x-intel64 (mentioned here)
I am in a situation where I need to run x86_64 versions of Python. What I normally do, is just start an Rosetta2 terminal (arch -x86_64 zsh) and then run the normal python3.x command (without the -intel64). Does Python automatically recognize that I run it in Rosetta2 or would it be better to explicitly run the python3.x-intel64 binary instead?
Typically macOS apps are shipped as āuniversal2ā which means they have the code for intel and arm CPUs in them.
This is the case of python which is described as āmacOS 64-bit universal2 installerā.
If you are using and old version of python that is intel only then when you run on an arm mac macOS will automatically use rossetta2 to run the app.
As an end user there is nothing you need to do. It should ājust workā.
As a developer you may need to understand the details; especially if you are building binary python modules.
I would still explicitly run the intel64 binary, just to be sure. Unless you have first verified that your regular python is a universal2 executable:
$ file $(which python3)
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3 (for architecture x86_64): Mach-O 64-bit executable x86_64
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3 (for architecture arm64): Mach-O 64-bit executable arm64
What are you being āsureā about?
On my apple silicon mac I see what 3.8, 3.9, 3.10, 3.11 and 3.12 are all universal2.
You have to be running an older version then that to have any problems.
I do wonder why there is an explicit python3.X-intel64 image.
The arch command will force the use of x86_64 for you.
When I do arch -x86_64 python3.12 and then check with lsof -p <pid> I see
/Library/Apple/usr/libexec/oah/libRosettaRuntime is loaded.
But when I run python3.12 rosetta is not loaded.
All my pythons (at least those from python.org) are installed from the universal2 installer. Still I was wondering what this eplicit -intel64 binary would do. I havenāt encountered any issues yet, except I was not able to run the Python debugger in Rosetta2 mode in VSCode. Maybe here the binary could be helpful.
I also wonder, why there is an explicit -intel64 binary, but not an explicit -arm64 binary. Since I like to keep things explicit, Iād prefer running those two over letting the arch be detected automatically.
If you know you have a universal2 build, then, yes, it will load the right executable. But Pythons installed by conda for instance may not be universal2, but only arm64 (if you have an M1 or M2), and they run also inside arch -x86_64 zsh (without loading Rosetta of course). Also, people may have an arm64-specific platform Python (my 3.10 one is arm64 only for instance).
I was surprised by this. But I verified using lipo -archs/lipo -info that the conda Pythons are arm64 only. I also wonder what this means when you try to load extension module code specifically compiled for x86_64 - it should really not be possible, I would guess. But simply opening python and importing some standard modules definitely is possibleā¦
In other words, simply running in that zsh does not appear to be enough to ensure youāre running an intel64-compatible binary.
If you are using conda then you must use modules compiled for conda I assume?
And conda will install arm on arm and intel on intel I assume?
As usual āit dependsāā¦
The ānormalā python3 binary in our installers is a Universal 2 binary and should pick up whatever architecture is the default (e.g. should run as x86_64 when in a Rosetta2 terminal).
You can also start Python itself as x86_64 by using āarch -x86_64 python3ā, which is what I primarily use when I want to use Rosetta2. This has a limitation though: it can result in subprocesses running as native arm64 when launching python subprocesses using fork+exec or multiprocessing. The python3-intel64 binary should be save in that respect.
So, if I use python3.x in an arch -x86_64 zsh session, the packages that are installed are at least x86_64 is there any way to check, which arch python is running at directly within python?
You can check using lsof as I explained above.
There should be other ways to find the arch, but I could not think of one quickly.
import platform
print(platform.processor())
This will print arm when running in arm64 mode and āi386ā when running using in Intel mode on an M1 system.
I got curious and looked at the code of platform as the i386 seemed an odd answer on a 64 bit mac.
I see that its os.uname() that is used to get a lot of the info.
: 9:10:06 witcher ~
: [1] barry % arch -x86_64 python3.12
Python 3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.uname()
posix.uname_result(sysname='Darwin', nodename='witcher.local', release='23.0.0', version='Darwin Kernel Version 23.0.0: Fri Sep 15 14:42:57 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8112', machine='x86_64')
>>> import platform
>>> platform.processor()
'i386'
>>> ^D
: 9:10:46 witcher ~
: [1] barry % arch -arm64 python3.12
Python 3.12.0 (v3.12.0:0fb18b02c8, Oct 2 2023, 09:45:56) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.uname()
posix.uname_result(sysname='Darwin', nodename='witcher.local', release='23.0.0', version='Darwin Kernel Version 23.0.0: Fri Sep 15 14:42:57 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8112', machine='arm64')
>>> import platform
>>> platform.processor()
'arm'
>>>
uname give the answers Iād expect of arm64 and x86_64.
I see that, after digging into the code, that platform gets the arm and i386 from the result of running uname -p in a subprocess.
Interesting that the subprocess is using arch for intel if python is run as arch -x86_64 python3.12.
That means that platform.machine() is from a os.uname() call and platform.processor() is from running uname in a subprocess.
Running in an arch -x86_64 zsh shell is not equivalent to running python itself with that option directly
arch -x86_64 python3
The last command is safer to use since it will immediately exit if python3 is not a universal2 but only an arm64 binary while running pyhon3 (only compiled for arm64) in an arch -x86_64 zsh shell will not error - until you try to load intel-only extension module code. Safer yet would be, as you said, to be explicit about it:
arch -x86_64 python3.x-intel64
Thatās really good to know. Then I should change some of my practices :). (Also checked it, and can confirm this.)