If universal2 Python is running in arm mode on macOS, will it be able to load binary modules compiled for x86?
I don’t have an Arm Mac, so I can’t test it myself…
I would not expect that to work.
The emulation is on a process basis.
I recall that you can force the use of a specific arch on macOs.
Cannot recall the command line option to do this of the top of my head.
Barry
No, a proces on Apple Silicon Macs is either running arm64 code or x86_64 code and not both at the same time. Extensions need to support arm64 to be usable in a Python proces that’s running native code on Apple Silicon.
That’s pretty easy to accomplish by building the extension (and all libraries used by it) as a Universal 2 binary on a MacOS 11 or later system (using system tooling), even if you cannot test the resulting binary on an Intel Mac. The macOS installer on python.org supports this out of the box.
1 Like
On macOs Yule the arch command like this to run in x86 mode:
$ arch -x86_64 python2.10
That’s pretty easy to accomplish by building the extension (and all libraries used by it) as a Universal 2 binary on a MacOS 11 or later system (using system tooling)
Thank you for this tip! Can you give me a link for more details? Is it possible to instruct pip install build universal2 wheels?
It’s easy for some code to turn into universal2.
But there are extensions where it is complex to re tool the builds to do universal2, for example if the extension needs openssl, building openssl is far from easy.
On a Universal 2 install of Python setuptools will automatically build Universal 2 wheels (assuming all C dependencies are available as Universal 2 binaries).
There is no way to ask pip to install universal2 wheels in preference of single architecture wheels. Sadly the pip project has chosen to prefer single architecture wheels and does not have an option to control this. IIRC there was a discussion about this on this forum.
I’m not even sure how to distinguish between single architecture and universal wheels. Presumably they have different wheel tags? If so, then the list of tags supported on a given platform is generated by the packaging project (which pip uses) - I don’t know if it was a deliberate choice or if it’s something that they would be willing to discuss. Might be worth checking for an existsing issue and/or asking on the packaging tracker.
The relevant issue in packaging is: Order of architectures in platform tags on macOS · Issue #381 · pypa/packaging · GitHub
Anyways… Universal2 wheels use “universal2” as the CPU architecture in the wheel filename. I’d like to have a way to force installation of universal2 wheels instead of the default in pip (single architecture wheels).
My primary use-case for this are tools like py2app, without such an option I’ll end up writing a library or tool that does this outside of pip. I might end up doing that anyway because there are also project that only publish single architecture wheels.
1 Like