I recently noticed that the Austin CI jobs on MacOS started failing for Python 3.11 and 3.10. The jobs use the releases provided by Releases · actions/python-versions · GitHub. In particular, the 3.11 images ship with the official MacOS installer since at least the rc2 release. Upon investigating, I found out that the binaries seem to have the symbols stripped. Was this a deliberate decision? Would it be possible not to strip symbols from the binaries (at least the library) to allow tools that rely on them to work?
@nad any insight here?
My understanding is that, on macOS, executable items produced by the linker (things like main executables and shared libraries) do not by default contain DWARF debug symbol table information that might be present in the object files due to their size. There is an LLVM utility, dsymutil, to produce a separate .dSYM bundle that collects and contains the debug symbol information for the corresponding executable file. Support for calling dsymutil within Python builds on macOS was recently added as a feature for Python 3.12, now under development; it is enabled by a new ./configure --with-dsymutil. Since we are still early in the development cycle of 3.12, that feature has not received a lot of exposure yet and has not yet been incorporated into the build and packaging of python.org macOS 3.12 pre-releases.
Hummm, but that’s debugging information.
The binaries should have a symbol table in the LC_SYMTAB, LC_DSYMTAB or the new export trie on LC_DYLD_EXPORTS_TRIE linker command (or the previous version of that).
In general, these should be present unless stripped or unless the linker decides not to include them for some odd reason when building framework builds.
Indeed I should have specified that I’m referring to the exported symbols, and specifically to the LC_SYMTAB command, in this case.
Sorry, this is not an area in which I have much expertise. @ronaldoussoren Any ideas?
A normal macOS build of the interpreter has symbols so the question is if there is anything in the build process for the macOS binaries on our side or whatever apple tools are used that strip the symbols.
Do you know if when building the installers there is something that can do this?
Not that I know of. But symbols on macOS are a bit complicated and have evolved. And of course the .o files from which the various installed artifacts are linked aren’t included in an installed Python (and were built on a different macOS system likely with different versions of the os and Apple build tools). So @P403n1x87, what commands against what files did you use to determine that there aren’t symbols present?
Can you reproduce this locally? On my laptop the shared library in the framework do contain symbols according to both nm(1) and by checking that a LC_SYMTAB load command is present. This is for Python 3.10, 3.11 and 3.12 (all of them installed using the latest release on python.org).
I have used the installer from https://github.com/actions/python-versions/releases/download/3.11.0-3328127706/python-3.11.0-darwin-x64.tar.gz locally and used nm on both the binary (…/bin/python3) and the library (…/Resources/Python.app/Contents/MacOS/Python). All I get is the __mh_execute_header symbol 
I don’t know what that download is and I’m away from a keyboard but that’s likely not the official installer. Go to python.org downloads to find it. We don’t release through GitHub.
The official installer is https://www.python.org/ftp/python/3.11.0/python-3.11.0-macos11.pkg
1 Like
The setup-python action maintainers claim they use the official installer now
Please, try to install the official installer from our download page in your machine and then check if the problem is there. If is not there you should then ask the GitHub actions maintainers.
Also, to be able to install on users’ systems without violating macOS Gatekeeper requirements, the official installer binaries are code signed and have specific entitlements. As Pablo suggests, you should try installing on your own macOS system using the official installer download. In a CI environment, you might need to copy the binaries and/or do your own code signing. You should not attempt to change a system’s SIP settings to work around this.
I’ve uninstalled my local installation of Python 3.11 from the setup-python release. Downloaded the official installer (which happens to have the same size as the .pkg contained in the setup-python archive), re-installed Python 3.11 with it, and still got
❯ nm -gU /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
0000000100000000 T __mh_execute_header
❯ nm -gU /Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python
0000000100000000 T __mh_execute_header
And Python 3.12 from the official installer
❯ nm -gU /Library/Frameworks/Python.framework/Versions/3.12/bin/python3
0000000100000000 T __mh_execute_header
❯ nm -gU /Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
0000000100000000 T __mh_execute_header
I can reproduce this result. This is on macOS Monterey 12.6, on an Intel Mac, with the Python 3.11.0 from python.org.
I get the same result for Python versions all the way back to 3.5.
I suspect the official python.org installer has always behaved this way, and if something has changed, it is the “releases” put out by the GitHub Actions. Which was mentioned in the Original Post. What’s less clear to me is what the OP means by the phrase “In particular, the 3.11 images ship with the official MacOS installer since at least the rc2 release.” Gabriele, can you help us understand what you meant there? It seems everyone has assumed that you were reporting a change in behavior in the rc2 release, but it sounds more likely that you are just stating a fact that you have only verified in rc2 and later.
It seems what they’re saying is that since at least 3.11 rc2, the GitHub Actions container images use the official Python.org macOS installers (and per the linked thread, this started with either 3.11 rc1 or 3.11 rc2, and will be done in the future starting with the first rc), which in turn exposed the stripped symbols in the official installer (that per your testing was an issue back to Python 3.5), that was evidently not a problem with GitHub’s own installers used previously.
@nad It looks like something in the process of building the installers strips the symbols. Can you share the compiler commands and scrips you use to create the installers to see if we can reproduce?
I suspect that maybe when creating the installable thing some apple software is doing some shenanigans to strip symbols for apple reasons.