How to debug and resolve CPython test failures in `test_embed`

Python Help
,
1

Hi! I’m working on Python 3.13 support in python-build-standalone — and I know we’re doing some pretty bespoke things over there but I’m stuck on some test failures in the test_embed module and am not quite sure what the tests are even for. The source code for the tests is pretty complicated and I think I’m missing some context on how they work.

There’s quite a bit of detail about what I’m encountering at in this comment.

Basically, the PGO builds are failing on aarch64-apple-darwin. It looks like test_embed is part of the test suite subset that’s run for PGO profiling — and the tests are failing. After quite a bit of effort, I got the CPython build to hand me test output despite running the test suite with the --pgo flag.

An example failure looks roughly like this:

cpython-3.13> ======================================================================
cpython-3.13> FAIL: test_preinit_python_config (test.test_embed.InitConfigTests.test_preinit_python_config)
cpython-3.13> ----------------------------------------------------------------------
cpython-3.13> Traceback (most recent call last):
cpython-3.13>   File "/private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Lib/test/test_embed.py", line 1179, in test_preinit_python_config
cpython-3.13>     self.check_all_configs("test_preinit_python_config", api=API_PYTHON)
cpython-3.13>     ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cpython-3.13>   File "/private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Lib/test/test_embed.py", line 899, in check_all_configs
cpython-3.13>     self.assertEqual(err.rstrip(), stderr)
cpython-3.13>     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
cpython-3.13> AssertionError: 'objc[48331]: Class TKNSString is implemen[3073 chars]ned.' != ''
cpython-3.13> - objc[48331]: Class TKNSString is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032eef50) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003ede58). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKApplication is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032eefc8) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003eded0). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKContentView is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef018) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003edf20). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKBackgroundLoop is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef068) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003edf70). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKMenu is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef090) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003edf98). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TkService is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef108) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003ee010). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKPanel is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef158) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003ee060). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKDrawerWindow is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef1a8) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003ee0b0). One of the two will be used. Which one is undefined.
cpython-3.13> - objc[48331]: Class TKWindow is implemented in both /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/libpython3.13.dylib (0x1032ef1f8) and /private/var/folders/6p/k5sd5z7j31b31pq4lhn0l8d80000gn/T/tmpp72efx2c/Python-3.13.0/Programs/_testembed (0x1003ee100). One of the two will be used. Which one is undefined.

Any context on how these tests work and guidance on the failure would be much appreciated!

2

As another datapoint, other builds, e.g. x86_64-unknown-linux-gnu, are failing during bolt-opt when running test_embed with a segfault:

cpython-3.13> Fatal Python error: Segmentation fault
cpython-3.13> 
cpython-3.13> Current thread 0x00007f5ad526b740 (most recent call first):
cpython-3.13>   File "/build/Python-3.13.0rc1/Lib/test/test_embed.py", line 1676 in test_global_pathconfig
3

I replied directly on the issue: WIP: Python 3.13 support by zanieb · Pull Request #6 · zanieb/python-build-standalone · GitHub

4

For reference, I resolved the first issue by fixing duplicate copies of the Tcl/Tk objects. The second issue (the segmentation fault) is far more complicated. I got past it by disabling BOLT optimizations — the BOLT instrumented binaries specifically have the problem.