CPython - configure and test failures

I have just tried to run configure in my CPython fork, but it failed - here is the last part of the log:

./python.exe -E -S -m sysconfig --generate-posix-vars ;\
	if test $? -ne 0 ; then \
		echo "generate-posix-vars failed" ; \
		rm -f ./pybuilddir.txt ; \
		exit 1 ; \
	fi
./python.exe -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform
[ERROR] _tkinter failed to import: dlopen(/Users/srm/Documents/dev/cpython/build/lib.macosx-13.0-x86_64-3.12-pydebug/_tkinter.cpython-312d-darwin.so, 0x0002): Library not loaded: /opt/X11/lib/libXft.2.dylib
  Referenced from: <E01DC604-EE2D-3B3A-82A0-2434E46978EC> /usr/local/lib/libtk8.6.dylib
  Reason: tried: '/opt/X11/lib/libXft.2.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/X11/lib/libXft.2.dylib' (no such file), '/opt/X11/lib/libXft.2.dylib' (no such file), '/usr/local/lib/libXft.2.dylib' (no such file), '/usr/lib/libXft.2.dylib' (no such file, not in dyld cache)
The necessary bits to build these optional modules were not found:
_hashlib              _ssl                                     
To find the necessary bits, look in configure.ac and config.log.

Following modules built successfully but were removed because they could not be imported:
_tkinter                                                       

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

Checked 111 modules (30 built-in, 76 shared, 2 n/a on macosx-13.0-x86_64, 0 disabled, 2 missing, 1 failed on import)

I have OpenSSL 3.1.0 installed (in /usr/local/Cellar/openssl@3/3.1.0) and ran configure with CPPFLAGS and LDFLAGS pointing to the include and lib folders in openssl@3/3.1.0:

$ ./configure CPPFLAGS="-I/usr/local/Cellar/openssl\@3/3.1.0/include" LDFLAGS="-L/usr/local/Cellar/openssl\@3/3.1.0/lib" --with-pydebug

Can someone please shed some light on this?

Thanks, Sandeep

Looking at config.log I see this warning repeatedly:

ld: warning: directory not found for option '-L/usr/local/Cellar/openssl\@3/3.1.0/lib'

Not a Mac expert, but I wonder if perhaps there’s a quoting problem here? Try specifying this directory without the backslash in front of the at sign, see if that makes any difference. I doubt it will, but worth a quick check.

What do you get from ls -ld /usr/local/Cellar/openssl\@3/3.1.0/lib (that’s parameters L for Lima and D for Delta)?

Thanks, will try it without escaping the @. Your ls gives me:

drwxr-xr-x  11 srm  admin  352 14 Mar 12:59 /usr/local/Cellar/openssl@3/3.1.0/lib

Same result if I ls without the \@, which might indicate it’s not necessary.

@Rosuav Those ld warnings have gone away, but configure still ends with this error:

[ERROR] _tkinter failed to import: dlopen(/Users/srm/Documents/dev/cpython/build/lib.macosx-13.0-x86_64-3.12-pydebug/_tkinter.cpython-312d-darwin.so, 0x0002): Library not loaded: /opt/X11/lib/libXft.2.dylib
  Referenced from: <E01DC604-EE2D-3B3A-82A0-2434E46978EC> /usr/local/lib/libtk8.6.dylib
  Reason: tried: '/opt/X11/lib/libXft.2.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/X11/lib/libXft.2.dylib' (no such file), '/opt/X11/lib/libXft.2.dylib' (no such file), '/usr/local/lib/libXft.2.dylib' (no such file), '/usr/lib/libXft.2.dylib' (no such file, not in dyld cache)
The necessary bits to build these optional modules were not found:
_hashlib              _ssl                                     
To find the necessary bits, look in configure.ac and config.log.

Following modules built successfully but were removed because they could not be imported:
_tkinter                                                       

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

So openssl in the shell is /usr/bin/openssl, which is LibreSSL 3.3.6.

Do I need to specify /usr/local/Cellar/openssl@3/3.1.0/bin/openssl directly to configure, and how can I do this?

Ah. Unfortunately that’s a problem. LibreSSL is slightly incompatible with OpenSSL. You can read up about the decision to require OpenSSL (and the prior issues; for example, some tests would fail when using LibreSSL even before this change) in PEP 644 ­– Require OpenSSL 1.1.1 or newer.

It’s definitely possible for Python to work with LibreSSL, but it’s not supported and there’s no information as to which versions of LibreSSL can substitute for which versions of OpenSSL. The PEP has a comment regarding LibreSSL 3.3.2 which is very similar to, but not compatible with, OpenSSL 1.1.1, so you MIGHT be able to get away with building against that version; but you’ll probably do better to get hold of OpenSSL itself.

(How you’d go about using the version in /usr/bin/openssl instead of the one in your Cellar, though, I don’t know.)

Works now. I had to install zlib and then set some additional values in the flags.

./configure \
    LDFLAGS='-L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/zlib/lib' \
    CPPFLAGS='-I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/zlib/include' \
    PKG_CONFIG_PATH='/usr/local/opt/openssl@1.1/lib/pkgconfig' \
    --enable-optimizations \
    --prefix /opt/python3.11

Excellent! Installing zlib is orthogonal to the openssl issues but definitely worth having; and this looks like you’re using actual OpenSSL now, so everything should work.