Setup.py removal (Python 3.12)

Heads up, setup.py is pining for the fjords! I’m going to remove setup.py from main branch (3.12) in PR 94474. From now on all build dependencies (headers, libraries) and build flags will be detected by configure and built by make.

If available configure will use pkg-config to detect settings. There are fallbacks and overrides in place, too. For example custom zlib build options can be set with ZLIB_CFLAGS, ZLIB_LIBS and PKG_CONFIG_PATH (see ./configure --help).

$ ./configure -help | grep ZLIB
  ZLIB_CFLAGS C compiler flags for ZLIB, overriding pkg-config
  ZLIB_LIBS   linker flags for ZLIB, overriding pkg-config

$ ./configure | grep -i zlib
checking for ZLIB... yes
checking for stdlib extension module zlib... yes

The build rules for extension modules are generated by Modules/makesetup from several Modules/Setup.* files.

MODBUILT_NAMES= ... zlib ...
MODULE_ZLIB_STATE=yes
MODULE_ZLIB_CFLAGS=
MODULE_ZLIB_LDFLAGS=-lz 
SHAREDMODS= ... Modules/zlib$(EXT_SUFFIX) ...

Modules/zlibmodule.o: $(srcdir)/Modules/zlibmodule.c $(MODULE_ZLIB_DEPS) $(PYTHON_HEADERS) Modules/config.c
    $(CC) $(MODULE_ZLIB_CFLAGS) $(PY_STDMODULE_CFLAGS) $(CCSHARED) -c $(srcdir)/Modules/zlibmodule.c -o Modules/zlibmodule.o
Modules/zlib$(EXT_SUFFIX):  Modules/zlibmodule.o
    $(BLDSHARED)  Modules/zlibmodule.o $(MODULE_ZLIB_LDFLAGS)  -o Modules/zlib$(EXT_SUFFIX)

Finally there is a new script Tools/scripts/check_extension_modules.py which checks and verifies all extension modules (shared and built-in)

Checked 109 modules (30 built-in, 78 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)

The build system no longer depends on distutils and thus does not require complicated workarounds for cross-compilation and multiarch. I have tested it on various platforms and with missing or broken dependencies. The wasm32-emscripten and wasm32-wasi build bots are using the setup.py-less build system to create static builds of Python for several weeks now, too.

22 Likes

Sorry, but I haven’t been following this.

Does this mean that a package using setup.py (using setuptools not distutils) for packages won’t work in 3.12?

It means that CPython no longer uses setup.py to build some standard library modules. The change has no impact on users or 3rd party packages. It only affects our internal build system and core development.

You can still use setuptools and setup.py in your projects.

4 Likes