Adding the old `distutils.ccompiler` to the stdlib

Would any of the standard libraries in question have any C components? I can’t recall any. If there are no binaries to include then it should be a simple matter of verifying the source against public record of the last maintained version.

Full disclosure: I’m not vetted either. :sweat_smile:

This is a reasonable concern. All I know is that the person who put them up (@youknowone) is one of the primary contributors to the RustPython project, and also contributes to the Rust python parser and ruff. If he’s a sleeper agent he’s in deep.

1 Like

Distutils was removed because it was an evolutionary dead end – so much
code relied on details of its internals that it was impossible to ever
change any part of it.

The trouble is that the new packaging stuff intended to replace it only
covers part of its functionality.

If something is to be added to the stdlib to cover the compiler
functionality, it would be better to design something new and better
than just pull in the old distutils.ccompiler module, since that would
just be bringing back all the old problems.

I think it could be something very small and simple. For me the main
benefit would be that it would know how to compile extension modules for
the current Python installation with all the right compiler options.

1 Like

This is exactly the reason why I think python core or PyPA have to do it.

I left a letter to the project page.

The packages will be frozen as they distributed to prevent supply attack and the cost to review the patches. But still don’t believe me, I will not catch something goes wrong even if my account or PyPI API key is hacked.
If you agree to the points, please ask Python core or PyPA to redistribute deprecated stdlib libraries to PyPI when they remove it.

1 Like

From what I’ve looked into, there’s nothing small about a platform-independent C compiler interface, since you pretty much have to reimplement it for every compiler you want to support. distutils.ccompiler is the only implementation that I know of that actually did this, so you’re probably just better off using the vendored copy from setuptools rather than spending however much time building one yourself. If you’re going to design a new one, I doubt it will make it into the standard library – it will likely just exist on PyPI.

Thank you for mentioning! I didn’t know about distutils at the moment. I am adding it to deadlibs.

Is there a proper overview somewhere of exactly what the full functionality is?

1 Like

If you are looking for a Python-based build system for non-Python projects then I can recommend meson. If you want to build Python packages with meson then you can use meson-python.

Meson can handle different compilers etc for you and is actively maintained. Meson-python is one of many possible candidates that you might choose to use when migrating from Python packaging to distutils via PEP 517 (a build-system independent format for source trees). Both numpy and scipy now use meson and meson-python for development and building. I have personally ported python-flint from distutils/setuptools to meson/meson-python and I am happy with the improvements in compiler detection, reduced build times, editable installs for C code, deterministic rebuilds etc.

As @gcewing says distutils has design problems that are unfixable without a complete rewrite. That is why no one wants to “maintain” it: all existing bugs and limitations are unfixable. The best anyone can offer as maintenance is what is described here

Please do NOT submit any new features or any compatibility work. This repository is archiving the old standard library installable via pip.

Any other kind of maintenance besides posting an archive for download would entail making changes to distutils and could only lead to any significant improvement by breaking most current usage of distutils.

1 Like

My proposal was just for general-purpose use, not specific to packaging.

Meson is for general purpose project building and is not specific to packaging Python projects.

My answer is not only intended for you though but also people such as @gcewing who wants to make extension modules for the current Python installation: that is something that meson can do with its Python module. Other people looking to replace distutils are likely looking to build Python packages and meson with meson-python can do that as well.

Basically distutils is unfixable and should be replaced by things that involve complete rewrites of any functionality that it provides. One such candidate for that is meson.

3 Likes