So if there were no open issues, what then is the reason for removal? I believe that I am set up now to be able to fix all of these. Locally, I have resolved the biggest obstacle, being able to test built installers without user interaction.
.tar.xz files cannot be uploaded either, should we remove lzma, too? As you state later, these files can be hosted elsewhere, so I do not think this provides any rationale.
Ah, sometimes else: there is PEP 632 -- Deprecate distutils module | Python.org plan (still a draft) to deprecate distutils. So if someone wants to continue maintaining wininst, I suggest to either contribute to setuptools, or to develop a setuptools plugin somehow.
If you’re volunteering to maintain distutils going forward, I’d suggest making a post on the thread you’re quoting from, since that’s way broader than just maintaining bdist_wininst in a separate location.
Note that just because Steve will withdraw the PEP, does not mean that some other person might not pick it back up and take it forward.
Various things that distutils does are straight-up non-functional/significantly degraded/incompatible compared to the rest of Python Packaging tooling. IMO letting distutils “die” is the best way to deal with a lot of user confusion and issues that have plagued us for a really long time. And that’s even if there’s some volunteer dev availability to maintain distutils (which will still be significantly less compared to the volunteer dev availability across third party build backends and build frontends).
I’ll also echo this – I’d rather have bdist_wininst in setuptools (core, or as a plugin) than to have it be improved upon in CPython’s distutils copy.
We’ve been making nearly all the improvements related to distutils in setuptools for years now (for various reasons that I hope everyone reading this post is familiar with), and I’d prefer that we continue with that, instead of taking a completely different approach for this.
Isn’t that exactly the monoculture that is routinely rallied against here? Shouldn’t we have competition in the build tools space? It is unfortunate that there is significant overlap between setuptools and distutils, but that doesn’t mean that they cannot coexist and drive improvements in each other.
Next weekeend, I plan to tackle as many BPO issues as I can and look into implementing a PEP 517 interface for distutils.
We do, but nobody can compete with “already installed on every machine”, which makes distutils the expected winner in every head-to-head. Moving its functionality to setuptools (and eventually removing the preinstallation of setuptools) removes that advantage and helps encourage users to investigate further.
This problem is exacerbated by our community’s tendancy to value “first party” over all other criteria.
BTW, the distutils PEP has already gone to the steering council (don’t have the link handy on my phone but it’s on their github repo issue tracker). You probably need to post on that saying that you’ll take over maintenance if they reject it, and they can weigh it against the other concerns.
Well, if we want equal competition, then (as @steve.dower said) distutils should be moved out of the stdlib - on the one hand so it doesn’t have an unfair “comes with Python” advantage, and on the other hand, so that it isn’t unfairly limited by Python’s development cycles.
Given that setuptools is essentially a version of distutils with many enhancements that users have wanted over the years and distutils never provided, I fear that you’re going to end up either reimplementing a lot of setuptools, or looking very underpowered in comparison with setuptools. But it’s your choice, I guess…
For a start that will mean implementing an equivalent of bdist_wheel for distutils. At the moment, distutils can’t build wheels at all, which is why everyone uses setuptools…
But if you want to enable people to use distutils without setuptools, then yes, a PEP 517 interface is essential. We’re (slowly) removing all the ways in which the packaging ecosystem special-cases setuptools, in favour of standardised interfaces like PEP 517. We’re not there yet, so there are still some areas where setuptools has preferential treatment (editable installs, for example) but the goal is 100% to make any build back-end that implements the standards an “equal citizen”.
One other thing you should probably look at is manylinux support - I don’t know to what extent code changes were needed in setuptools to handle manylinux, but I’m pretty certain that any which were won’t have been backported to distutils.
Wait, what? I’m directing you there to avoid you having to redo years of work done in setuptools. How’s that promoting a monoculture?
distutils is legacy code that has been ignored for multiple years now and all the development effort that would be needed to bring distutils up-to-date with the current packaging standards and mechanisms has been done already: that project is called setuptools.
I don’t believe distutils should be considered a competitor of setuptools – distutils is the legacy codebase that setuptools is built upon, and newer versions of setuptools don’t need it to exist in the Python standard library anymore. Notice that setuptools (on import / install) also takes control of the distutils import names.
We’ve been telling users for years to stop using distutils, because setuptools fixes many of distutils’ flaws and improves upon it in even more ways.
[snip] Others have already talked about the “comes with Python” and its implications, so I won’t touch upon that further.
PEP 632 -- Deprecate distutils module | Python.org has been approved by the SC. I suggest you to fork distutils as “distutils3” and make it super attractive. Thanks to pyproject.toml, people will be able to select it rather than setuptools. There is no need to make it into the stdlib.
Having something in the stdlib leads somehow to a monoculture. Removing distutils from the stdlib should help the competition, no? Because now all distutils alternatives are at the same level.
Note: there was a distutils2 project in the past, but it was abandoned after 1 or 2 years.