Debundling the next pip release will require handling pyproject.toml-based build-backends

Packaging
1

This is mostly a heads up for redistributors, since I can’t really figure out any other good place to put this. There’s been a bunch of discussion around this in pip’s issue tracker, and I figured that this is important enough to call out in a more “visible” spot.

The upcoming pip release will have rich vendored in it. You should be able to find references to most of the discussion and lists of things we’re going to be doing around this in this issue: Adopt rich, for handling terminal output · Issue #10461 · pypa/pip · GitHub.

The main thing I want to say is: I’d strongly encourage redistributors to NOT reach out asking Rich to switch to setuptools. Rich is packaged with Poetry[1] and I expect other projects that pip vendors to also start moving to other build backends over the next year. Pulling back any part of the Python ecosystem that uses newer build systems is NOT the solution here. FWIW, even if the maintainers of Rich are very accomodating and change their entire workflows to use setuptools… in a few months, when packaging will move away from setuptools, I’m unlikely to be as accomodating[2]. :slight_smile:

Also, I don’t think there’s any version of this where pip drops this dependency either[2:1] – I don’t think it is reasonable to block significant UI/UX improvements in pip, for being accomodating toward redistributors who aren’t ready for handling pyproject.toml-based projects and don’t do what we recommend in our vendoring policy.

Rather, I encourage any such redistributors to fill in the gaps in tooling they have, so that it can deal with pyproject.toml-based projects. build (Python API and CLI) should have everything necessary to generate wheels from pyproject.toml-based projects and installer (Python API) should have everything necessary to install them in a way that pip would[3]. If there’s something missing in these projects, please reach out to those projects on their issue trackers and talk to their maintainers there. Let’s not have an extended discussion about them here.

PS: I strongly encourage everyone to start saying pyproject.toml-based projects, rather than using a PEP 517 to identify the newer build system interface; especially since we now have two PEPs that specify it (PEP 517 and PEP 660). Or more, if you wanna include stuff like PEP 518 and PEP 621. And, if you know what each of those numbers refer to without looking it up, then… you’ll hopefully also understand why this is bad for communication with a wider audience and how using the file as the identifier can help. :slight_smile:

  1. Since it’s inception I think. It also has a setup.py file for making GitHub happy, which should not be used. Also… yes, footnotes are here. ↩︎

  2. I’m not speaking for the other maintainers on the project, who may feel differently. ↩︎ ↩︎

  3. For a single wheel with no dependencies. ↩︎

10 Likes
2

I sympathize, but I find it unlikely that this will deter redistributors from raising the issue with pip or rich as there is still no way to replace pip with CLI tools.

I would recommend that Add CLI by FFY00 · Pull Request #66 · pradyunsg/installer · GitHub gets resolved as soon as possible to avoid further frustration. That is the only missing piece to solve all of our issues. I understand that time does not grow on trees, and I thank you for all of your effort making this better so far, but if you have a little bit of time to help the installer CLI move forward, I think it would be very worth to do that final sprint as we are very close to the finish line.

4 Likes
3

FWIW, distros which use setuptools for installation are unlikely to mind that rich is built with Poetry, and with rich being as popular as it is they would’ve already packaged it for distribution. Distros which use pip for installation, i.e. those distros that have actually embraced PEP 517, might struggle if they do also devendor pip. I don’t know if that cohort exists at all.

4

I agree, it’s probable that people will still try to make that request. But I think it’s fair to have a note like this that states clearly that “the packaging community”[1] would support rich saying “no” to any such requests.

Honestly, if we don’t feel capable of defending the consequences of people adopting the standards that we’ve agreed to, then I think we need to take a better look at our standardisation process. Maybe we need to insist that all standards include a detailed transition plan, so that we have well-defined and agreed views on how this sort of thing should be handled. Although I suspect what that would do in practice, is make it much harder to get any standards agreed :slightly_frowning_face:

While I agree that would make things a lot easier for this case, for the normal user, pip wheel is fine. I don’t think we need to go too far out of our way to accommodate people debundling pip’s dependencies against our advice, and from the sound of it not following the process we recommend (it’s hard to be sure, as this is all mostly theoretical at this point).

  1. Whatever that is, but at a minimum it’s “the people developing the standards that modern packaging is built on”. ↩︎