On behalf of the PyPA, I am pleased to announce that the pip team has just released pip 22.1!
Highlights
DISRUPTION: On Python 3.11+, use importlib.metadata for querying the filesystem-based installation database.
DISRUPTION: When building packages with --no-build-isolation, pip now validates that the declared packages are available in the environment. This cannot be disabled – users are encouraged to use build · PyPI for such usecases.
DISRUPTION: Prioritize URL credentials over netrc.
Provide user interface for supplying config settings to build backends.
More improvements to the UI, error messages and the upgrade prompt.
Speedup improvements!
This release contains many other usability improvements, enhancements, and bugfixes. You can find the full list in our changelog.
Thanks
As with all pip releases, a significant amount of the work was contributed by pip’s user community. Huge thanks to all who have contributed, whether through code, documentation, issue reports and/or discussion. Your help keeps pip improving, and is hugely appreciated.
DISRUPTION : When building packages with --no-build-isolation , pip now validates that the declared packages are available in the environment. This cannot be disabled – users are encouraged to use build · PyPI for such usecases.
I’m glad this is no longer an issue, but I find the fact that it was not discovered before the release shows pip development is sometimes a bit distant from practise, because the main reason to use --no-build-isolation at all is to build with pre-release versions of dependencies, which (to my knowledge) is not possible otherwise with PEP 517 compliant tooling.
This was purely a validation-adding change that was supposed to prevent invalid actions. However, when adding such restrictions, you need to be very careful not to break use cases that are actually valid.
The reason I’m making such a fuss about this is that I’ve already been troubled with similar cases in different projects, even in the standard library. Once such a breaking validation is integrated, it means a great uncertanity, whether you will be able to make the project’s maintainers understand your problem so they fix or revert the change, and especially how much time that convince-the-maintainer process may take.
How do you suggest we do better? We knew of no examples of the problems until our users started reporting them. Whenever we’ve produced beta releases, pretty much no-one has used them. When we issue releases, people immediately use them in production with no advance testing in a non-critical environment. We need users to test changes and report back - how do we get that?
I understand a lot of the reasons why things are the way they are, but we have tried everything we can to catch errors like this before release, and unless our users change their practices we are out of ideas. “You need to be very careful” isn’t an actionable suggestion.
Of course a general “you need to be very careful” is not helpful, and that’s not what I’m saying . I’m saying that some developers would do good to take a more reluctant attitude towards changes that only add validation. In my experience, these are particularly likely to cause breakage, while the advantages are relatively small (as no features are added, but only certain actions forbidden). Hence, such changes should not be merged unless they have been thoroughly tested on real-world use cases. For instance, you could have searched in sourcegraph for projects that depend on the option in question, see how they are using it, and what impact the change would have on these projects. Of course, whether this effort is justified dependes on the project. For pip I think it would be, as it is used very widely and forms an integral part of the Python environment.
In general, I am under the impression that Python setup tooling is changing rather rapidly, and I’m not sure if this is good for programs so many people rely on to be stable and functional.
My personally experience is that additional validation provides better guides for users to use a tool, which helps prevent a class of bugs caused by users have wrong expectations to and misuse a feature. The advantages are not relatively small, especially for very established tools, such as pip. But this is likely anecdotal, and something different people certainly can have different opinions on.
I hate to bring out cliches like “open source maintainers are not paid”, but this opens that floodgate. Esentially this means we get to do QA for everyone for free, and although I fully understand you made the suggestion with good intention, this kind of advice is, unfortunately, generally received very poorly in FOSS communities.
My general rule for myself when providing such suggestions is, “would I agree if the project maintainers invite me to do this exact thing for their next release?” (or at least help find someone that does). If my answer is no, I would try to refrain from making such suggestion, since in practice, it is much more likely the maintainers already know about this issue, but lack the resource to achieve it, and my speaking up would only add frustration.
My general rule for myself when providing such suggestions is, “would I agree if the project maintainers invite me to do this exact thing for their next release?” (or at least help find someone that does).
I find this rule confusing. You can’t contribute anywhere, in particular if you’re not familiar with the codebase and the tooling that is used, nor do I think anyone expects you to. If you are working on larger projects, it’s just not feasible to dive deeply into any dependency.
I can fully understand if development is going at a slower pace, particularly if it’s an open-source project driven mainly by volunteers. What I do not understand as well, however, is if development is going quickly, but not careful. Especially for tools that are intended to be used widely, it is of vital importance to provide stable, well-tested code, and progress for progress’s sake should be avoided.
Repeatedly suggesting that the pip developers are not “careful” is neither polite nor accurate. Please consider that you are not helping your case by this line of discussion.
“Making a fuss” as you describe it clearly isn’t helping. Please stop. You’ve been told already that your approach is unproductive.
This is not about pip developers specifically, it is about the general design decision whether to do development rapidly or rather slowly for a widely used tool I don’t mean to do unfriendly criticism, but instead argue for a perspective to consider. In the end it does not matter to me whether you consider my ways productive or “unproductive”, it matters to outline my perspective in a reasonable way for people who are open to think about it. That said, please take a step back and consider that posting an ad hominem comment against me just because you don’t agree with my argumentation isn’t helpful either.
In which case can you find a better place to make your point than on a thread about a pip prelease, please?
Anyway, this is simply turning into a game of who can have the last word. And I’m done at this point, so feel free to reply if you really want to and I’ll say nothing further.
Well, to be perfectly clear, I should perhaps have said “This is not about pip developers personally, but about the decision whether to prioritise stability or faster progress, in general and for the case of pip”…
. I’m saying that some developers would do good to take a more reluctant attitude towards changes that only add validation. In my experience, these are particularly likely to cause breakage, while the advantages are relatively small (as no features are added, but only certain actions forbidden). Hence, such changes should not be merged unless they have been thoroughly tested on real-world use cases. For instance, you could have searched in