It seems to me like you’ve identified three classes of users:
Brand new users who know absolutely nothing about Python and want to learn. These users should not be exposed to multiple different ways to handle packaging and running python. I don’t think that’s a controversial point of view.
Users that are deeply familiar with what they need as far as packaging and tooling.
Users stuck between 1 & 2.
I definitely agree we could have better documentation/help for the third group of users. But putting that help right on the front page of python.org doesn’t seem to me to be the right solve.
I think the issue is that the people in group 1[1] want more from the out-of-the-box experience than they are currently getting, but it is complicated to add those kinds of features to the base install without effectively pushing out a lot of third-party projects. There’s no appetite to anoint one of those third parties as the recommended default, so we’re in this state, and it’s been discussed a lot at this point.
I think the issue is that the people in group 1 want more from the out-of-the-box experience than they are currently getting,
Out of curiosity, what do you think is missing from the out-of-box experience from a packaging perspective? Specifically for Group 1 users (Brand New and Wanting to Learn).
The start of this thread is an example[1]. A lot of users want their programming language to come with workflow tools: an official way to manage projects and environments. But pip is not a project manager, it’s an installer. There is no project manager that ships with Python (in part because there are several good choices).
Improving the documentation for Python is always helpful, but there’s an invisible mass of “documentation” that we cannot control, which is the background that new users are coming from. If they come from a language that includes workflow tools, they can become adrift in an ecosystem that offers them more choice and less guidance.
not that baggiponte is a novice, but the same question has come up before ↩︎
Some people might even suggest this is a good reason that baseline, reference packaging tools that adhere to extensible standards and protocols, but that themselves only extend only as far as simple pure-Python cases, ought to be included (anyone remember “batteries included”?) in and documented in the standard library.
It doesn’t solve everyone’s use case yet, but my point is that I view Hatch similar to Cargo+Rustup where the former is what people generally will want to download and that is what manages Python for you.
I’m certain that in the future (maybe 5-10 years) folks will at large download an application rather than a Python installation.
Could you please provide clarification regarding “minimal baseline of expected capabilities” and its relation to my post you were replying to?
For context, Pip can break a user’s environment when they execute more than one install command. Consequently, to ensure a Python packaging tool that can guarantee the absence of environment disruption, the workflow is as follows:
If Pip is not bundled, users can utilize ensurepip or another bootstrapping process
Pip can be used to bootstrap another tool for managing the user’s environment
The third-party tool can then be employed to install additional packages
So here is my confusion about “minimal baseline of expected capabilities”, if it does not require installing any third-party tool then why is ensurepip included in the standard library or Pip bundled with the python.org installer? However, if the phrase encompasses the installation of multiple third-party packages while ensuring the integrity of a user’s environment, then Pip does not currently meet this requirement.
In the days prior to ensurepip, there was no standard way to consistently and securely bootstrap any flavour of cross-platform Python package management. The closest option was to download and run a bootstrapping Python script from a given URL, and even via HTTPS that option will rightly raise the hackles of even slightly security conscious computer users. The situation was so bad that a lot of major libraries and frameworks vendored their dependencies (or avoided having any non-stdlist dependencies in the first place), since they couldn’t be sure that their users would have any kind of dependency handling tools (at that point in history, “Python package installation” for a lot of people was still “download tarball, unpack it, and run ./setup.py install”).
Thus PEP 453 successfully making the case for adding ensurepip to the standard library (and PEP 477 subsequently making the case for backporting it to the Python 2.7 binary installers).
The arguments that applied then (and still apply now) for ensurepip haven’t applied for anything else, since “pip install it from PyPI” is a far more palatable bootstrapping solution than the alternatives that existed prior to PEP 453 and PEP 477.
The one kind of tool where I could potentially see a case being made for the ensurepip treatment is something like pipx (running Python applications in independent virtual environments), since that style of installation is often more appropriate for tool bootstrapping than regular installation into the current Python working environment. However, those tools brings sufficient other new user experience problems of their own (mostly around keeping those virtual environments up to date, and handling subsequent Python version upgrades) that there hasn’t been a serious attempt yet to argue for the inclusion of something along those lines.
For those reading this who are young enough in their Python careers, and therefore lucky enough, not to have seen it before, here’s a snippet from get-pip.py (which is still available!) which may help shine a light on what some of the history here looks like:
# Hi There!
#
# You may be wondering what this giant blob of binary data here is, you might
# even be worried that we're up to something nefarious (good for you for being
# paranoid!). This is a base85 encoding of a zip file, this zip file contains
# an entire copy of pip (version 23.3.2).
This used to be The Way to get pip, as @ncoghlan was saying.
It’s great to push for improvements. And asking for something doesn’t require that you know all of the background.
Understanding why some features are accepted and others are not often does require learning quite a bit though.
pip’s place in the world is unique to it. We may compare it with other language tools (e.g. cargo), but it’s the only Python tool with its history and position vis-a-vis cpython.
A few years ago, on GitHub, I was part of a group of users who (somewhat loudly) asked for a pip publish command. It seemed easy. twine was already maintained by “the same people”, the code was written, the interface seemed more or less trivial, etc etc. That was my first time interacting with some of the folks on this forum. They explained why it was not obviously a good feature, why it is more complicated than it seems, and perhaps some other details I’m forgetting.
I didn’t get it at the time, but I do now.
My story has three purposes:
For the pip maintainers, please keep explaining pip to people! It’s a lot of work to explain things. Sometimes it won’t bear fruit immediately. But it is worth doing.
For those asking questions, I encourage you to treat this primarily as an opportunity to learn more about the ecosystem. If your goal is to change things, without a broad view of the problem space, your odds of coming away disappointed are high. Remember that “I learned some stuff that will help me contribute in the future” is a win.
Do we need some write-up somewhere on “Why pip is not a workflow tool”? Like I said, the energy poured into explaining things is great and a good investment. But I worry about the cost of having to explain this in different ways to different people, over many years. Maybe we can make the process less taxing for community members?
Agree with you on all three points. This is one of the first times I have interacted this much on the forum and I’m glad I did - I learned nuances I did not get from the docs. I would gladly volunteer to help about the last point you mentioned.
