School computers will probably be set up well enough. But students may be expected to do homework, and their only computer might be running Linux.
Using third-party libraries can still be useful in a learning environment, to show something cooler than the stdlib.
I’m not claiming that “this solves the problem for them”, but making a specific virtual environment act as the user’s default environment is no harder than any other language/utility. Just add the following to your shell startup file:
export PATH=/path/to/venv/bin:$PATH
export VIRTUAL_ENV=/path/to/venv
It’s a bit manual, and having some sort of helpful “installer” to put a friendly face on it would probably be welcomed by a lot of the less CLI-aware users, but it’s not exactly hard.
The problem is really more about the fact that Python gives you the machinery to build a solution, but doesn’t provide a pre-packaged solution. That’s mostly because Python was created in an era when developers expected to get the low-level mechanisms, and honestly, tended to dislike friendly wrappers for being “too magical”. Times have changed, and newer languages offer a different UX. It’s quite possible that Python would benefit from a more modern UX, but it needs someone to write one, while still maintaining backward compatibility for the users who prefer the traditional approach. That’s not easy, and no-one has yet had the combination of time and motivation to do it.
Having spent many many years with the equivalent of this isolation (by virtue of building CPython from source and thus having a completely separate python3.Y unrelated to the system’s python3.X), I agree that this is easily the best way. But it’s still not perfect. For example, there are some Python packages that are better provided by the system package manager, eg if they come with binaries (the Python part is just a wrapper, the executable does the work - imagine for example FFMPEG), so the package can just depend on the other package. I think it should be a recommended solution, but not necessarily the only one.
Which, yes, means things are still complicated. There’s no avoiding it.
Well, I was going to suggest just alias my-python='path/to/venv/bin/python', but sure, one could also do it properly 
Well, yes. That’s exactly why we’re trying to prevent people from being root if the problem can be better solved in another way. Maybe it won’t cause a problem for hardly any users. But it’s poor discipline if people are encouraged to break the security seals without understanding them.
That is not the opinion of Debian python developers that advocated for the externally managed feature. Less so for Fedora python developers, but Fedora did embrace the feature.
The externally managed restriction is new as in PEP 668 from 3 years ago.
My understanding is that people were happily using pip to install stuff into the global environment until PEP 668 introduced a non-backwards-compatible change that broke their workflow. Hence they are now happy to use --break-system-packages which re-enables what they were already doing before.
For whatever it’s worth:
In my experience both of these are untrue.
General computer education is much more “middle-out”. For US university students, for example, this is usually a single semester of teaching how to do the things they will need for their classes, with a smattering of general computing topics thrown in. There is too much to cover, so the focus is on practical usage and accomplishing tasks, because that is what is most valuable.
Again, in the US, computer science and software engineering students are not taught about computers bottom up. In fact it’s often quite the opposite, with application programming taught before low level concepts like binary representation and how assembly language and memory work. Further, it is often the case that you are expected to teach yourself the tools necessary to keep up with class work. This why things like The Missing Semester exist!
Whether or not you think this is a good or bad approach is not relevant - in reality people learn how their OS works by doing things and will learn as much as they need to to perform a particular task, even for CS students and aspiring software engineers.
It’s perhaps worth pointing out that the change by itself isn’t
“breaking”. Someone else has to choose to activate it, by putting in the
marker file. Debian and its derivatives do. If that’s overly horrible,
that would be whom to talk to…
I don’t have this impression. But I would like these things to be true, and I would like to discuss ways to make them more true, and I would especially feel validated as a person if I actually managed to do so.
I’m not sure going down this path is getting us much. People coming to this ecosystem have whatever background they have, which may not be computer science at all. This isn’t new by any means. A lot of users of computer software aren’t specifically interested in “developing software”, they’re interested in solving the problems they need to solve, and they learn as much about their tools as is necessary to be able to use them, but many don’t go further (you can argue many don’t go far enough to learn to use them effectively, but that’s way off in another realm of problem).
That’s why modern PEPs have a “how to teach this” section. I’m not sure if that always gets enough attention, but it’s an important idea.
For now the most upvoted SO answer is corrected. I encourage everyone to work on this: write documentation, spread links to the better resources (links are essential to the search engines, you know how it is).
Here is the repository for the Python packaging guide. These days there is still enough activity that PRs get addressed and merged relatively quickly. So it is a good time to contribute. Maybe there could be a page to address --break-system-packages (even though it should be up to Debian/Ubuntu to write it). Maybe the pages about virtual environments could be improved (made more accessible to newbies). There is a lot that can be done, lot of (relatively) low hanging fruit.
If people want the two cents from someone knowing little about Python. There are still lots of install scripts out there for software dating to python 3.10 - and python 3.11 was only rolled into Ubuntu a year ago. I used ‘–break-system-packages’ for a production install simply so I could move on.
Best option I can see for someone not used to virtual environments is pipx. Are there downsides to pipx instead of pip3?
Oscar Benjamin mentioned something I’m keen to look into - how to “have pyenv manage this automatically via .python-version so that an environment is activated just by cding into a directory.”
Yes, if what needs to be installed is a an application (as opposed to a library), then pipx is a very good solution that will work in many cases. Pipx is mentioned in the error message (on Debian and Ubuntu). Pipx will give you automatic isolation in a Python virtual environment, and it will also automatically make the binaries (the executable entry points) available to the command line (via the PATH environment variable).
Surely there are some, but I can’t think of any right now.
With the collective weight of the ensurepip authors and maintainers behind it (and I don’t think any of us would be actively opposed to this idea), I think we could actually make a pretty decent case for having ensurepip accept a --with-pipx option.
The current situation really does feel like a very similar ecosystem UX bootstrapping issue to the one that PEP 453 – Explicit bootstrapping of pip in Python installations | peps.python.org and PEP 477 – Backport ensurepip (PEP 453) to Python 2.7 | peps.python.org solved: if projects can assume that pipx is available, they can give good, cross-platform command installation advice that doesn’t completely break just because the user upgraded their Python environment (which is the usual concern with recommending --user installs as a potential approach).
If the standard library itself can bootstrap a new pipx install, it doesn’t matter if an older pipx user level install breaks because the Python runtime powering it no longer exists.
PEP inclusions that would be needed to make that politically feasible:
PATH with no longer working versions, but potentially also being able to export them to a requirements file for reinstallation into a virtual environment or the user site packages for a new Python version).ensurepip --with-pipx as only working outside a virtual environment (just install pipx normally if you’re inside a virtual environment), and performing a --user install outside a virtual environment if no working pipx version is found on PATH (where “working” is defined as pipx --version giving a sensible result).pipx on PATH is shadowing the new --user installation (and offer suggestions for how to clean it up if the problem is due to an obsolete user site packages folder for a no longer installed version)pipx package, so it gets installed by default on desktop systems, but not on servers or in embedded environments, and by patching the --with-pipx option to explain how to install the relevant system package if it’s not installed)I don’t think we would need to make the --with-pipx option natively work for offline installs the way we do with ensurepip itself (so we wouldn’t be bundling pipx with CPython, just facilitating bootstrapping it in a way that redistributors can easily override).
Given that foundation, bootstrapping Python version independent tools like uv, ruff, black, pdm, poetry, hatch, flit, tox, nox, etc would become:
pipx install uv (for example)python -m ensurepip --with-pipx.python3 -m ensurepip --with-pipx (Linux distro Python, no default alias)python -m pip --user install pipx (older Python without the --with-pipx option)python3 -m pip --user install pipx (older Linux distro Python)(Edit: removed some stray -- in the list of commands at the end)
I’m a strong -1 on this. Apart from the fact that in my experience user installs are frustratingly difficult to use (not bugs as such, just design choices that I disagree with and find hard to work with), the user-site directory is specifically for the user’s purposes. System tools should never (IMO) write to the user-site.
More generally, I don’t like the idea of simply adding something to the stdlib that makes it easy to install pipx. The point of ensurepip itself was to make bootstrapping simple, so pip install --user pipx is the intended way of bootstrapping pipx. The fact that it’s not sufficient suggests that our model of “what is needed to bootstrap” is what needs to change, not that we should special-case bootstrapping pipx as well as pip.
But honestly, with the ecosystem where it is today (4 months since my previous post is a long time in uv-development time 
) I think that a standalone installation of uv, which provides an out of the box uv tool run (shortcut uvx) is more than sufficient for this need. And given that uv also offers uv run to run scripts with PEP 723 dependency metadata, I find it hard to get enthusiastic about this proposal.
Why wouldn’t option 3 be always sufficient? If you have to code the fallback anyway, why not always invoke the fallback?
(edit: I mean the $PYTHON -m pip --install --user pipx spelling)
Yeah, there would be other reasonable options to consider beyond my first idea of updating ensurepip to handle it.
One that would align well with your question would be to spell out the process for cleaning up obsolete user site-packages folders (or at least warning about their presence on PATH causing problems, similar to the way tools already warn about script directories not being on PATH), and pip then being the tool that built on those to provide a new pip ensurepipx bootstrapping command (since it can implement the enhancement even for existing versions, whereas ensurepip would only benefit 3.14+).
Fully bootstrapping an environment would then look like:
$ python -m ensurepip
$ python -m pip ensurepipx
(and in most environments these days, the first step wouldn’t be needed)
The difference from a regular install would be that if pipx --version already worked, pip wouldn’t bother installing a new copy with the currently running python instance.
If pip ensurepipx only warned about obsolete user level pipx installations messing things up (and suggested the specific file or files to delete), it wouldn’t even need a PEP, just agreement from the pip team that adding this extra bit of complexity was worth it for the potential bootstrapping benefits.
A PEP might still be a useful way to work out the details, it would just be a Packaging PEP (i.e. convincing @pf_moore that the proposed approach would be both effective and sustainable), instead of a Python one (since we wouldn’t be changing ensurepip, and wouldn’t be making any unilateral changes to the user site package environments, just potentially suggesting removal of some specific wrapper scripts when they don’t work)
Documenting that in a PEP would also be a better way to alert pip redistributors to the fact that may want to tweak how they package pip to account for the ensurepipx bootstrapping command (since what I wrote above about ensurepip would still apply, just at the level of pip redistribution rather than CPython redistribution)
Whether uv might make a better bootstrapping target than pipx was on my mind, too. It’s impressive that they’ve managed to make it both play nice with existing Python installations and provide a foundation for getting Python in the first place.
It wouldn’t be a good idea while the rate of improvement is still so high, though.
I strongly disagree with this approach, as “bootstrapping Python developer tools” is a long way from pip’s core feature set. Also, I don’t want pip to be in the game of choosing which development tools are “blessed” by the community. We spent a lot of time and effort decoupling pip from the user’s choice of build backend, and I would strongly oppose going in the opposite direction when it comes to tool management utilities.
My position remains very clear (to me, at least):
by which I mean “maintained by the core development team” ↩︎
Considering that possibility, we do also have the runpy standard library module maintainer participating in this thread (that would be me, for anyone that wasn’t aware).
I had been thinking of this in ensurepip terms, but really, all the parts that need to be independently updatable would (or could) be delegated to pip, which would make the situation more like venv (where it’s a regular standard library module that delegates some of its behaviour to ensurepip).
Since the x suffix is a popular way of marking distribution package execution, a runpyx module could potentially work for this use case (drawing on pipx and uvx tool as the basis for its design)
Even it was mostly just used to run python -m runpyx install pipx or python -m runpyx install uv, that would be a win.
Someone (not me) would still need to write a PEP, though. Some ideas on things to include:
*x utilities can work with the same set of installed tools to managepyx command (I don’t think we really need it, but an explicit list of pros and cons would be good to have)Edit: the opportunity for redistributors to patch the module to perform native package checks would also be worth mentioning
15 posts were split to a new topic: How to install a single Python application while keeping the dependencies up-to-date?