PEP 668: Marking Python base environments as "externally managed"

[andraxin]

@kknechtel

If you, for example, build your own Python parallel to the system Python and put it somewhere that won’t impact on the system (for example, using make altinstall), by my reading nothing changes. It won’t have a marker file, because you didn’t put one there.

I know. I’m glad we build our Python images from source at work.

@kknechtel

If you want to install packages into a Python that is neither a virtual environment nor a separate installation of Python (sans marker file) that didn’t come with the computer, that leaves the one that did come with the computer - i.e., the system Python.

Wrong; although one could argue that packages installed with PIP_USER set are “sort of” like a virtual environment, they really aren’t. One might also argue that those packages are installed for “the system Python”, which is partially correct, since they are, in fact, intended to be used in conjunction with the same Python executable and some of the same system-provided packages that this so-called “system Python” uses, but they do not end up in the system’s site-packages.

@kknechtel

If you are installing things into the system Python then you are, inherently, not using it as intended. That Python is there to help your computer do operating-system-y things, not to provide a way to develop Python code.

Says who? That sounds like a fairly arbitrary distinction to me. If I’m using the same Python version anyway, why would I need to create a virtual environment and duplicate stuff unnecessarily.

@kknechtel

The distro is not “breaking” anything by trying to prevent you from installing into the system Python. It is protecting the integrity of the system Python - so that system scripts don’t have to think about dependency resolution (or whether the APIs they’re using will get deprecated later), and so that the exposure surface is minimized if someone manages to hijack Numpy or something and inject malware.

The problem is that the distro is not only preventing me from “installing into the system Python”, it also stops me from installing packages into my user site-packages. THAT is what all this noise is about.

My simple rules of hygiene I alluded to in my previous post above are as follows:

• unprivileged users have PIP_USER set in their shell’s start-up configuration
• system commands are only ever executed as a privileged user

These guarantee that my user installs NEVER pollute system packages, or break system commands built in Python for a privileged user. Yes, running some system-provided Python-based commands as a non-privileged user might break, but there I’d argue that those were not intended to be used by them; in fact, that’s one motivation behind sudo.

So, from my perspective, I wouldn’t complain if that scary flag was only needed when privileged users attempted to run pip install (which I’d never do; so, had that approach been taken, I wouldn’t even have realized this PEP existed). My issue is with the unpleasantries of this PEP being forced on those (like myself) who prefer user installs over virtual environments.

AND, make no mistake, this preference is NOT based on the storage overhead, although that’s a clear waste too, no matter how small, but the convenience of a friction-free user experience; but, I know that’s very subjective.

@kknechtel

And yes, people do sudo pip install all the time despite that they clearly shouldn’t, because there are any number of people out there who are willing to counsel you to do that - simply because it “worked for them” for some problem, that one time. If you’ve been actually modifying your system Python without sudo somehow, well… your distro trusts you a lot more than most would.

One the on hand, I’m stunned that people would do that. On the other hand, I’m not really surprised, since people do all kinds of weird stuff they shouldn’t. BTW, this is exactly why I predicted that this PEP’s approach to protection is unlikely to be effective.

@kknechtel

Yes, and there have been many times that countless beginners writing code in C let their pointers step outside the bounds of an array, and got the right output from the program anyway.

Well, I like to think that my foolproof rules of hygiene outlined above are more akin to using provably correct loop invariants that prevent bound violations; but I’d be the first to admit that as a beginner in C a few decades ago, I probably did much worse than just letting pointers step out of bound…