Greetings!
I understand that, in general, the “–yes_i’m_a-blithering_idiot” (–break_system_packages) is, generally, useful.
However, it excludes the case where Python IS the system packages - like a hardware robot where Python needs low-level access to the hardware without an additional abstraction layer.
I would like to see a global switch somewhere, like containers=off,that would force Python into legacy mode.
Does this already exist? Can we do something like this?
Thanks!
Jim
Update:
This particular robot currently uses Raspbian Buster, Python 3.7 and I want to update to later versions of Raspberry Pi O/S which use a later version of Python.
Porting the software and testing it with the robot is already difficult enough. Adding containers creates another level of doubt: Is the problem software related, hardware related, or perhaps something wrong with the way the container is configured?
Thank you for your consideration.
Jim
It’s a little hard to say without making some assumptions about what you’re trying to do so apologies if I’m slightly off here.
If you’re trying to install something with pip and seeing the This environment is externally managed message you don’t need a container to get around this or to potentially break things with --break-system-packages on every pip invocation. A virtual environment is sufficient[1].
This is essentially just a folder with its own set of packages and a python ‘executable’ that will include these packages in its sys.path when run.
Essentially if your robot currently uses:
/usr/bin/pip install -r requirements.txt
To run with:
/usr/bin/python robot_script.py
The direct replacement would be something like this:
/usr/bin/python -m venv ~/robot_venv
~/robot_venv/bin/pip install -r requirements.txt
To run with:
~/robot_venv/bin/python robot_script.py
Replacing ~/robot_venv with wherever you want to store the virtual environment.
Note: If you are relying on some system python packages alongside those installed with pip you need to add --system-site-packages to the initial venv command to include those in the environment.
I believe python3-venv is usually installed by default on Raspberry Pi OS? You may need to install that if python -m venv fails. ↩︎
There is a file, EXTERNALLY-MANAGED, that you should be deleting upon container creation if you want to be able to install stuff within. See the docs.
What I want to do is globally disable containers.
Docker containers?
I’m a little confused here, I think you have the purpose of this error backwards?
The error is not for containers, it’s for the regular operating system to prevent pip from installing packages in the system python environment and breaking applications installed through the system package manager (in some cases breaking the system package manager itself).
A virtual environment doesn’t have the same level of separation as a container. With --system-site-packages it will even be able to use libraries installed through the system package manager. If you need to access hardware and need to install additional libraries for your script this should be fine.
If you still want to install into the system environment with pip and aren’t concerned about potentially[1] breaking applications the system relies upon and are happy to handle fixing the system or starting from a fresh image if it does end up broken I believe you should be able to set the environment variable PIP_BREAK_SYSTEM_PACKAGES, there may also be some other ways to configure pip to ignore the restriction. I would strongly recommend against doing so though.
You might be fine, but no guarantees. ↩︎
The problem isn’t that containers are bad per se, the problem is that containers aren’t a panacea, they’re not a silver bullet.
I am saying that there are situations that exist outside the normal baseline.
As I said before, the “normal” situation is a stand-alone application on an independent system like Windows or Linux Mint.
There are systems, smaller systems, more constrained systems, where the Python installation IS the system - and where a container is more trouble than it’s worth.
All I’m asking for is a way to re-enable the legacy behavior in those situations where it’s necessary.
Again, this does not require a container, I’m not suggesting you create a full container for this. A virtual environment is not the same as a docker or podman container. When created it’s a ~13M folder, almost all of which is a copy of pip.
The “old behaviour” was that you could break things and we didn’t warn you about it or try to stop you[1]. You can go back to this behaviour on pip with configuration as I mentioned before, but I’d still recommend against it if possible.
There wasn’t a standard mechanism, distros may have modified pip or used some other mechanism to try to prevent you from doing so but no guarantees. ↩︎