A rare annoyance with Pip

Scenario: suppose we are on a Debian-based Linux distro, such that the system Python is nominally externally managed (and doesn’t include Pip by default, but does include some useful third-party libraries such as Requests). Suppose further that we create a virtual environment with the system Python as a base, specifying --system-site-packages so as to have access to those.

(This might also require Python 3.10 or before, such that there is no marker file for external management - I plan to upgrade to Mint 22 promptly when it becomes available, and that should include 3.12, so we’ll see.)

In this configuration, pip install (with the venv activated) will respect the -U/--user flag (because the venv has access to system libraries), but pip uninstall will not be able to uninstall the package (because the installation is outside of the venv). In theory, uninstalling from the system Python should simply use the system Pip, but that doesn’t exist here. Other attempts to make the venv’s Pip exectuable operate on the system Pip all also fail, as far as I can tell.

I don’t naturally find myself in this scenario[1], but it came up while I’ve been cleaning up my (temporary) Mint 21.3 upgrade and simultaneously doing research for some Codidact Q&As.

  1. I don’t consider the system Python to be suitable for development; I don’t mind doing a redundant installation of Requests etc.; and thus I don’t use --system-site-packages for my venvs. I also don’t use --user because I don’t want to install anything for the system - that’s mainly why I’m using venvs in the first place. But I can imagine someone following a tutorial or something that suggested the --user flag, or some higher-level tool that used it automatically. Apparently this was an issue for VS Code in recent memory. ↩︎

The system python never installs with pip anything in a distro
that sets the externally managed marker. The distros package manager is used.

I meant, if I’d installed the system package for Pip and then used that pip to do pip install --user, I’d expect to use the same pip uninstall to uninstall. Since pip install --user goes to the same place regardless of environment, it stands to reason that the same Pip would work to uninstall things that were user-installed “in” (but not actually in) a venv. But I can still install that package without having a system package for Pip, which is the problem.

Why do you need the -U/--user flag, once you’ve created a venv and activated it?

Is it really intended behaviour, to allow -U/--user to permit a venv pip to alter a system Python outside of the venv? Or is this behaviour another quirk of Linux distro repackaged Pythons?

I thought that you cannot pip3 install --user on debian, works on fedora at the moment. Becuase that can break system tools when the user runs them if you
install an incompatible newer package. Arn’t you forced to use a venv now?

I personally, certainly don’t need it, and I’m not sure why anyone else would. But I would definitely expect that someone out there has ended up in this situation. Considering the discussion around the external-management marker system in the first place, it seems that some users try some really wild things. There’s even a cargo cult around the use of sudo with pip, it seems. I’ve found some really awful stuff in old Stack Overflow Q&A.

Now, i.e. if you have an up-to-date Debian release where the system Python is 3.11 or newer. There are distros downstream of Debian, and a lot of users prefer to stick with LTS releases anyway. Mint users would have to be on 21.x (and 20.x hasn’t EOLd yet!) and install 3.11 via apt explicitly (or wait for 22 to come out which should come with 3.12, via Ubuntu 24.04). Nothing forces people to stop using software past its EOL support date, either (cough, Python 2.7).

Indeed. Thought’s gone into pip install in a --system-site-packages venv.
Until now, I assumed the venv and pip therein could access the system site packages on a read only basis.
Understandably, pip uninstall has received less attention. Possibly because deleting a venv and making a new one should be easy.

Note that in the pip install command, the -U short flag stands for the --upgrade flag (not --user).

Oh, that’s my mistake in the OP as well :sweat_smile:

1 Like