Yeah, I know…but I fear that without some sort of small reference module to handle this, for many smaller projects that make use of pip like this, it is the “easiest” solution that doesn’t require a fair bit of platform-specific hackery, manual user effort or simply breaking. Furthermore, unless developers are specifically aware of this change (which inevitably many if not most won’t be until something breaks), may not immediately realize what’s happening or what to do about it when they get user bug reports about the change, and their first reaction is likely to just tell people to install pip.
Large applications like IPython and Spyder can usually afford to implement one or another workaround, so long as someone understands the problem and is willing to spend a bit of time doing it. In fact this actually could be a significant boon for Spyder at least since we can just bundle the latest pip zipapp in our standalone installers and use that instead of relying whatever versions users may have scattered through their various working environments (though this isn’t really the same for IPython, since it must be installed in each working environment it runs in and isn’t a standalone application).
However, I maintain or am involved with several smaller projects (off the top of my head, Mjolnir/Brokkr, Lektor, Pyroma, etc.) that also rely on pip being present in different types of environments to do various things, and as they are generally pretty lightweight, often structured more as libraries than applications, and may be deployed without an internet connection, pretty much any approach imposes some sort of cost:
- Adding a dependency on
pipprevents usingpipstandalone, but otherwise is the least cost to return to the status quo - Not supporting
pipin pip-less environments breaks the functionality or requires tedious user action which negates much of the benefits - Downloading and installing a zipapp is not trivial and requires an internet connection, which is not always available in the contexts which many of the above packages run
- Adding a config option requires some form of config system and manual intervention from each user (unless built into automated deployment)
- Attempting to find the user’s installed zipapp
piprequires a non-trivial amount of platform-specific hacky code that may or may not be reliable, and may still not work in a many cases
I can’t speak for Mattias & co, but while the overall idea may be somewhat similar, it is potentially a lot more complex since conda environments are highly structured and centralized, and conda will be installed in either the environment corresponding to the current Python executable that IPython is running on (if its base or the user has an environment-specific conda/mamba install, which is not common) or in the base environment, which is not that difficult to deal with. By contrast, a standalone pip could be just about anywhere, and could be in a conda env, virtualenv, venv, pipenv, pipx, or somewhere on the path.
For Spyder, though, since it already needs to find virtualenvwrapper and conda envs on the system and the Python executable within them, it isn’t necessarily as much more complicated, though still not as reliable, unless you use some form of standardized install method rather than just handling people a standalone zipapp they can do anything they want with. And of course, its fairly easy if we just bundle it in our installers ourselves (though we still have to deal with it when Spyder is installed via pip/conda/Anaconda/WinPython/MacPorts/Linux distros/source/etc…).
We (Spyder) could assume sys.executable -m pip (i.e. that pip is installed in whatever user working env we were operating on), or even just use our zipapp copy if running from our standalone installers, etc., which would be more or less true for a while, and just issue a very clear error message if it wasn’t with instructions on how to configure the path/etc. in our preferences. Most likely, we’d at least try to find it via the above approach first, which should cover most cases and minimize user complaints to an acceptable level…hopefully.
However, for IPython it may not be as nice; they are likely not going to bundle a zipapp pip just for a few magics, and I’m not sure what the obvious way would be to set and store the user’s desired pip path, (particularly persistent across kernels and environments); its probably possible but AFAIK they don’t really store that much persistent cross-environment state like we do. Perhaps setting an env variable, but doing to that effort defeats much of the convenience and ease of user of the %pip magic. But that’s really up to them. For other projects, things may be even worse, as noted above.
Also, to note, just executing pip without calling it with the sys.executable of the target environment could very well run pip on the wrong environment, since there is no guarantee it will point to the same Python executable we’re running, particularly with conda environments where activation is quite non-trivial.
At least as I understand it, you have to pass the zipapp to Python, you can’t pass Python to the zipapp as it isn’t executable on its own.