PEP 704 - Require virtual environments by default for package installers

[CAM-Gerlach]

Some additional more content-relevant notes on the PEP, originally made on the review:

• What counts as a “virtual environment”? Only an environment created with venv/virtualenv? What about Conda environments? Or PEP 582 environments? Or other types of isolated environments? IMO, the PEP should either explicitly define this, or link (:term:) to a precise and authoritative definition, e.g. in the PyPA glossary.

• [Also, what counts as “an installer”? Obviously, pip counts (as its mentioned by name), and I’m assuming Hatch, PDM and Poetry count as well when used in that capacity, while apt, dnf, brew and choco don’t, though that isn’t explicitly stated.

  If you limit it to “Python-specific” installer, what about tools like shiv, pex, etc.? And what about, of course, Conda? IMO, the PEP should provide a precise definition of that term and examples of what tools would and would not quality (similar to PEP 668 for what qualifies as an externally-managed environment).

• Further, what “Python version” is being referred to—the version in the environment being installed in to? The version in the installer’s own runtime? Something else? The PEP should be specific here.

Workflow tools (which manage virtual environments for the user, under the hood) should be unaffected, since they should already be using the virtual environment for running the installer.

They will be affected if they don’t use venv/virtualenv virtual environments (e.g. Conda envs, I’d assume PDM PEP 582 envs, and possibly others), assuming tools don’t detect them. Either way, seems like that should be explicitly addressed in the PEP.

@tacaswell

In the sciences a very common pattern, and one that is being actively encouraged, is to put the reusable parts of the analysis into a library and have a separate “application” which uses it for the (embargoed) science analysis. In my now ~15 years of using Python, I am not sure I was ever in a situation where in-tree was a suitable, let alone ideal, solution.

FWIW, this is also my experience this or something somewhat similar is a pretty common workflow as well, what I often practice and recommend to others. Additionally, another very common scenario I run into a lot, both inside and outside of the sciences, is multiple libraries and/or applications developed together that interact and must live in the same environment.

@pradyunsg

@tacaswell

A better solution is for the pip that conda ships to simply patch out this behavior (or at least change the default)

My concern with this is that people would update pip with pip, in a Conda environment, following the instructions that pip prints.

Yeah; that was my concern above as well. My impression is that given how common this is on the interwebs and in printed messages, despite the fact that it is usually (though not always) a bad idea, that enough users are likely to do this such that this would be likely to be a large-scale UX problem.