PEP 704 - Require virtual environments by default for package installers

[konstin (konsti)]

This would be a great improvement!

As @CAM-Gerlach said in maturin we already have this behaviour, and from the replies i haven’t seen a case that goes beyond complaining that it’s inconvenient.

---

@pradyunsg the rendered link in the first post is outdated, i think it should now be https://github.com/pradyunsg/peps/blob/require-venv/pep-0704.rst

---

One important thing missing is how to detect an activated venv from a cli tool, and subsequently how this mechanism works with both normal venvs and conda. Currently in maturin, we look for VIRTUAL_ENV or CONDA_PREFIX, but it would be nicer to have a common key. We could e.g. have PYTHON_ENV_ROOT which activate and conda set in addition to VIRTUAL_ENV and CONDA_PREFIX. This would also handle a lot of the edge case scenarios which could be handled by just setting the correct PYTHON_ENV_ROOT to declare that this is an isolated (or virtual) environment.

In the PEP, what is considered a package installer? Obviously pip, poetry, etc. count but is maturin develop also included? From my perspective it makes sense to ban modifying non-isolated site packages, except for the case that the user opts in to that.

FWIW i think the long term solution is not having environments at all, but that’s still a bit into the future :​)

---

The installer SHOULD also provide a mechanism to disable this behaviour, allowing a user to opt-in to running the tool outside of a virtual environment. The installer MAY include this in the error message that it prints, and MUST include it in the documentation page it links to.

If the installer includes a link to a documentation page in the error message described above, the documentation page SHOULD be written with a focus on new users. It should explain what virtual environments are, why they are required, and how to create and activate a virtual environment. It SHOULD include instructions for the most common shells and platforms.

I’d change this to:

The installer MAY also provide a mechanism to disable this behaviour, allowing a user to opt-in to running the tool outside of a virtual environment. We advise tool authors to provide information what virtual environments are, why they are required, and how to create and activate a virtual environment with a focus on new users in error messages and documentation

While i’m not sure if maturin counts as an installer, i think it’s reasonable for tools to not support non-venv at all, and i don’t think error handling and docs should be part of the specification. I’d also add in the “How to Teach This” section that this goes into the 3.13 release notes and that this is a case where tools are meant to point the users to the right docs in case they aren’t using a venv.

---

@dstufft

One of the key things we found in the EXTERNALLY-MANAGED PEP was that there are use cases where installing into the “base” environment directly is the right thing to do. For instance, the python docker containers have a custom version of Python that you’re intended to install into directly, and while you can use a virtual environment, doing so would be pointless-- and many do not, so this would break all of those users for no real reason.

In my experience you need venv in docker again anyways for multistage builds. Additionally, python:3.13 can provide a container with an already activated .venv.

@dstufft

Which then the PEP goes on to then say we should force users into learning that information, which we just declared is a lot of information for somebody new.

So this doesn’t appear to actually make the amount of information for new users less, it just forces them to learn it up front, even if they don’t need it because they’re using another isolation mechanism.

@dstufft

It feels like it adds, or at least makes more likely, a class of new user confusion where they get confused about what environment they’re operating in and under what conditions. Dictating the .venv name does help some (you can imagine different tools deciding on different names, and a new user getting conflicting information) but I don’t think it mitigates it entirely.

From both my own experience and from supporting other python users, learning by having a completely broken/unreproducible global environment is much more painful. I believe the solution is to have tools that abstract the whole venv handling away from the user, as e.g. poetry successfully does with poetry install and poetry run.

@pf_moore

There is a whole bunch of complexity around creating an environment - do we really want to make it the installer’s responsibility to help new users navigate that?

Yes, and i think there should be standard text blocks and link that everybody just copies, essentially like:

venv = detect_venv()
if not venv:
   raise RuntimeError(
       "Can't install when not in a virtual environment. "
       "Please create and activate a virtualenv using `python -m venv .venv && source .venv/bin/activate`. "
       "Read more at https://packaging.python.org/en/latest/tutorials/installing-packages/#creating-virtual-environments"
    )

---

Open Question (and possibly an entry for the Rejected Ideas section): If a cli tool sees a .venv but it isn’t activated, should it just use that anyway (vs. saying “please run source .venv/bin/activate and try again”)