PEP 832: virtual environment discovery

PEPs
123

We want to share uv’s stance on this topic while the direction of the PEP is still under consideration.

Our goal with uv is to obviate the need for our users to directly interact with environments. Ideally, tools would ask uv (or another environment management tool) for a path to an environment that satisfies the user’s desired configuration (Python version, dependency groups, extras, and workspace package selection), so that a suitable environment can be located, created or synchronised on demand. We intend for uv-managed environments to be disposable and recreated as needed.

Since I appear to have misunderstood the options, I should clarify that our votes were for .venv as a directory or redirect file, a symlink thereto, or a junction to a directory, with .python-envs assumed to be a lower-priority source. Apologies for the misunderstanding.

We plan to support storing uv-managed environments outside project directories, using .venv symlinks or junctions as a compatibility mechanism for tools that do not yet integrate directly with uv. Some users already rely on .venv symlinks and junctions in their workflows, but support in downstream tools (e.g. editors) often appears accidental, which can lead to user-facing workflow issues, particularly around change detection. Standardising these forms of .venv would formalise existing practices and establish a clear expectation that tools support them, thus helping provide users with a more reliable experience.

Although uv will need to manage multiple environments per project, our users expect to use uv to specify their desired configuration rather than choose between those environments. The multi-environment solutions discussed here instead centre on providing other tools with a set of environments to select from. This environment-selection model is incompatible with uv’s workflow and risks giving our users a confusing, incoherent, and fragile experience. We would therefore not use .python-envs to enumerate uv-managed environments.

If we needed to use .python-envs for compatibility, we would have it refer to only one uv-managed environment at a time. Doing this with direct paths to our environments seems like it would be difficult to get right. Our idea would be to put (literally) .venv in there and maintain it as the last entry. This does not really improve our situation and seems undesirable in general.

In summary, we support standardising .venv as described above. However, we have reservations about the proposed approaches to handling multiple environments. We are not asking for this PEP to standardise uv’s declarative environment model; we ask only that it not hinder our ability to offer users a coherent and reliable experience.

We would also be happy to participate in further discussion to find a solution that addresses a broader set of needs with fewer compromises.

5 Likes
124

It’s all good! And yes, I would assume .venv would take precedence over .python-envs (or at least be an option).

What integration are you referring to? Tools that have put in the work to use uv directly via subprocesses?

And by “compatibility mechanism”, does that mean you expect to drop .venv?

How are you planning to have users specify those environments? How about selecting an environment to use?

And how do you expect tools like VS Code to know about them?

That’s why we have these discussions (although admittedly the initial support listed in PEP 832 – Virtual environment discovery | peps.python.org for the initial version of the PEP made me think I had enough support to do a PEP in the first place :sweat_smile:)!

Here are the questions I have about any solution from a VS Code perspective that would work across workflow tools (although I don’t think they are VS Code-specific, e.g. AI models need to know some of this stuff as well):

  1. How is VS Code to know what workflow tool a person (or project) wants to use?
  2. How is VS Code to know where the environment a user/workflow tool wants to be used?
  3. Can VS Code know what environments are available (I would take either existing or possible)?
  4. Is there a way for VS Code to trigger the creation of an environment (either from a list or by specifying the extras, groups, and editable install)?
  5. How would VS Code know if the environment was changed by the workflow tool if someone decided to use the workflow tool directly instead of going through VS Code?

I was conveniently proof-reading Why I wrote PEP 832 -- virtual environment discovery when your post came in, Tomasz, so hopefully that also outlines what my motivations are.

[EDIT: added question 5]

2 Likes
125

Mainly directly calling uv sync $OPTIONS_FOR_SELECTING_THE_VENV --output-format json to find the requested venv location.

That’s unlikely.

All the primitives are already there as settings in pyproject.toml and flags to uv sync. You can ask for a specific python version, platform (e.g. you can totally install a windows venv on a non-windows machine, given the right wheels), various dependency groups, or extras.

I’d imagine an IDE to present the user with a selector for all of the above in some intuitive way; only showing options that the project declares support for, prioritizing in a sensible way (e.g. I imagine most users will never want a platform that’s different from their current system so I’d probably hide that by default). Then the IDE would ask uv for an env with the given constraints. The user doesn’t care where the env is, as long as the IDE knows, and can spawn an activated shell.

By reading pyproject.toml, primarily.

Hopefully this answers some of your questions, but let me attempt them directly:

I don’t have strong opinions on this, but my default stance is that VS Code as a general purpose editor shouldn’t really support anything that isn’t a standardized feature of pyproject.toml, so it could simply pick one tool to manage venvs that supports all of these features. If a user wants to do something custom, they can use extensions. A project can always define .vscode/extensions.json with a list of (project-specific) suggested extensions.

I’ve elaborated more on these above, but tldr: VS Code should ask its chosen venv manager to materialize a venv with the user-requested characteristics, instead of trying to find an existing one and hoping it will work. The set of available environments are then purely defined based on pyproject.

I don’t yet understand why the current approach (of essentially watching for filesystem changes inside the environment) doesn’t work in this world.

1 Like
126

We’re proposing to standardise a field in pyproject.toml that explicitly identifies the tool to use. Does that meet your expectations? Or would you prefer the front-end (VS Code in the current example) to guess?

FTR, this is the basis of the protocol proposal - rather than defining “where to look”, it defines “who to ask (and how to ask them)”. Then the venv manager can allow the available environments to be defined anywhere it likes in any format it likes, provided it can respond to the “what environments are available” question when asked.

Let’s say I want to provide a workflow tool that runs all the code on remote machines (e.g. because I want my tasks to run in sandboxes, rather than locally). And the machines that are available are determined by my administrator through a browser UI, rather than locally. How does watching for filesystem changes work in this case?

My point being, we need to define the protocol for informing the front end (VS Code) rather than the behaviour of the backend (venv manager), so that backends can implement new things.

127

We would not want to force editors to guess, and are not opposed to having a workflow tool discovery mechanism. Our disagreement is over what we would need to expose as part of this mechanism—specifically, what questions an editor could or should ask the workflow tool, and how the answer would be interpreted.

Under our model, if “available” includes every environment that uv could provide, an exhaustive answer would require an entry for every supported combination of settings. This would be a very long and not particularly useful list.

If the intent is for an editor to ask which environment it should use and to be notified when that answer changes, this would fit our model. However, the discussion so far has focused on listing available environments. We don’t plan to expose such a list, nor do we plan to expose the configurations for which we currently happen to have environments. We want to avoid presenting uv’s environment-management model through an “environment list” lens.

A more extensive protocol could allow the editor to expose a configuration UI as well, but rather than trying to design such a protocol in isolation, we would prefer to see this idea explored in practice first to establish whether it would provide users with value beyond just using the uv CLI for configuration.

Editors such as VS Code currently support remote development workflows by running components that need filesystem access, such as workflow tools, debuggers, shells, and language servers, in the remote workspace. Supporting such workflows involves concerns beyond environment discovery alone. It would likewise seem difficult to specify a protocol of this scope without first seeing the interaction implemented end to end.

1 Like
128

I think your mental model is that environments are some combination of things that are restricted based on things defined in pyproject.toml. That’s a valid way to look an environments, but it isn’t the only way either. For instance, Hatch lets you specify what goes into an environment statically, so for them it is a list of environments to pick from. For others, they prefer to build up the environment a package at a time.

As I said, your view isn’t wrong, just that there are others which differ and the question is does everyone get to keep working the way they want or would someone get cut off for the greater good?

That’s not possible without vendor lock-in which I don’t support. The easiest thing for me to do is a VS Code-specific solution baked into the Python extension that frankly matches my personal workflow (e.g. VS Code puts the environment in a VS Code-specific directory and doesn’t share the location outside of the editor). But that leaves every person on a team that hasn’t standardized on VS Code and isn’t VS Code user out of luck as the solution won’t work for them. This is why I’m trying to make sure VS Code isn’t making the choice for users.

I don’t think I have ever concretely said what WSP would expose, so I think your disagreement might be with .python-envs. I have clarifying questions at the end I would love to have you answer!

It sounds like your plan is to have users specify whatever combination of extras, dependency groups, etc. they want, create the environment somewhere, and then link it into the environment via .venv. Is that correct? As I said above to @zsol , there are multiple ways to specify an environment, so I want to make sure I understand what angle you are coming from.

That has always been the core, motivating model as it’s the simplest.

OK, so your key concern is that you plan to continue to have users specify what they want in an environment upfront and then get them such an environment. How you do that is an implementation detail and all the user gets exposed to is a .venv symlink. Is that a fair assessment?

If so, the question becomes what about folks who have a different approach like Hatch or Tox that would have a static definition of what goes into an environment? Are they asked to figure out the mapping from selected extras and dependency groups to any environment they are managing? To be clear, I’m not suggesting this isn’t reasonable, I’m just making sure I understand correctly.

If my assumptions about where @zsol and @kramkowski are coming from for uv, my key questions for them as uv representatives become:

  1. Are you opposed to WSP as a concept, putting aside what might be supported by it (i.e. the concept of having workflow tools communicate how to work with them)?
  2. If you are not opposed to the concept of WSP, are you specifically opposed to WSP over JSON-RPC?
  3. If you are opposed to JSON-RPC, do you have an alternative proposal (e.g. a description for how to use the workflow tool’s CLI or something)?