If we’re not doing the WSP approach then I’d much prefer that this PEP not handle multiple environments at all and just have it be about .venv. It would be extra confusing for users who checkout a new project that uses multiple environments to then have no environment picker without first manually using the tool directly. It’s a bit better for users of other projects in the sense that a single environment workflow wouldn’t cause the expectation for a choice (but of course introduces the same amount of friction).
I’ll do a PEP for the WSP approach toward the end of summer.
Note: this isn’t a rash decision; I told Brett ~6 weeks ago that I would write such a PEP should this one not propose the workflow server protocol (WSP), a term for which I’m very grateful that he coined
Does this imply that IDEs/(front-end tools) should look inside a path for a pyvenv.cfg (if that’s all they want to support) and ignore any that don’t have it? And back-end tools should be generating something specific so that they can recognise their own environment from path/file-system alone?
That’s up to them if they even care. My assumption is any tool that uses .python-envs already has a specific place they put their environments, so they can tell just from the path which environments they created.
I alluded to this in my first reply, but for tools that do want to know which environments belong to them, this is not necessarily trivial.
To confirm, we are happy with .venv taking priority, so we won’t necessarily run into this issue ourselves, but this came up internally so I feel it makes sense to mention here.
Environment locations may be configurable. If that configuration changes, references to environments at the old location could get “stuck” in the file.
Moreover, since .python-envs is intended to be shared between tools, it seems like it would need to specify a mechanism for safe concurrent updates. Appending paths could be atomic on some filesystems, but removing or modifying existing entries would normally require an operation which is not inherently atomic.
These are not impossible to solve, but they do add some complexity for tools that intend to maintain entries within a .python-envs.
True, but I still think that’s up to the workflow tool to decide how to manage that. An orphaned environment only comes up if:
A user or workflow tool change where they store environments
The tool doesn’t bother recording any details to know it’s their environment to maintain
No one bothers to clean it up, user or tool
The environment is somehow considered invalid while still being listed
And even if all of that passes, the user can just delete the offending environment(s) from the .python-envs file easily. This is no different then the situation happening but the user forgetting the environment exists instead of it being written down in .python-envs.
I don’t think that’s necessary due to not being a common enough situation, but if people disagree then the PEP can suggest writers use something like the fcntl module for file locking while writing to the file.
I’m currently sick, but I wanted to get the latest PEP update out while it was still the month of July.
Everything in the update I think has been discussed in some way (or clearly inferred as a ramification of something that was discussed). The key point of this update is moving from .venv redirect files to .python-envs files. WSP is listed as a deferred idea.
Looks good. One question that came to mind was around clarifying file entries should point to the “root” directory of environments rather than to any internal locations (like the Python binary or venv config file).
IF a path is for a virtual environment, THEN the path MUST be to the directory of the virtual environment (i.e. the directory containing the pyvenv.cfg file).
At least for virtual environments it seems to be clear.
I wonder if
Paths MAY be relative, and if they are, they MUST be relative to the .python-envs file.
means that all relative paths have to start with ../ because they are relative to a file (in contrast to a directory)? Alternatively, relative paths could be defined as relative to the root of the project, which by definition should be the directory where .python-envs is located, shouldn’t it?
Although it may seem trivial at first glance, perhaps an example of a .python-envs file including a relative path may make sense?
I just read the current version of the PEP and did not follow the complete discussion. In the PEP, I missed the motivation for listing multiple environments. I assume it is up to tools what to do with it but maybe an example can be given in the PEP even if it is just letting the user manually choose between several environments.
I agree this should be clarified. My expectation is that relative paths should be relative to the directory containing .python-envs (and I’d explicitly state it that way, rather than saying “the root of the project”, just to be 100% explicit).
I have 2 related minor concerns for consideration:
A security concern: an editor attempting to helpfully automatically set up a virtualenv, blessed by this spec, might decide that if it doesn’t see a .venv it might helpfully go ahead and create one, now that it’s a standard. Naively it might appear to the casual IDE author that while executing stuff in pyproject.toml or installing dependencies from it should probably require some kind of user interaction, just setting up the venv is safe, right? But if it’s following this spec, the .venvlocation might itself become a malicious DoS, overwriting some path where the user keeps useful data. This is, of course, a minor security concern, in the face of the fact that you’re probably about to trust this codebase to run all kinds of arbitrary stuff on your computer, but still, even if we were to add a super safe sandboxed-by-default virtualenv setup to every platform, this would still be an out-of-band attacker-controlled path. So at the very least an implementation note that venv creation should be a step that users should be notified of might be good.
A bikeshedding issue: as others have raised, I like keeping my virtualenvs entirely outside of my source directories. The reasons why aren’t important, but I really want to retain the ability to configure that location in a way that supersedes any configuration that I might have gotten from a newly-downloaded repo that’s got its own weird ideas about venv location. Case in point, related to point 1, maybe I download a project that has some very peculiar and specific ideas about its virtualenvs needing to be located in /opt/shared/devel/company.corp/caches/venvs/peculiar-project, because that’s the Standard Location, but I’m on a computer where I don’t have write access to /opt and I still want to work on this project. (This is a lightly fictionalized version of a thing that has happened to me once or twice, in a previous life with different tools.) It would be great if there were some explicit affordance even if it were just “tools should continue respecting VIRTUAL_ENV as a superseding configuration” or something like that. Even better would be to let me write my own project-path-to-env-path function located somewhere centrally, but I can understand why that might be a spacebar-heating level of configuration for a relatively simple feature.
I can add a note about that as a concern, but since the PEP isn’t directly involved in the virtual environment creation step, I don’t think it can be more than a “watch out” note.
That’s fair and I can note that, although it will be generic to afford e.g. activated conda environments.
See the whole discussion around WSP for the level of configuration you’re after (which is a deferred idea).
Clarify that relative paths in .python-envs are against the directory containing the file
Say that tools SHOULD respect any activated environment if the user has not previously selected an environment to use, and allow completely overriding any previous selection
Give a rationale for supporting multiple environments
While trying to prototype an implementation in Poetry, I stumbled over the expected behavior when the default environment does not exist or is broken.
I found the following relevant snippets:
Tools reading a .python-envs MAY choose what sort of environments they support and thus MAY ignore any lines they do not understand (although there is a specific restriction in regards to the default environment not being supported; covered later).
IF a tool does not support the last environment listed THEN the tool MUST either ask the user which environment to use OR error out.
For environments a tool understands but are somehow malformed (e.g. a virtual environment whose symlinks no longer resolve), it is up to the tool to decide how to handle such a situation.
Thus, I conclude that tools can decide what to do if they think they understand the environment from the last line but the environment is missing or seems to be broken [1] - error out, recreate it, ignore it and choose the second last line, remove the last line… Is this correct?
It is probably easy to detect a missing environment but it might be difficult for a tool to distinguish between a broken environment and an environment it does not support (and does not know about its structure). ↩︎
I’ve had a chance to look at the latest version of this PEP and the main question I have is regarding .venv as a redirect file.
An earlier revision dropped it in favour of .python-envs and the latest revision does not reintroduce it, but as I elaborated on in earlier comments, we think there are still important use cases for .venv as a “file which contains a path” rather than a symlink or junction, even in the presence of .python-envs. In particular, this matters when a tool wants to store an environment outside the project but cannot create a symlink or junction for whatever reason. This is not uncommon on Windows (but happens elsewhere too).
We are already doing this in uv behind a preview feature which moves environment management out of the project directory. As a bonus for us, the original version of this PEP would have standardised it as a discovery mechanism if accepted.
As written, the PEP seems to require .venv to resolve to a directory, which would exclude this approach.
Would you consider including .venv as a redirect file (or something similar) in the PEP again?
Is “we” in that sentence “uv”, because the initial feedback I got from Zanie was they didn’t want the .venv redirect file. So has the uv perspective officially changed?
Correct.
If you can convince others, then sure. But I also don’t see why .python-envs prevents this? I can see why you now want the .venv redirect file as uv users already have .venv in their e.g., .gitignore file. But functionally I don’t see it as a drastic shift from .python-envs beyond any concern you have of some other tool deciding their environment is more important (e.g. Nox or Tox).
Yes, I am speaking for uv. I checked with Zanie and, from what we recall, our objection was to the “SHOULD” wording which made .venv something that tools were expected to provide. We have plans to allow users to opt out of maintaining a .venv altogether, as there are use cases where its presence causes problems for users. The current “MAY” wording makes it optional, which we’re happy with. We never had any fundamental objection to .venv being a file rather than a directory.
.python-envs is a shared list with no single owner. It may be trivial to add to it, but we can only reliably present users with a single uv-managed environment through this mechanism, the path to which may change depending on configuration. An environment we previously listed may still exist while no longer being kept in sync with the project, so we’d want to remove our old references to avoid confusion, without disturbing other tools’ entries. But not only is it more complex to mutate a shared file correctly, there are also a number of corner cases which mean we may have to guess which entries belong to us. That adds complexity and uncertainty that we would prefer to avoid.
The objections regarding multiple environments and other environment types, also listed in the rejection rationale, are addressed by .python-envs. But this does not appear incompatible with also allowing a .venv redirect file. Backwards compatibility remains a concern, but we would only want to create .venv as a file as a last resort when we cannot make a symlink or junction. If the PEP excludes that fallback, we cannot provide a conforming .venv in those cases. A tool that does not understand redirect files would not discover our environment through .venv in either case, but a redirect file at least tells the user where the environment is, so they can point the tool at it manually. A redirect file also makes a tool that does not understand it less likely to create a separate environment, avoiding what would probably be the most confusing outcome.
If the ability to remove entries would be helpful, maybe a name tag on the entries in .python-envs would allow you to identify the environments your tool owns.
My mistake then! I had thought Zanie didn’t like the failure conditions of .venv being a file for code that wasn’t aware of redirect files.
Incompatible, no. But it is asking me to expand the scope of the PEP and be willing to defend including both .venv redirect files and.python-envs files to whichever council will be deciding this PEP.
As I said, if other people step forward in support I’m willing to consider bringing a .venv redirect file back.
See the PEP as to why I won’t be doing that and the suggestion on how to solve it.
At this point I’m waiting on:
PyCharm to weigh in (I’ve asked, but I’m starting to lose faith it’s ever coming)
If anyone else beyond uv wants .venv redirect files brought back
If any other tools want to publicly state any support
I already started a discussion at 
I’ll include one in the next update.