I’d like to propose adding a python-version field to pyvenv.cfg. Unlike version and version_info, it records only the major and minor version and is resilient to patch version updates.
This is a small change: The implementation in virtualenv is for example an 8 line change (pypa/virtualenv#3193), in uv it’s a 36 line change of which 4 are production code (astral-sh/uv#20569).
While it mostly affects packaging tools, with the required change to the stdlib venv module this seems to be standards track PEP more than a packaging track PEP.
How will this work for implementations other than CPython (e.g., PyPy or GraalPython)? The spec itself is unambiguous (sys.version_info is available in all implementations of Python) but will it work the way existing tools expect? The discussion in the PEP didn’t cover what tools currently do with non-CPython implementations.
Alternative Python implementations emulate a specific Python version, e.g. pypy3.11-v7.3.23 emulates Python 3.11, which tools query through sys.version_info. Unless a tool wants to have implementation-specific behavior, use the implementation name or use implementation version (sys.implementation.version), there is no difference between CPython and other interpreters.
FTR, I closed the issue and the related PRs on the main repository because the workflow for PEPs is to first have the PEP acknowledged at least and discussed. PoCs or reference implementations can be opened in forks or separate repositories since this allow discussions on them directly.
Is the key benefit the version number or is this something where an ABI string (such as the one that would be embedded into extension module filenames) would be more directly useful?
Assuming build_details.json becomes prevalent (which, as an already accepted PEP, we should assume), does that fill the need, even though there’s an extra step?
Do we have an ABI string that we could use cross-interpreter and that tools can parse the version from? I’ve mainly picked major.minor for being simple and familiar.
build_details.json is (iiuc) bundled with (C)Python, but not with the venv, so we can’t use it match venv and interpreter together.
I guess I’m mainly trying to understand what this would actually be used for, and why it’s needed and the existing version number isn’t sufficient (I get that it may legitimately get out of sync, but nobody is forcing you to compare the whole thing… I’m not even sure who’s comparing it to anything or why)
Okay, in terms of PEP writing, it’d be good to have the impact of the problem also expressed in the motivation - the point of the document is to put all the information in one place so that it’s easy to review. Even jumping to an appendix is going to spoil the reading flow, and the motivation is really where you have to be at your most persuasive to convince the reader (who are all highly qualified steering council members/core devs) that you’ve provided all the context they need and that your “problem” is a problem they should care about. Don’t make them work too hard to reach that point
From our perspective, it’s a totally valid objection to say that the field is unspecified/undocumented/unsupported, and the problem is that other tools are trying to imitate it and getting it wrong. CPython as the reference implementation is allowed to say that its implementation is the reference[1], and also that anyone relying on the behaviour is also out of scope. What that means is that your proposal isn’t to fix a problem in CPython, but to actually define the field.
You’ll need to overcome the idea of “what if we just define version as '%d.%d.%d' % sys.version_info[:3] and let all the other implementations adapt” (in other words, declare that CPython was correct all along). I don’t see a reason given in the PEP text why this isn’t a suitable path, so you’ll need to dismiss that one explicitly and thoroughly, I’d suggest.
In order to let 3rd party use[2] dictate how we design CPython, we’d need a compelling rationale for why CPython updates will propagate throughout the ecosystem faster than the 3rd party tools. Generally, this is almost never the case, but it should be addressed in the text.
I’m sure there’s more points, but I need to go right now so I can come back and keep looking for possible gaps later, but I think if you manage to cover those points in the PEP text, you’ll have a really strong case (or you’ll talk yourself out of it - happens more often than you might think )
Personally, I don’t think we should say that about a lot of stuff CPython does, because it’s not a very reference-implementation-like implementation, but in this case I think it’s not horribly offensive to suggest that other implementations should’ve just copied CPython. ↩︎
Which I’ve already mentioned could justifiably be seen as “misuse” ↩︎
Maybe a better approach would be to have a build_details key in pyvenv.cfg, which gave the location of the base interpreter’s build_details.json file?
The point here is that if we can avoid duplicating data, we should do so. And the version information you need (along with a lot more, which tools might find useful) is available in build_details.json.
Everything else is defined by the “implementation is the reference” (says the person who has changed the implementation , but has also independently duplicated it in microvenv).
I’m not sure if this requires a full-blown PEP or if we document the version key in the venv docs as a way to make this an implicit “SHOULD” for other virtual environment implementations to implement in the same way. I’m also fine with saying in the docs that virtual environments should not be changed to point to a different major.minor version in-place and instead are expected to be recreated.
Can the path be worked out from the home key in pyvenv.cfg?
But even if it can, part of the motivation of putting a version key into pyvenv.cfg is to have something out-of-band to tie the virtual environment to the Python version it’s intended for in case someone changes the interpreter from underneath the virtual environment. But personally, I think the “consenting adults” guideline kicks in at that point and people are told “don’t do that”.
The main motivation for the python-version is having a very simple solution to tools disagreeing about version and version_info, leading to bugs (such as the motivating pre-commit-uv example). Its major.minor format is design to capture the Python interpreter that was used to installing the packages in it, so you can determine whether the venv packages are still functional by comparing its value to the version of the underlying Python interpreter. (/usr/bin/python can unfortunately jump versions when doing a distro upgrade, breaking your venvs using it in the process.)
For that to work, all the packaging tools need to agree on the name and the content of the field, so we should get consensus for it and document it on packaging.python.org (with uv, pdm, poetry, hatch, virtualenv and python -m venv being the relevant projects). Unfortunately, there’s no process for making this kind of PR to PPO with just a consensus, and you’ll get reminded that this kind of change requires a PEP. Hence I wrote a packaging PEP, to get consensus (for a roughly 10 - 36 line change per tool) and eventually get a pronouncement by @pf_moore. After merging PEP, I was advised that this PEP needs to go on the standards track since it affects the venv module, which is under CPython governance, so I retargeted the PEP and opened a thread here.
If the solution is to merge a non-PEP PR adding python-version to CPython instead, and then update PPO because it’s now an implementation-defined feature of CPython, that would be even easier from my perspective.
I’m obviously fine with that, but I don’t know what others think (I didn’t even know that PEP 405 had been classified as a packaging PEP to put virtual environments under packaging.python.org).
It records only the major and minor version to be resilient to patch version updates.
If I remember correctly, at least on Windows using virtualenv and the official CPython installer, updating the patch version often (always?) broke our virtual environments. Unfortunately, I do not remember exactly how the breakage manifested itself, but I believe it was not quite obvious so that it became best practice for us to throw away all virtual environments when doing a patch update. Assuming this is not just a cargo cult, it may be useful to also record the patch version in pyvenv.cfg so you can decide to recreate it in case the patch version changed.
I think one problem I remember is that the redirector executable used on Windows changed between minor versions, and that could cause issues. Like you, I don’t remember the details, but it may have been that. What’s most important, though, is that I don’t believe that CPython guarantees that a virtualenv built using Python X.Y.Z will continue to work if the base Python interpreter is updated to X.Y.(Z+1). In practical terms, it works most of the time nowadays, but I don’t think there’s any guarantee.
I don’t think this is particularly relevant to the PEP, though. The python-version file is only a cache of data that can be obtained by running the interpreter itself, so tools that can’t deal with it being inaccurate should query the interpreter. I’d suggest simply deleting the statement you quoted from the PEP.
Maybe the python-version field would be better described as “the version of the interpreter used to create the venv” rather than making any claims about being the version used by the venv. Anyone using the field should be aware of the implications of the distinction, so using the accurate (and hence clearer) description is better.
(That also suggests that maybe copying the build-details.json file from the base interpreter into the venv, as “the build details of the interpreter used to create the venv” would be a more complete solution here, but I’m happy if the PEP authors want to claim YAGNI applies to that…)
This shouldn’t be the case anymore (since about 3.9, I think, or maybe 3.7?).
Prior to that, Windows was making half a copy of the entire install for your virtual environment, but when we switched to the redirector executable it stopped caring about even different major versions. It’s just looking up the home location and launching the python.exe there with the same arguments and a “secret” environment variable to provide the venv path.
Mixing different builds of python3x.dll would’ve been a problem, for sure. The redirector avoids that entirely.
Correct. It works, and it works intentionally, but venvs are meant to be non-relocatable and version-specific, so if we need a release where it breaks, we’ll do it. (Part of the PEP 582 proposal would’ve reduced/removed those limitations, but we all know how that turned out…)
The venv module already sets version, so that would just need to get promoted to the same level as python-version. But I think the other replies have pointed out it isn’t really necessary for the specific case mentioned. But it’s also cheap to record as the full version used to create the virtual environment.
, but has also independently duplicated it in