I think both Petr and Serhiy have raised good points on this front. Quoting the relevant section from PEP 425:
The version is py_version_nodot. CPython gets away with no dot, but if one is needed the underscore _ is used instead.
The rules we adopt for “py” and “cp” aren’t binding on any other implementation prefix, and we’re only required to add the underscore if it’s needed (presumably to avoid ambiguity).
Suppose we adopt Serhiy’s rule:
from Python 4.0 on (whenever that happens), py_version_nodot will zero-pad the minor version number
we decide that we won’t have any more than 100 minor versions of any major version (even with annual releases, we wouldn’t hit Python 3.99 until 2109, and I’d expect the language to go through a Linux style “The syntax and semantics have evolved far enough from 3.0 now that it makes sense to call it 4.0” major version number bump well before then)
What this means is that 310 would still be unambiguous without the underscore, as 31.0 would be emitted as 3100, not 310.
It also means that conversion to integers will work indefinitely, and ordering those integers will give the correct result (since 1.x stopped at15/16, 2.x stopped at 27, 3.x would never go past 399, and 4.x would start at 400).
Right. I understood the plan to be 310, and Setuptools’ change to underscore surprised me.
I don’t really care that much about the wheel tag itself, but about similar tags in other places. If the next Python goes from 39 to 3_10 in some places, 3.10 in others, and 310 in yet others (e.g. pyc tags), I don’t think we can properly decide & document which should be used where. We can’t realistically survey all uses of 39. I’m worried about relying on implementation details, e.g. “whatever happened to use version_nodots now uses underscore; whatever used {major}{minor} directly stays. Except where we happen to change to something else.”. This doesn’t set any expectations about what should or will be changed to something else. I guess CPython can manage, but there are other projects: for example, if Tox uses -e py39 now, what should it use for 3.10?
Going from 39 to 310 everywhere seems simple and straightforward in comparison.
But either way this goes, it would be great to know what to expect ASAP, so that the alphas can be tested and third-party projects know what to do in their own tags. (And don’t tell me third-party libs are out of scope. If we don’t provide at least a bit of guidance, we’ll just get uncertainty across the ecosystem.)
@pablogsal, as the PEP delegate, what are your thoughts on this? What are we missing in the discussion?
I guess I should expand on that more.
Going from 39 to 310and then to 400 seems straightforward, in the sense that anyone who finds a 40 tag should know what to do.
I don’t mind individual occurences of 3xx (like the wheel tag) changing from 3_xx or 3.xx. But version_nodots is not an individual occurence: it is widely used in and outside CPython (for various constrants, sometimes even “plain [0-9]+ integer”).
The PEP is purposed to solve the problem which will occur only in Python 31.0 (because 1.10 and 2.10 did not exist). We have a plenty of time.
I propose two alternatives:
Always use two digits for minor version number. First affected version is 4.0 (as “400”). Additional advantage – versions are always correctly sorted by the numerical value: int('310') < int('400').
Use the same number of digits for major and minor version numbers. First affected version is 10.0 (as “1000”).
In any case we does not need to change anything right now.
As a PEP delegate, I am following close the discussion but I am trying to not influence it in any way so I can be as much objective as possible (I already said my initial thoughts on our first meeting in which I mention that my priorities would be unblocking the wheel situation for 3.10 as soon as possible and minimizing breakage as much as possible, but many new considerations have been raised since).
My biggest fear is that we are not considering something important that will only be revealed when 3rd party projects are subjected to our decision.
Also @storchaka raises some important points and alternatives that I would like to gather feedback on:
For now, as I understand from @hroncok analysis the breakage can be annoying to some projects but it will not be something fundamental that will involve a considerable cost. Given that this is a new release we are talking about, I find that those changes are not a deal-breaker.
I would also like to get some feedback here from the folks that initially requested this from Pypi.
Also, @bernatgabor, do you think this is something that will affect tox or its users deeply?
I mean, Brett already said that packaging went for 3_10 because he preferred the aesthetics, but if we’re finding it unworkably inconsistent across the board then we can tell Brett to deal with it (via rejecting the PEP)
tox and virtualenv already use only the first digit to denote the major version and is widely deployed so it’s unlikely are going to change. Granted once we release python 10 we’ll run into problems with this strategy but I’m not expecting this to be anytime soon. How the stdlib paths are handled should be less of a problem, as both projects use sysconfig to acquire them. Some plugins might hardcode them… but it will be up to them to fix it.
I have reviewed the PEP several times and I here are my thoughts:
Many of the discussions regarding the format are based on the current status quo. This is of course an very important point to consider, but I am also considering the format itself as if this were the first time we are thinking about it. Of course mainly of the considerations are regarding breakage and backwards compatibility.
Seems that the bad usage related to %if %{python3_version_nodots} < ... are restricted to EPEL (in the outer wilds I have seen it only on SUSE). It seems that correcting these is acceptable as it is not widespread (please, correct me @hroncok@encukou). In other build systems, I have seen a considerable amount of: python_version_nodots=$(echo $PYTHON_VERSION | tr -d '.'), For what I understand for rest of the usage of python3_version_nodots that is not directly comparing we don’t expect any important breakage, just a name change.
There are some interesting points regarding integer comparison, but I am not sure that we should support direct comparison of tags based on integer comparison alone (we should not intentionally break it, but losing this property should not be a show stopper IMHO). Additionally, as stated in the PEP, the usage of the _ has the advantage that int("3_10") and int("310") will preserve comparisons inside Python (which I know is not the only place where these comparisons are made). In any case, as far as I understand, usage of python3_version_nodots is not critical enough to really weight that much on preserving integer comparison (which as stated in other places, is quite a dangerous way to compare tags anyway). If someone really thinks this property is really critical then the PEP should be updated to address this problem explicitly (or to counterargument the concern).
I think this is the most important point that I think the PEP currently doesn’t address. The most fundamental aspect that I don’t feel comfortable is the potential proliferation of all formats. It is true that every tool can decide how to adapt to this but certainly as @encukou and @pf_moore indicated I think that for the PEP to be accepted there should be some recommendations for 3rd part projects regarding the naming (and possible “migration”). We are doing this PEP precisely so this change does not pass silently, so we should also explicitly address this, I think.
Does anyone miss any important point that is not reflected here?
I was referring more if tox can addapt nicely to this regarding the version of Python to use. In other words, any of these changes are expected to affect how users refer to environments currently in tox (such as py27, py39, …).
Today tox supports only py310. However, adding support for py3_10 is not too hard. The change actually needs to happen partially in virtualenv - see virtualenv/src/virtualenv/discovery/py_spec.py at main · pypa/virtualenv · GitHub, but we can just accept there optionally a _; so would not be too hard to implement. For what’s worth virtualenv already accepts explicit version constraints, and for that uses the . character instead of _.
> TOXENV=py%{python3_version_nodots} tox
Technically breaks, but here you already relied on that tox follows python3_version_nodots syntax, which is not true (and if someone actually used it would not be too hard to add support for it, for what it’s worth). Also, I’ve never seen anyone use it like that until today.
Is there a way to get a timeline for a decision? The weeks are going by and it is currently impossible to install packaged c-extension modules as wheels on 3.10 until this is resolved.
Matti
If the argument for disambiguation doesn’t apply to all formats, it doesn’t apply to wheel tags either.
If we can’t apply it consistently, we shouldn’t apply it at all (i.e. reject this PEP, modify packaging).
We can completely and safely redesign the tag scheme at any major release between Python 4 and Python 31, which sounds like it should give us plenty of time.
If the point of the PEP is advertising, then we ought to have one to advertise any other changes. I’m not volunteering to write it, so unless someone steps up, assume there won’t be one and decide on this accordingly.
I personally only care about .whl files, so I have no comment on other file names (e.g. I really don’t care about .pyc files since those are an implementation detail and I personally never look in my __pycache__ directory).
A decision is necessary since the fact that pypa/packaing chose to implement this idea before python did causes a situation where projects cannot package wheels for the 3.10 alpha. The clock is ticking. If pypa/packaging needs to revert the change, it will take time to percolate out to new releases of pip and wheel. Is there still any information missing to make a decision?
Pip has their next release in January and I suspect we could ask @pf_moore and @pradyunsg and others to hold off on a pip release if it requires updating ‘packaging’. So at least in that specific case I don’t think there’s an issue getting the change out.
(via rejecting the PEP)