I agree that the message doesn’t convey any urgency–although this is usually true, because pip is very stable and old versions typically work just fine. If the message was always screaming at me I’d probably learn to ignore it.
Revisiting one idea mentioned earlier–if there was an estimate date for the new format, pip could grow a new warning message along the lines of Warning: you are using pip X.Y.Z which does not support wheel 2.0, expected by {wheel2_date} that was only displayed when that date was coming up soon (like if (wheel2_date - today()) < 6 months: display_warning()). And this warning could get more granular as the date arrived (i.e. more recent versions might not warn about this until the date was closer, and they might have a better estimate of the date).
If we only raise the message when we see a 2.0 wheel, and if we have an agreement that PyPI will not accept 2.0 wheels until pip, uv, and any other major wheel installers have support, then we can add a statement now, knowing that if that statement is ever printed in the future, then it will definitely not be a lie (assuming the user isn’t pointing to some other weird index) - the presence of a wheel 2.0 on PyPI indicates that there must exist a newer pip release with wheel 2.0 support. (This is what I was trying to get at in pip#12723.)
I think we can determine in the course of PEP 777 when exactly wheel 2.0 is ready for PyPI and just how much support needs to be implemented in pip/uv/etc., but we don’t need to determine that in order to implement the error message now. Also, if we end up deciding not to do a wheel 2.0 at all, then the error will never be printed and so we won’t be lying to users.
Oh, one thing just occurred to me - what if we make the new wheel format use a different filename like .whl2?
My understanding is that existing versions of pip/uv/etc. will just silently ignore an extension they don’t recognize in an index, and they’ll look for a .whl or an sdist instead. So this is transparently backwards-compatible. (We should double-check that the behavior of current pip/uv/etc. when a .whl is missing is what we want, though. And the error message from pip install something.whl2 needs improvement.)
This also gives us a lot of flexibility about optional improvements: you can ship a deflate-compressed .whl and a zstd-compressed .whl2, and users with an installer that supports .whl2 will get a smaller download, and users without will still get a working wheel. Or you can ship a library with copies as a .whl and a library with symlinks as a .whl2, or you can ship a library that doesn’t support being used as a build dependnecy as a .whl and one that does as a .whl2.
This doesn’t help with the problem of storage consumption on PyPI, though, only with bandwidth. But at some point you can decide to stop uploading .whl files and only ship .whl2 files.
This approach lets us completely change the .whl2 file format, and it’s also clearer what’s going on in the offline case; you can type “whl2” in your search engine and find something (and you can probably even guess from first principles that you need an upgrade).
There are some nice upsides to this, as you point out. But the price will be paid by package authors, who will have to upload both .whl and .whl2 for a long time, to avoid a fallback to building from sdist. That price is way too high - I’d prefer to see an error when pip is too old.
There’s another trick here though! An installer that supports .whl2 will always ignore a .whl of the same version. So if you are fine with an error message when pip is too old (instead of a successful install of a 1.0 wheel), you can upload a .whl that contains Wheel-Format: 2.0, and that will trigger an error message in current versions of pip, while not affecting any versions of pip that support .whl2. You don’t have to upload a usable 1.0 wheel.
Uploading tools can maybe even do this automatically; the dummy 2.0 wheel is a small constant. Or maybe Warehouse can inject it into indexes if there’s a .whl2 but no .whl.
If we’re talking about 2.0 formats anyway, I think we should make an attempt to solve this problem instead of starting with it as a fixed constraint; it’s a problem worth solving for plenty of other reasons. Preventing unwanted attempts to build sdists is my proposal, using the same trick of coming up with a new file extension that existing installers ignore, but there are other proposals too.
FWIW this is the migration plan I am expecting. My expectation is build backends can add it behind a flag, installers should add support as soon as possible, and PyPi will only allow uploading new wheels X years after major installers implement it, or based on data on pip/uv version from incoming requests.
I think this has many nice features, such as different wheel versions living side-by-side. It also would allow for some re-thinking of the wheel filename spec if we wanted to make it easier to extend. However, installers are currently not well set up to handle this. They currently will ignore any wheel filename that doesn’t match the pattern they expect, or error with it “not being a supported wheel.” Also when talking to Ee about zstd compression in wheels they used the name “zwhl” which I think sounds nice, but doesn’t communicate the version.
I think this has bad UX though, the error pip gives today doesn’t give a clear indication of what actually went wrong (obviously this could be improved).
You could alter the tags to indicate the new features, so that older installers would say “that tag is not in the list” and ignore the wheel. For example we used to have a cp32dmu ABI tag which means cpython 3.2 + debug + math + unicode. If ‘lz’ meant ‘symlinks and zstd inner archive’ then a py3-nonelz-any wheel (bikeshed) could contain those features.
A stored (in ZIP) + zstd compressed inner archive plus *.dist-info the same as before in the outer ZIP file would work. The inner archive could be “in addition to” the existing *.dist-info/ for “python -m somefile.whl” purposes. A feature to avoid installing an arbitrary set of files like __main__.py that might be in the root of a .whl to improve standalone usability.
Just confirming that uv does indeed behave this way today, with one exception: uv will bail if the major version is greater than 1, but it only warns if the minor version is greater than 0.
Backtracking here would of course be possible but non-trivial since (as you mention later) Wheel-Version isn’t part of METADATA, it’s part of WHEEL, and we don’t even try to get that information until we install the wheel itself (i.e., resolution never has access to it).
I think in uv today we actually assume that any extension beyond .whl is a source distribution (and then we error later on if we don’t know how to unzip it), but we could change that assumption
As a uv developer, i agree with @pf_moore’s great write-up. Adding the wheel version to the index api in some form is the only option i see to add a new wheel version without major user disruption. This would be the same mechanism as exposing requires-python in the api prevents failing source dist builds or as old pip/uv versions will ignore platform tags they don’t know. The two main options for adding this to the index api i see are adding a field such as wheel-version or choosing a new suffix for the new wheels version (say .whl2), since clients already ignore unknown file extensions due to legacy distributions on pypi.
I support adding the wheel version (or even the entire WHEEL file) to the index api indepedent of the proposed wheel format changes, exposing this value is a cheap way of future-proofing packaging.
Changing the major version of the wheel format is a disruptive change and it should have clear use cases from specific packages that cannot be solved without a major version or large ecosystem benefits (switching to zstd would be massive improvement for performance, storage and pypi traffic with minimal implementation changes).
Another question is the interaction with PEP 517: What happens if a frontend does a source distribution build and gets a wheel 2.0 but it only supports installing wheel 1.x? Or assume i have a frozen requirements.txt from a pre-wheel 2.0 time, some entries of which are only available as source distributions. I try to install it with a wheel 1.x only tool, but suddenly some of the wheels source dists build wheel 2.0 archives because the wheel package got updated.
I think that is too much of an abuse of wheel tags as it changes their definition. Right now they define what environment is required to install something, while doing this would expand that to the tool required and the environment. Plus we don’t know how long the list of installer features could get, potentially making the file name unreadable.
I also think this would be the easiest way to make it cheap for installers to know if they can install a file.
An installer that supports