WheelNext & Wheel Variants: An update, and a request for feedback!

Packaging
1

We, the WheelNext group and contributors, are writing to share an update on our open-source effort to address some long-standing challenges within the Python packaging ecosystem, particularly the distribution of packages with hardware-specific or platform-dependent builds.

For years, the community has navigated the complexities of installing scientific computing, AI/ML, and high-performance packages. The familiar process of directing users to a web page to construct a specific pip install command with a special index URL for libraries like PyTorch or JAX highlights a gap in the current packaging standards.

Screenshot 2025-08-14 at 11.52.41
Screenshot 2025-08-14 at 11.52.41943Ă—317 28.2 KB

While wheel tags have served the ecosystem well, they lack the expressiveness for the growing heterogeneity of modern hardware, from GPUs and their corresponding driver versions (CUDA, ROCm) to specific CPU instruction sets (AVX512) and library backends (MKL, OpenBLAS).

To address this issue in a standardized and open way, the WheelNext open-source initiative and project was founded. Our goal is to collaborate with the community to evolve the Python packaging standards, better serving these important use cases across various Python communities.

Introducing… Wheel Variants

Today, we are pleased to announce a first full-scale experimental release following a soon-to-be-published proposal called “Wheel Variants”. This is a proposed extension to the wheel specification designed to make platform-specific package distribution easier for package maintainers, and installation more smooth for end users. The key concepts are:

  • A Provider Plugin system, allowing vendors and communities to ship small, install-time packages that can detect system capabilities (e.g., CUDA version, CPU features).

  • A declarative metadata format for wheels to express their specific requirements (e.g., nvidia :: cuda_version_lower_bound :: 12.8). Yes, absolutely inspired by Python classifier format.

  • A mechanism for installers to automatically run these plugins to match the user’s system with the best-fitting variant wheel, simplifying the user experience to a familiar [uv] pip install torch.

We have carefully designed this system to ensure that installers unaware of variants will safely ignore them, maintaining backward compatibility with the existing ecosystem. We aim to complete the first version of the PEP by the end of October, taking into consideration the feedback from our “real-world experiment.” This will cover the overall design, the impact on end users and package maintainers, and the necessary changes to install tools and package indexes.

To move from theory to practice and gather crucial real-world data before a formal PEP submission, we are happy to announce an important milestone. In collaboration with the PyTorch team, we are testing our design with the experimental release of PyTorch 2.8.0 as a wheel variant. Paired with an experimental build of the uv installer, this allows us to stress-test the end-to-end packaging and installation experience, from build and publication to discovery and installation, on one of the ecosystem’s most popular and complex packages.

We are committed to a transparent, community-driven process. This experimental release is a crucial step in gathering the necessary feedback to refine and strengthen our proposal. We aim to present a robust, well-vetted PEP to the community that builds on these real-world findings.

We are eager to hear (here on DPO, on Github, or PyPA Discord) from anyone who tries out this experimental release, and in particular from active members of the Python packaging community if they have any questions, concerns or new ideas.

Try it Today !

Linux / MacOS

curl -LsSf https://astral.sh/uv/install.sh | INSTALLER_DOWNLOAD_URL=https://wheelnext.astral.sh sh
uv venv
uv pip install torch

Windows

powershell -c { $env:INSTALLER_DOWNLOAD_URL = 'https://wheelnext.astral.sh'; irm https://astral.sh/uv/install.ps1 | iex }
uv venv
uv pip install torch

Automatically, uv pip install torch will determine which variant is the best fit based on a variant-plugin based analysis of the system, including the availability of NVIDIA GPU, NVIDIA CUDA Driver and NVIDIA CUDA Compute Capability.

If no NVIDIA GPU is present on the machine, the CPU-only build will be installed.

Engineering Note

For testing purpose, it is possible to influence the NVIDIA variant plugin resolution using:

export NV_VARIANT_PROVIDER_FORCE_CUDA_DRIVER_VERSION="12.8"  # Any value >= 12.0
export NV_VARIANT_PROVIDER_FORCE_SM_ARCH="9.0"  # Any value >= 5.0,<13.0

Blog Posts - Four different perspectives

We would like to share four blog posts (published yesterday-today) that explore the concept of wheel variants from different perspectives. These posts provide useful background and technical details on the topic. Reading all four will give you a comprehensive understanding of the various approaches and implementations of wheel variants. We recommend checking them out to get a well-rounded view.

We would also like to welcome you to watch the Webinar organized by the PyTorch Foundation: Register to attend - Today (Thursday 14th 2025, 10am Pacific). A replay is usually published on YouTube for those who missed the livestream. I will update my post with the link when published.

In case you missed it, you can also watch of PyCon 2025 talk available on YouTube: Reinventing the Wheel: A Community-Driven Roadmap for Python Packaging

Thank you for your help !

Thank you for your help in testing and reviewing our work. We look forward to the exciting conversations that will follow.

Jonathan (on behalf of the WheelNext contributors)

6 Likes
2

FYI, I know this concern has been well discussed in other forums, but it still feels like a sticking point for me.

The opt-out design means that users who have the expectation (rightly or wrongly) that installing, or downloading, a wheel doesn’t run arbitrary Python code, will have that expectation violated.

I don’t mean to start a big discussion now, but when the PEP is published I am likely to advocate that installers either SHOULD be opt-out opt-in forever and have a global flag like --allow-variant, or be opt-out opt-in initially with a deprecation period warning the users that they must provide a flag like --disallow-variantif they don’t want to run arbitrary Python code when installing or downloading wheels.

Edit: Sorry I made a typo, hopefully it should be obvious, but I have corrected the above “out-out” for “out-in”.

7 Likes
3

That wheel variants do this is very surprising to me, I would have expected to see this noted far more prominently in the overview of the proposal.

A

3 Likes
4

First thanks for the feedback @notatallshaw.

We are keenly aware of that discussion and ultimately it might not be a decision taken by the PEP but rather the tool makers (uv, pip, poetry, pdm, etc.)

That expectation is already untrue today … All installers are opt-out today for building sdistsand really ultimately it’s no different between –-only-binary and --disallow-variant (or similar). If you build an sdist - you have remote code execution.

I would say though that I believe the majority of users would want “default ON” not “OFF” and users who care can always deactivate.

That being said - we are actively working at a proposal which includes a path for completely “static” resolution (no install time execution at all). We are finalizing that part of the proposal - it will most certainly look like a flat file (txt or toml) that can serve as a “static analysis” of the machine.

And a mechanism of variant pinning (like version ==A.B.C but for variants) will also be included in the PEP.

I hope these ideas will help to address these concerns.

5

A huge, advertised benefit of the wheel format is that there is no code executed. Yes, if you need to build the wheel, there is code executed, but once you have the wheel you know it is statically installable.

7 Likes
6

As you say, users can run --only-binary :all: and expect not to run arbitrary code, but there are other cases you are missing where they do not expect arbitrary code to run, e.g.:

  • pip download
  • Proxy PyPI and filter out all sdists
  • Build sdists in a sandbox and provide internal users a wheel only index and expect those users are not running arbitrary code

There are probably many other cases where it is not true to say they are “completely wrong” to have that expectation.

6 Likes
7

I think this misrepresents Damian’s point, which said “installing … a wheel doesn’t run arbitrary Python code”. Given Wheel Variants are being advertised as just another kind of wheel, this does seem to represent a large change in semantics and behaviour.

A

3 Likes
8

@davidism but that’s sort of my point no ?

There is no system that guarantee you today to only use wheels. Unless you pass a flag to block sdists. I struggle to see how different this is from --no-variant.

If having sdists is even possible to install by default - I don’t understand why it’s a different situation with variants.

As you say, users can run --only-binary :all: and expect not to run arbitrary code, but there are other cases you are missing where they do not expect arbitrary code to run, e.g.:

  • pip download
  • Proxy PyPI and filter out all sdists
  • Build sdists in a sandbox and provide internal users a wheel only index and expect those users are not running arbitrary code

So if “having flag” is an acceptable answer - that’s easy.

For the “proxied systems” - well nothing prevents you to also proxy these “plugins” (and do a security audit on them) - in facts if it’s well made you should not be able to access at all pypi.org or any other index (beside your private index). And if the plugin is not found - can’t execute.

9

Am I just being dopey or does this not work? No matter what I do, I always get the same wheel.

#~ $ NV_VARIANT_PROVIDER_FORCE_CUDA_DRIVER_VERSION="12.8" NV_VARIANT_PROVIDER_FORCE_SM_ARCH="9.0" uv pip install -v torch
...
DEBUG Using variant wheel torch-2.8.0-cp312-cp312-manylinux_2_28_aarch64-00000000.whl

Edit: Discussed via private message, I’m getting that variant because there is no pytorch wheel for aarch64 + CUDA < 12.9. If I use NV_VARIANT_PROVIDER_FORCE_CUDA_DRIVER_VERSION=12.9 then it works as expected.

I’m not really concerned about the literal answer to the above. My real concern is that, with my end-user packaging hat on, people are going to be building applications from wheels that they pip download/install-ed on a build machine (probably a CI machine with no GPU) then copying the resultant application onto another machine (most likely with a different GPU setup). As long as the override to break the install platform == runtime platform coupling works, is available, is seen as a reasonably 1st class feature and isn’t to hard to find or use [1] then I’m not too worried about this.

  1. i.e. not like the deployment target _PYTHON_HOST_PLATFORM=macosx-11.0-arm64 override is for macOS ↩︎

2 Likes
10

Seriously, why not just give the packages different names? Does it really hurt that much to pip install torch_cu128 if you want the build for that variant? And it works today, even with all the concerns being raised in this thread.

The “magic” ought to come from some arbitrary code in the installer (or in the package itself) that knows that when you pip install torch that it should check your system configuration and choose the real one for your system. This is pre-resolver, which means it can be calculated independently of any other dependencies and won’t have to be retried, but it must be arbitrary code. There’s no way to get around that.

This proposal could be very simple. It already looks like it won’t be.

3 Likes
11

One last post for my 2 cents on this specific topic (opt-in vs. opt-out of variants), as I said I didn’t want to get into a long discussion, but it’s probably better having these concerns raised now rather than when the PEP is published and therefore it hopefully doesn’t derail the PEP.

The PEP shouldn’t mandate UX for installer tools but it should recommend default behavior as SHOULD for how this feature works, otherwise we get into a bad situation where different tools produce completely different results.

There are many systems that allow you to avoid build sdist, what I think you meant to say is something like “there is no common installer that defaults to not building sdists”.

While true, users have built up an understanding around this and implemented systems to avoid end users building sdists.

With at least a deprecation period from opt-in to opt-out, I would be uneasy with users upgrading to the next version of pip and their expectations of how running arbitrary code works is broken with no warning.

4 Likes
12

There is a lot to read through in the proposal, but I have some initial thoughts:

  • I think variant support needs to be opt-in, not opt-out. Variants have negative consequences on multiple concerns from reproducability[1] to security, and changes multiple places where people could previously assume no remote code was executed.

  • There needs to be a closer look at security concerns here. Nothing in the draft indicates to me that I as a user can be sure the variant selecting code can’t select something that didnt exist on a remote server previously. The knowlege of available variants needs to exist in the distribution information and be enforced by installers. Lock file support being an “open issue” makes it basically impossible for me to do anything but say “as presented, I would be unable to use or interact with this at work, and would refuse to interact with it in personal projects”

  1. Wheel variants are less portable across machines. ↩︎

2 Likes
13

To clarify the language a little… I think there’s some missing nuance between installing a wheel and selecting which wheel to install. In this proposal, the former still doesn’t require execution of third-party code, while the latter does by default.

I think Jonathan has a fair point: that third-party code is already executed to build sdists without some opt-in. I think that’s a reasonable default, because the alternative is to fail. With wheel variants, it’s a little more complicated, because the alternative is to pick a non-optimal wheel — which is better than failure but creates subtle mismatches in user expectations. The motivation for using variants by default is to avoid that user confusion, especially for beginners — the category of users that cares about avoiding third-party code execution is generally more knowledgable. I think it’s important to have an easy way to avoid third-party code execution during wheel selection, and I think the static variant metadata proposal is a reasonable way to power that.

I’m not entirely sure why this needs to be debated in the specification itself though. Allowing sdists by default is a tool-specific choice, no? Can’t each tool decide whether they want variants on by default?

I feel like this would completely break package resolution? While I think there’s an interesting angle to explore regarding distinct packages for each variant, I think this suggestion an oversimplification of the problem here. Maybe it’s worth considering why we’re not using such a system for the existing platform tags?

2 Likes
14

I would push back against this sentiment, I think we should have a responsibility to promote good security practices (avoiding arbitrary code execution on installation), especially for beginners and those that are less knowledgeable.

I would expect to see a detailed Rejected Ideas section in the PEP for why something along the lines of Steve’s proposal, or more generally based on static analysis, is unworkable.

A

5 Likes
15

Unfortunately, things aren’t that black and white.

Yes, we need to find a solution that works for all users. However, that also means that the solution must work for an average user. Now, what we really want to achieve here is that an average user can run pip install foo and get a fully functional package. The problem with an opt-in approach is that it immediately requires the user to actually know up front that they need to enable a specific flag. In my opinion, this disqualifies such a solution.

I think a better solution would be a TOFU approach: if running on interactive terminal, ask the user on first install whether to use the plugin or not. This would probably imply opt-in for non-interactive installs, but it will still be a big gain for the average user.

16

I’m somewhat concerned that so far this proposal has not been discussed at all on Discourse. While the concern may have been “well discussed in other forums” I don’t want the discussion here to be based on an edited or summarised version of discussions that have happened elsewhere. Packaging standards are based on a consensus-driven process, and when significant parts of a proposal’s design happens outside of this forum, that consensus based process is at risk. If nothing else, there’s going to be an understandable reluctance to entertain significant changes to the proposal because of the amount of effort already invested in what you have.

I agree with @notatallshaw here - this is a significant and IMO serious deviation from the current safety guarantees around installing wheels.

I see that there’s already been significant pushback on this point, and I don’t want to just repeat what others are saying, but I will say that your responses are missing the point. It’s not about what options the user specifies, or about the user interface, it’s very fundamentally that the wheel format itself is by design a format that can be installed without running any code. That’s the most important aspect of the format, and we will not remove that guarantee lightly.

Let me put it another way - if the community consensus was that “no code gets executed when installing a wheel” was a showstopper requirement, how would the proposal need to change to accommodate that? Because I think that’s a real possibility.

10 Likes
17

What is black and white is that we’ve emphasised for years that if you install from a wheel, you’ll be safe from having any arbitrary code execution. We’ve even blocked requested features (like post-install scripts) on the basis of this guarantee.

We can’t just decide to drop that (security!) guarantee because we now have a proposal that would be easier without it. If we’re going to drop it, we need agreement from all stakeholders, especially people working in high-security environments.

4 Likes
18

I won’t argue against secure defaults for users. I strongly agree that making secure behavior the default is the best way to make the ecosystem secure. I’m just explaining the thinking behind the default — I think it’s out of scope for the specification in the end. I would rather focus on the security of the specification than discuss tool behavior.

I don’t think this should change and I don’t think the suggestion is that it will per my above comment

Do you agree with that?

1 Like
19

From the draft PEP:

This specification extends the wheel filename to include an optional variant label. The complete filename follows the following pattern:

{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}(-{variant label})?.whl

Files not featuring the variant label part are regular wheels. Variant wheels use it to uniquely identify each variant. It can either be a 8-character variant hash, or a custom string of 1 to 8 ASCII characters from the range [a-z0-9._].

For example, the following are valid wheel variant names:

mypackage-0.0.1-py3-none-any-fa7c1393.whl
mypackage-0.0.1-cp310-abi3-manylinux_2_28_x86_64-fast.whl
mypackage-0.0.1-3-py3-none-any-fa7c1393.whl

Being able to quickly tell what a wheel is for is a benefit of the current filename schema. I find the hash-based approach unhelpful as it requires looking up some other data source for what an aribrary hash maps to. Can we find a way to encode the variant type in the filename?

A

3 Likes
20

Specifically, existing flags like --only-binary :all: should block variant plugin execution, since the current expectation is that setting those options is sufficient to prevent arbitrary code execution at installation time (beyond the execution of the installer itself).

Where things are less clear is:

  • if building from source is explicitly allowed, should dynamic variant selection also be implicitly allowed?
  • what are reasonable defaults for installation tools? (preferring binaries, while allowing dynamic variant selection, actually seems reasonable to me)
  • if an installer offers it own plugin extension model, how are the variant selection plugin security concerns any different from those of any other plugin for that installer? (my instinct is that the security concerns for variant selection plugins should be pretty comparable to those for any installer plugin mechanism, unless installers start automatically downloading and installing selection plugins just because a project’s package index metadata indicated it could benefit from them)

If folks want to dive into more of the history of the proposal, the best places to look would be the draft PEP repo (but please keep in mind that it IS very much a draft!) and the #wheelnext channel in the PyPA Discord.

2 Likes