Tag ordering is currently not spec-defined, beyond a guideline that more specific wheels should be preferred. This has not been a big problem, usually there is only one compatible wheel and where there is not, ordering is clear and/or insignificant.
With wheel variants, there can be several compatible wheels per project. We define a total order to give package authors and users control over wheel preference and to ensure that all tools pick the same wheel without ambiguity.
As intuition, we want to choose the wheel that has the best platform support property, with tie breaker fallbacks to remove ambiguity. To do so, we define a simple three-level hierarchical ordering between properties: We first order by highest-ranked namespace. If that doesn’t suffice, we order inside the namespace by its highest-ranked feature, and then inside the feature its highest-ranked value. We then compare wheels by their best property: If one wheel’s best property is better than the best property of another wheel, it wins. As a tie-breaker, for two wheels that both have the highest-ranked namespace-feature-value property tuple in the set, we look at lower-ranked tuples too. A precise algorithm sorting all wheels of a release is given below [see current spec text].
The ranking for namespaces, their features and properties comes from the package and the providers together. Overrides from the user are possible through tool-specific configuration.
As examples, if there is a compatible wheel with gpu :: cuda :: 12 and one with cpu :: x64_64 :: sse4, and we have given gpu > cpu, the gpu wheel wins. If one wheel has gpu :: cuda :: 13 and the other has gpu :: cuda :: 12, and we have given gpu :: cuda :: 13 > gpu :: cuda :: 12 that wheel wins. If in the same situation both wheels have gpu :: cuda :: 13, but one has cpu :: x64_64 :: sse4 and the other has cpu :: x64_64 :: sse2, and we prefer newer SSE extensions, the SSE4 wheel wins.
I think that seems OK. I’d love for other participants in this thread to comment, as I’m feeling like I’m too close to the problem at this point to really look at it as an “average reader” @ncoghlan what do you think?
One minor typo:
for two wheels that both have the highest-ranked namespace-feature-value property tuple in the set
The suggested ordering specification above seems fine, but feels a bit verbose. That’s not a big issue, as it becomes more accessible and reduces chance of ambiguity.
I interpret it as:
Order variant wheels:
by their namespace
then by feature in each namespace
then by value in each feature
Ties are resolved by looking at the next highest triplet
This is similar to Python tuple ordering. The package and providers define the default ranking
The current PEP says to only take the highest ranking value for each feature, but the above doesn’t indicate that. Also, the above is not explicit in taking only compatible (with the current environment) features.
I agree. There’s a really difficult trade-off here between making the algorithm intuitive, and making it precise (given that I’d rather not have the example Python code be the normative form). I tend to focus more on the “intuitive” part, maybe too much given that precision is crucial for a standard…
My understanding is that your description is missing a level (or, with hindsight, that you glossed over a level). Wheels don’t have a single /namespace/feature/value, so the ordering is actually:
For each wheel pick the highest priority namespace/feature/value. Order wheels by comparing these triples (again, by the priority lists given). On ties, look at the next highest priority within the wheel.
Or to put it another way, consider each wheel to be a tuple of namespace/feature/value triples, with the triples ordered according to the namespace/feature/value priority lists determined earlier. Order wheels using lexicographic ordering of their tuple-of-triples.
OK, thanks. Your description, along with my 2 attempts at rewording your rewording of the explanation by @konsti, has fixed my understanding of the process.
@konsti, if there’s any way you can see to include the “tuple of triples, ordered lexicographically” picture into your explanation, I think it would help.
We’re aiming to do both: first an intuitive explanation (which I agree makes the spec much easier to understand) and then a subsection with the precise version (needed, since it’s easy to get wrong and we really want identical behavior across implementing tools).
We’re trying, but this exact wording may be misleading, since lexicographically means the shorter of two tuples comes first, while we defined best as “more specific” → longest first. Example:
Then the PEP 825 order (best first) is ['cu130_v4', 'cu130'], while lexicographic order would be the opposite. It may be good to call that out explicitly in the text.
We’ve posted another update, integrating the metadata consistency appendix and (hopefully) making the sorting algorithm clearer. We’ve decided to include a “high-level description” followed by the precise algorithm.
Thank you. The new high-level overview of the ordering algorithm is excellent - it covers what is happening exactly the way I’d hoped it would. And the metadata consistency section, and the associated appendix, are equally good - apart from being a great document in support of PEP 825, it will be an incredibly useful resource for future discussions around metadata consistency in general.
At this point, I consider the PEP ready for approval. As a pure formality, could the PEP authors please request a decision on the PEP, and I will post my official response.
I would like to formally declare PEP 825 as provisionally accepted. The “provisional” aspect of the acceptance here is an acknowledgement of the fact that this PEP is part of an extended series of PEPs covering the whole wheel variant feature. Once the full suite of PEPs[1] has been accepted, this acceptance will become final. While the PEP is in provisional status, changes are permitted where necessary to adapt to any issues encountered during implementation, or requirements imposed by future PEPs (such changes will require approval, but unless there’s a major interoperability implication, this should be a formality).
Congratulations, and many thanks, to the PEP authors for their work on this proposal
I’d like to express my personal thanks to the PEP authors for their patience and flexibility in addressing the feedback that came up as part of the process. I know it’s easy to get frustrated and discouraged when faced with negative feedback, and your willingness to adapt and respond positively to such feedback is what got us the successful outcome. I genuinely believe that the resulting PEP is enormously better for that.
It’s taken quite a while to get to this point - not least because of my limited availability[2]. Thanks to everyone for your patience and support.
Here’s to the next PEPs in the variant series - and my best wishes to the new Packaging Council when they work on them
My expectation is that the revised PEP 817 will define precisely what that “full suite” will consist of. ↩︎
thanks to some unavoidable personal issues I’ve had to deal with ↩︎
Thank you Paul for your effort and openness to understand the PEP. The suggestions and rewrites improved the PEP and its clarity. Thanks to all for moving this to provisional approval since it is wonderful news for the future.
Thank you for sticking with us through this long and complicated process with all its revisions and changes, and thanks to all the contributors, reviewers and everyone else who made this possible! I’m excited for the future of hardware support in Python packaging.
