Continuing the discussion from The next manylinux specification:
This idea is technically a competitor to the perennial manylinux proposal, since it would aim to provide at least some of the same benefits, but in a different way.
Summarising the core problem that both ideas attempt to tackle:
- one of the delays in rolling out new versions of the manylinux spec is that it requires end users to upgrade to a newer version of
pipin order to consume wheels targeting the newer baseline
pipis still aiming to improve its default behaviour in various areas, a lot of end users and distributors are quite cautious about
pipmajor upgrades, since they may bring new deprecations and changes to default settings
- this means that publishers may delay targeting the newer baseline solely due to their users using the older
pipversion, rather than due to their users actually running older incompatible Linux distros
- it also means that if publishers do switch to targeting the newer baseline, then some of their users may no longer be able to install the published wheel archives
The perennial manylinux concept encodes the tuning parameters for a compatibility checking heuristic directly into the archive filename. This allows a new iteration of an existing heuristic to be published and remain compatible with existing versions of
pip, but the definition and rollout of new heuristics would still require a pip upgrade.
There’s a different way of looking at this though, and that focuses on the question of “Why are folks reluctant to upgrade
pip?”. And the answer to that is because it’s a complex piece of software with a lot of technical debt still to be paid down, so switching to a new version is something where folks will often want to allow for a bit of testing time first.
Looking at the question that way leads to the follow-on question: what if there was a dedicated package on PyPI that just generated an ordered sequence of manylinux tags for a particular target system, and the logic in packaging.tags was updated to use it, rather than embedding the logic directly?
For reference, that logic can be found at https://github.com/pypa/packaging/blob/5ef37d364bfb18461da28fd667e4d538ba67255b/packaging/tags.py#L285, and the key point is that when run on the target system, it generates an ordered sequence of
manylinux platform tags.
pip always uses its vendored copy of
packaging.tags to calculate the linux tags, but it would be possible to amend that to say:
- If the target environment includes a newer copy of
manylinux_install_tags, use that to generate the list;
- Otherwise use the vendored copy
The potential benefit of this approach over encoding heuristic tuning parameters directly in the wheel filename is that it would also allow the introduction of new heuristics (such as ones based on the contents of
/etc/os-release), without requiring a full
pip upgrade (just an upgrade of
If we’re able to get
manylinux_install_tags to be seen as a data file update (akin to folks updating
pytzdata), then the barriers to getting compatible systems updated to accept the new tags should be much lower than they are now.
With this approach in place, then even switching over to the perennial manylinux wheel tagging concept would no longer technically require a pip update - just a newer version of