That’s only for python 3.10. Across all consumers it’s <1.5%. Leaving out python<3.8, it’s still well below 10%.
The good thing is that there’s less to do these days. It’s comes down to making a choice of RHEL clone (small differences, nothing major[1]), and this time we don’t have the end of CentOS-as-we-knew-it to redigest. Also pip etc. won’t have to be updated anymore (thanks to PEP 600) for new wheels.
You can open an issue on the manylinux tracker of course, but IMO you’re still way too early on this.
any of rhubi, alma, rocky or centos stream would work in principle ↩︎
I may have spoken too soon on this… RHEL recently announced that they will not make their sources available anymore (outside of CentOS Stream), which basically breaks the model of Alma, Rocky, etc. There’s an ongoing elaboration if / how Alma can continue. If it can’t we might have to switch everything over to rhubi (which at least now has the devtoolset backports that ruled it out as a candidate at the time).
As a small update here, with the help of @mayeut, manylinux-timeline now has a plot that shows the distribution of glibc-versions only for those downloads that are on Python versions that aren’t yet EOL. And since Python 3.7 became EOL per the end of this month, we can now update these numbers.
In short, 90% of Python ≥ 3.8 users now have better than glibc 2.28, and ~15% have better than glibc 2.34 – the latter still quite far from indicating ecosystem readiness for a potential manylinux_2_34.
That’s an average across many projects of course, and it could be conceivable that specific cutting edge projects might want to start using newer glibc features sooner than the overall average rises above a given threshold. But that would then be on the project to show that its user base has broad support for 2.34.
As a concrete example, I recently stumbled across some download numbers for pillow (that @hugovk thankfully collected), and the distribution is as follows:
glibc python
2_28
% (within 3.x)
2_17 == 2014
% (within 3.x)
Total
%
3.11
227,035
97.58%
5,638
2.42%
232,673
5.89%
3.10
802,130
94.37%
47,878
5.63%
850,008
21.53%
3.9
926,003
86.52%
144,232
13.48%
1,070,235
27.11%
3.8
931,824
51.90%
863,617
48.10%
1,795,441
45.47%
Total
2,886,992
73.12%
1,061,365
26.88%
3,948,357
100.00%
I found this interesting because it illustrates the strong skew across python versions, and how prevalent glibc < 2.28 still is.
Should the discussion regarding any future manylinux (say, manylinux_2_34) remain here or move to a new thread / issue? As this one has was basically a thread about manylinux_2_28.
I’d say it’s probably better to start new threads rather than repeatedly dredging this one up and making it longer.
But the work done last time round should also mean that we don’t need a long discussion and a new specification to move manylinux forwards. Auditwheel can already label wheels as manylinux_2_34 & 2_35 since over a year ago, and there’s a PR open to add standard images to make building 2_34 wheels convenient.
I’m probably missing an obvious answer, but how can people keep updated on the newest manylinux versions and their characteristics? Is there some very low-traffic mailing-list that one can subscribe to, or similar?
Also, on which criteria is it decided to add newer manylinux versions?
Auditwheel seems to have profiles based on the glibc versions available in CentOS, Debian & Ubuntu LTS. So if you build on one of those distros and then run auditwheel, you should be able to make a manylinux-compatible wheel. The manylinux container images, which are convenient but not required to make manylinux wheels, are mostly based on Red Hat derivatives, apart from the 2_24 image based on Debian.
Ok, so to clarify, any base library profile in one of CentOS, Debian (I assume stable?) and Ubuntu LTS can implicitly become a new manylinux profile without going through any particular decision process. Is that right?
My next question would be: are there any recommandations for package maintainers? Which manylinux version(s) should binary wheels target? Is it sufficient to target an old manylinux version such that most users can install it, or should maintainers also produce wheels for more recent manylinux versions?
That’s more or less my understanding. It needs a ‘profile’ in auditwheel to verify the wheels and apply the tag, but that’s just up to the maintainers of auditwheel (and I think you could conceivably fork auditwheel if necessary, but hopefully it’s not necessary).
I’d start by building with the oldest version of the manylinux Docker images you can comfortably use (i.e. without too much effort to work around limitations of old systems). As of early 2024, manylinux_2_28 (CentOS 8) will work for most users, but not quite all. manylinux2014 (CentOS 7) is a conservative choice.
If you’re building something that’s likely to be performance-sensitive, measure whether these broadly compatible builds have a performance penalty relative to builds on a newer platform. If so, you might want to make wheels on a newer manylinux version, either as well as the more compatible ones, or instead of. If there are multiple manylinux wheels uploaded, I believe pip should choose the one with the highest manylinux version.
My understanding is that there’s some decision making that happens on the auditwheel issue tracker, which is where the policy files live and that is basically the contract for what a manylinux tag means.
Someone more closely involved with auditwheel might have more context though.
