I’m concerned that there’s no discrimination in the filenames between extensions that are compatible with GIL / FT / both.
That is incompatible with overlapping python library installations as Debian has been doing since Python 3. (Thanks to PEP-3147 + PEP-3149). I don’t think anyone else is relying on these mechanisms much, but we are, and we expect to continue to use them for system packages, as we migrate to free-threading, too.
I commented on the SC issue, but I agree with this.
Treating .abi3.so as the same thing as abi3t is breaking the contract of abi3, and as far as I can tell entirely unneeded since the importlib machinery supports multiple suffixes to look for, and thus the FT build can look for .abi3t.so and the GIL build can look for both .abi3.so and .abi3t.so.
The PEP is already treating these as distinct items in wheel tags, because treating them the same would be horribly backwards incompatible across the entire ecosystem. It’s not any different for treating them the same in the filename except that maybe less (or at least different) people would be impacted.
I replied on the packaging thread.
I’m adding this in PR #4862, along with a reorganized/expanded rationale, more rejected ideas, a nanobind quote, and generally making things more consistently/clearly worded.
The steering council have commented on the “packaging” PEP 803 thread that this is accepted: PEP 803: Stable ABI for Free-Threaded Builds (packaging thread) - #117 by corona10
Hi @encukou and @ngoldbaum,
On behalf of the Steering Council, I am happy to share that PEP 803 has been approved. Congratulations, and thank you for your work on this PEP.
We are particularly pleased to see this align well with the expectation expressed in the PEP 779 acceptance post that Stable ABI for free-threading should be prepared and defined for Python 3.15.
This is a meaningful step toward that goal, and we are glad to see the free-threading work continue on the right path.
Warm regards,
Donghee
on behalf of the Python Steering Council
Reposting this here for visibility. Sharing the same message as in my earlier post: PEP 803: Stable ABI for Free-Threaded Builds (packaging thread) - #117 by corona10
Thank you for your input, everyone here!
I’ve marked the PEP as accepted, filed an implementation issue and will add the bits, tests & docs for the upcoming alpha release.
I’m aware that the PEP doesn’t make everyone happy, but, it’s the step we’re making in 3.15.
As a reminder, features can be changed during the beta period, so if you find that something’s off in practice, there’s still time to change it.
Just a quick note that it would be great if stable ABI element access to PyVarObjectcould still be added. This would be needed for nanobind to support abi3t.
nanobind is used as the binding layer of several flagship machine learning projects. Some of them have free-threading packages that stand to benefit: LLVM’s MLIR and XLA projects, Google’s JAX framework, Apple’s MLX framework.
I know, but it’s looking like a 3.16 feature at this point. I’d like to do it right, and while I probably could do it in the month we have until feature freeze, there’s too little time for volunteer reviews (and also for explaining the change – PEP 697 was already quite dense).
The workaround is shipping extra builds/wheels for cp315-cp315t, for one more year. I’ll make sure the docs point that out.
I think nanobind can switch its object layout of nb_func which currently uses PyVarObject to use regular PyObject header. If this is done then it doesn’t need to wait for 3.16 to add PyVarObject stable ABI support.
I have created use regular object layout for nb_func by kumaraditya303 · Pull Request #1324 · wjakob/nanobind · GitHub to implement this and it seems to work.
@kumaraditya303 A good part of nanobind’s speedups (over pybind11) stem from a reduction of pointer chasing. It co-locates binding data structures with the underlying Python objects. The fix you proposed splits the function objects from its internals and is therefore not an acceptable solution to me.
For what it’s worth, that’s also my main concern with this PEP - that it turns “look-up the object struct” into a pointer-chase involving at least two type objects.
I do get that there probably isn’t another good way of solving the problem while the GIL and free-threading builds co-exist. But long-term I’m not-very-secretly holding out hope for a free-threading-only API with non-opaque objects
It exists - just don’t opt into the stable ABI. That means making more builds, but it’s a pretty clear tradeoff, and builds are almost certainly automated these days.
It would also help if you could help us calm down people who get angry when the non-opaque objects change and “break” them,[1] as that will help avoid the message we keep hearing that we need to provide 100% speed, 100% flexibility and 100% stability all at once. It’s taken years, but we’ve landed on a stable option and a fast/flexible option, but you can’t have them both at once.
You can get angry at us if we change them at the wrong time, but between releases is not the wrong time. ↩︎