PEP 803: Stable ABI for Free-Threaded Builds (packaging thread)

The confusion I see is that the term abi3 is being used in multiple ways. And in this particular context I’m not even sure which way people are meaning :slightly_frowning_face:

I know the following could be viewed as nitpicking, but in this case I think the details are important - after all, the whole problem is that “you know what I mean” isn’t true…

The platform tags abi3 and abi3t aren’t defined in terms of a set of symbols, they are defined much more operationally - a platform supports the tag abi3 if it says it does, and a wheel should only use the tag abi3 if it will run on a platform that supports abi3. In that context, abi3.abi3t is well-defined as a wheel tag, meaning “runs on a platform that supports either abi3 or abi3t”. There’s nothing about symbol sets or build options involved here, except in the sense that build tools should choose what tags to apply by default[1] based on the build options chosen.

On the other hand, .so names also use the strings abi3 and abi3t to mean something. I’m not 100% sure what, exactly - my impression is that they are disambiguators to allow multiple .so files to exist in the same directory, with Python interpreters preferring one or another based on the rules built into the import system. In that context, .abi3.abi3t is only defined if the import system says it’s defined.

There’s another aspect of the wheel tag situation, which is that it’s up to us (i.e., the PEP) to define what systems should claim to support which of abi3 and abi3t wheel tags. The PEP as it currently stands isn’t at all clear on this (because it’s framed in terms of what “installers should do” rather than what “platforms should say”[2]) but it strongly suggests that free threaded interpreters should say they accept abi3t, and GIL-based interpreters should say they accept abi3. This gives a very clear interpretation of what platforms wheels tagged with abi3, abi3t, or abi3.abi3t should mean - abi3 is GIL-based only, abi3t is free threading only, and abi3.abi3t is either.

But I think what @dstufft is referring to is a .so file extension of .abi3.abi3t, and not wheel tags? If that is the case, then I have no view on what we decide - how the interpreter loads .so files has no impact on packaging standards. Although I can understand (and agree with) his argument here. Overlapping installation directories are not covered by the packaging standards on how wheels are installed, and so the packaging ecosystem simply doesn’t care about disambiguation. As far as wheels (and hence wheel build tools) are concerned, just using a bare .so extension is 100% fine. ABI-tagged .so files are only needed for custom, non-wheel builds by distributors who are taking advantage of the fact that we support overlapping installations in the interpreter. So it’s only they who need tagged .so files (and they don’t typically use Python’s packaging tools to install them, they have their own installers and distribution formats).

I will note that PEP 803 as currently written has an error. In “Recommendations for build tools” it says:

Such extensions should be tagged with the compressed tag set abi3.abi3t.

But compressed tag sets are a concept that’s part of the wheel tag specification, and as such you don’t tag extensions with them. Rather, you create (one or more) extensions with a .so file type, and bundle them in a wheel tagged with compatibility tags.

Maybe what should be said here is something along the lines of

If all of the extensions contained in a wheel are compiled to be compatible with both abi3 and abi3t, the wheel should be tagged with the abi3.abi3t compatibility tag. If any extensions are compiled to be only compatible with abi3, the wheel must be tagged as abi3 only. Similarly for abi3t. Wheels containing a mix of abi3-only and abi3t-only extensions are invalid and must not be created.

The question of what name to give an extension built to be compatible with abi3 or abi3t (or both) is a core interpreter question, and should not be in the section on “the abi3t wheel tag”. Not least because if I’m compiling a C extension by hand, and not using a wheel building tool at all, I still need that information.


  1. “default”, because the user can always rename the wheel… ↩︎

  2. the latter is, I believe, a much clearer framing ↩︎

1 Like