In technical terms, it includes all of PEP 803 and leaves other sensible decisions open - the technicalities aren’t my main concern, and we haven’t historically used PEPs to define each Stable ABI’s specific interfaces/etc. (apart from significant, discussion-worthy additions).
The main part of this proposal that I’m interested in seeing happen is a long-term plan/schedule/process for managing changes to the ABI. Most of the details are open to being changed, so feel free to argue them - this isn’t a “my way or the highway” proposal. I mainly want to formally establish my opposition to further changes to the Stable ABI without any long-term planned stability. Eventually, I expect discussion will settle down and PEPs will go to the steering council for a decision.
(Pre-emptive response: yes, the interfaces API could be added separately from the rest of this proposal. It’s bundled together because I believe we need it for actual ABI stability (which includes no additions) to be viable, and it isn’t sufficiently valuable without that (because we can just keep changing the Stable ABI, so no need for added stability). So discuss the interfaces API here, by all means, but don’t confuse it for the main point of the PEP - it’s essential, but not central.)
I really don’t like calver as a versioning scheme. I’ve no problem with
it when applied to applications (eg. Python itself) but semver provides
so much useful implied information about a library/API/interface. Your
use imposes timescales which appear to be completely arbitrary.
I’d prefer using semver but removing the association with the Python
version. As the new ABI will be incompatible with abi3 then the new
version will be v4.0.0. There would be no problem moving to v5 during
the life of Python3, or in 5 years, or whenever it is appropriate.
Aesthetically, I don’t love using the year, but I think as long as the number is a single integer and increases monotonically, it’s fine.
One other possibility would be to indicate the Python minor version instead of the year, e.g., start with abi15, and then the 2031 increment would be abi20.
To avoid having this thread be entirely bikeshedding:
Migrating from one ABI (e.g. abi2026) to the next (e.g. abi2031) should be a manual task. There is enough overlap between ABI updates that most projects only need to support one at a time, and can update all of their builds at once if their own support matrix allows. There is no expectation for package maintainers to immediately support each new ABI.
I think this is saying that there would be no such a thing as an abi3.abi2026 (or abi2026.abi2031) wheel, and in most cases you would just target the newest ABI supported by the oldest Python you support. So you might get abi3 and abi2026 wheels for the next few years until GIL-enabled builds are gone?
Or is it more that abi3.abi2026 would be used during the free-threading transition to support free-threaded/GIL-enabled builds, and then after that it would not be worth anybody’s time to try to support multiple ABIs?
I tend to agree, except by design the ABI only ever has a single version field from SemVer. Only breaking changes are allowed - there’s no concept of a non-breaking change (in this case, it’s called “don’t make the change at all”).
And yeah, the timescales are 100% arbitrary. If you have alternate arbitrary timescales to propose, feel free Right now they’re “at least 5 years” because that seems to be how long most package developers support older runtime versions for.
We actually tested the various versioning schemes with the group of core developers at the sprints, and using the year seemed to be best received. It aligns somewhat with other languages that specify their standards based on the year of publication, which is essentially what we’re doing here.
Yeah, it’s possible, though it also looks like a two digit year. It’s also less meaningful for other implementations, who aren’t necessarily following CPython’s versioning (and also the ABI is actually totally unrelated to the language, it just so happens that we pin the reference runtime version to the language version because otherwise we’d all go mad ).
Correct. There’s a transition period where you need to build abi3 wheels for Python 3.14 and earlier, and can either build abi2026 wheels for 3.15 and later or keep abi3 and not support free-threading. But there’s no way to build an extension module that simultaneously supports two ABIs - that means the ABIs are identical and so they wouldn’t be two ABIs.
Indeed. The aim is to have a single migration process that may involve multiple wheel builds now, and by the time we’re deciding whether free-threading becomes the default then we aren’t having to factor in making everyone switch to a new ABI at that time.
I tend to agree, except by design the ABI only ever has a single
version field from SemVer. Only breaking changes are allowed -
there’s no concept of a non-breaking change (in this case, it’s called
“don’t make the change at all”).
And yeah, the timescales are 100% arbitrary. If you have alternate
arbitrary timescales to propose, feel free Right now they’re “at
least 5 years” because that seems to be how long most package
developers support older runtime versions for.
I support 5 year old versions only because that corresponds to the
oldest supported version of CPython at any particular time.
We actually tested the various versioning schemes with the group of
core developers at the sprints, and using the year seemed to be best
received. It aligns somewhat with other languages that specify their
standards based on the year of publication, which is essentially what
we’re doing here.
That makes some sort of sense if you typically want to make breaking
changes every year. However (unless I’ve seriously misunderstood) you
seem to be precluding any change for 5 years, even non-breaking changes.
So a shiny new feature may be added to the standard API but (unlike now)
won’t be allowed into the Limited API for years???
It can be added as an interface (new feature, described in the PEP), which basically forces anyone who wants to use it to dynamically test whether it exists or not (and also keeps new functions out of the ABI by passing them through as function pointer tables).[1]
These can be regular functions for the non-stable ABI, though I don’t necessarily think there’ll be a lot of value in having two different ways, especially since the interface has ways to be faster than a regular API.
But it’s there specifically to allow shiny new features to be available immediately, and to make callers safely handle earlier versions of Python that don’t have those features.
I’m reminded that I also need to add an API for interpreter/runtime-level interfaces, not just object-level ones… it’ll look identical to what’s already there though, just different names and context parameters (i.e. takes the interpreter state rather than a specific object). ↩︎
Right, the ABI tag will be abi2026 (or whatever number) in place of where you’d currently have abi3, and there’s no need to filter by runtime version. Any runtime that supports the ABI should be able to provide the entire ABI without conflict.
You might need to point at exactly which part needs changing for me - those specs should only be defining the ABI tag, but not exhaustively listing all the possible values. Those have always come from the runtime.
Sure, that has no bearing on the ABI requirement. It’s purely a signal for “you might need an older version of this package to work with an older version of Python”. There’s no text there about ABI requirements, so there’s nothing to change - you can still use them both if you have specific version requirements and want to use the Stable ABI. Nobody is going to force you to support older versions than you want to support, and if you don’t want to use the runtime tag, then Requires-Python is another option.
The ABI requirement is implicit because it is currently tied to the
Python version. Implementing a new version scheme for the ABI means that
it needs to be stated explicitly using a new metadata key (and not
changing an existing one).
You can’t use Requires-Python because you don’t know what upper bound to
specify, ie. you don’t know now what the last version of Python will be
that supports 2026.
What you’re proposing now would be better off handled in the discussions in the Packaging category (particularly the discussion about supporting upper bounds at all in Requires-Python). It’s downstream of what we do in the core runtime.
If you offer a wheel that says it requires a particular ABI, and the runtime you’re installing for doesn’t provide that ABI, then installers will look for a different wheel of the same version but with a compatible ABI tag. Whether they decide to also look at older/newer versions of the package is up to the package installer, and is already too complicated to resolve. This proposal actually helps by increasing the chance of the ABI tag being compatible, and reducing the need to look for another package. But as I say, that’s all downstream of this, and it largely already exists.
If the package you’re installing doesn’t have any extensions, there’s no ABI to worry about.
If it does have an extension, the ABI version is included in the name of the wheel you download for installation. This is part of the wheel specification.
If you’re compiling the extension from source (which is generally not desirable) you’ll either succeed or fail depending on whether the source is compatible with your python installation. But doesn’t the python version requirement cover this possibility?
With the 10-year ABI support window, I think the problem is practically moot. The solution for someone who uses an unchanging part of the API is to build a new wheel every five years or so, and for those who are affected by deprecations to make a maintenance release every ten years or so. python-requires with only a lower bound seems sufficient.
Agreed that this is a problem for packaging, but if people might respond to this by thinking they should start setting upper bounds, it might be worth a note in the PEP to head that off.
I think the new API is sufficiently viable without a new versioning scheme. I also think it’s somewhat underbaked still – especially for freezing it for five years immediately (with no user feedback) and making it the only mechanism for exposing new features in the stable ABI.
I feel like to review this I’d need a crystal ball, or at least plans for the next 5 years.
PEP 809 has the same Motivation and Rationale are the same as in PEP 803, which aims for smallest change to the status quo to get the goals (while remaining maintainable).
I’d love to see the motivation/rationale for the things this PEP does differently: the new API and the versioning change. Is there a real-world problem being solved here?
Right now they’re “at least 5 years” because that seems to be how long most package developers support older runtime versions for.
).