Just to be clear, I am not suggesting any changes to the API, nor changing the size of PyObject, just allowing changes to how the fields in PyObject are used.
The stable ABI that you get if you compile for 3.13 or later has the changes we need to do this, but it does expose the size of the PyObject header. abi3t hides the size of the PyObject header, which is an API change for code that embeds PyObject in structs, like Numpy.
Moving to abi3t requires code changes. The one time ABI breakage I am proposing does not.
Don’t break the ABI without creating a new name for the new ABI.
We can’t just go breaking that promise on our users like that. If you can’t work around it, add it to the list of things that require non-stable ABI and that we can handle when we eventually do come up with a way to “break” the ABI.
The Steering Council decided not to create a system for managing new incompatible versions of the stable ABI. So we are still in the “no breaks” situation.
Whenever abi3t becomes the stable abi[1], it’s going to be breaking. How much of a benefit is it to break multiple times rather than try and cordinate any additional beneficial breakage to happen at the same time as the known needed upcoming breakage?
My understanding of the situation there is that is the desired end state. ↩︎
Wrong question, ask this instead: how much of a waste is it if everyone stops using the stable ABI entirely because we burned them by changing it?
It’s going to be a new stable ABI, and abi3 will eventually stop working, but abi3t will work everywhere (GIL-ful and GIL-less) for a period. But there isn’t a single point where abi3 stops and abi3t starts - they have overlap and the deprecation period is essentially as long as the GIL-enabled build remains available.
I was somewhat presupposing that whatever breakage was being considered would end up as a new stable abi (mostly because I can’t see anyone agreeing to call something incompatible with abi3, abi3)
Right, but at some point during that overlap, projects will require switching over. So maybe a better way of phrasing this is “can we find a reasonable way to not do this more than once”?
Was there ever a pronouncement on this? The thread on PEP 809 seems to have petered out without any explicit rejection of the idea – which is why I proposed above to come back to that, as it seems a much saner approach to the subject of (rarely, but not never) changing ABI in a structured way.
I’m generally in favour of an ABI break every now and then, whenever it’s deemed helpful for CPython’s development. It seems way more important to me that the API does not break too heavily (such as by hiding the PyObject struct fields), so that packages can be recompiled for new ABI revisions without major code updates.
From what I hear, package maintainers/distributors are worried about having to build a wheel for each Python version on each supported platform. The number of wheels shouldn’t scale linearly with the number of supported versions. Whether it’s one wheel or O(1) wheels (like 2, 3 or 4) per platform doesn’t really matter that much. If CPython renewed its ABI every 5 years, we’d be at 3-4 wheels per platform to build for a reasonable Python version support range of 10-15 years.
There is one issue on the user side that may arise, if they want to run old ABI wheels on a new Python that only has a newer ABI. Thing is, dependency issues like these will always arise, because applications tend to have more than one dependency and those also have to work together. And we can consider building ABI shims if the need truly arises.
As long as maintainers keep producing fresh wheels for a reasonable range of ABI versions, including old ones, I don’t see a problem with introducing new ABIs from time to time.