The numbers I quoted are only a week or so old (perhaps even less – I don’t know how often the ClickHouse database site is updated with more current numbers from PyPI).
Yes, there are a few more packages shipping abi3 wheels nowadays than say a year ago, but not substantially more to make a significant difference.
Incidentally, a lot of those packages are Rust based. Perhaps the PyO3 is defaulting to abi3 where possible. For a few others, they started shipping abi3 wheels for more recent Python versions, but not for older ones (I guess they were missing some APIs in the ABI3 which were added now).
The main difference is that you cannot change the APIs in the limited API at all, without breaking the ABI. Without this limitation, the APIs could be changed subject to the normal deprecation procedures and participate in the evolution of the APIs.
And because the stable ABI doesn’t even specify an expected lifetime in years or number of releases, it means that no changes are possible until we move to 4.x.
Lifting the requirement to be stable across all 3.x versions would help with this problem, of course, but then I don’t think we’re that far off from the regular deprecation process, which also supports compatibility for at least 3 releases.
I’ll check the issue you mentioned, if this has already been mentioned.
BTW: It’s not only the PyObject structure that’s affected, the corresponding type objects are affected as well and with them the way e.g. slots are used, found, managed, etc. A type defined based on the stable ABI in say Python 3.7 would still be expected to work in a future Python 3.20, unless I’m missing something. This hinders evolving the internals.
I don’t think we should be supporting such an unusual use case.
They appear to allow use of Python in plugins, but then let the user decide which Python version to link against. IMO, the version should be fixed by the application embedding Python and thus managed that way and by the application maintainer – not leave this decision to the user.