Use the limited C API for some of our stdlib C extensions

[encukou (Petr Viktorin)]

@malemburg

The stable ABI has been around for around 14 years now and pick up by the community is very very slow. Shouldn’t this tell us something ?

It hasn’t been widely advertised, and it’s been quite incomplete until recently. Notice the recent uptick in usage :‍)

@malemburg

You keep saying that the Python C API is hard to evolve due to the backwards compatibility requirements. The stable ABI / limited API makes this even harder, so I don’t follow your reasoning.

I know of one issue that makes this hard – exposed PyObject fields. That is solvable with a one-time ABI break, i.e. abi4.
If you know of any more issues where maintaining the stable ABI is more painful than API compatibility (PEP-387) or frozen per-version ABI, I’d love too hear them; ideally please comment in issue#4.

@malemburg

The only change would be to recompile the packages for new Python versions, but that’s easily done using cbuildwheels, while at the same time removing a lot of the build headaches.

Sure, cibuildwheel works for projects on PyPI.
How would this work for, say, vim, which is now able to use whatever Python3 you have installed? (The commit message for that has great summary btw)