Use the limited C API for some of our stdlib C extensions

Core Development
18

All in all, a good discussion. I recommend not hurrying with actions, this may be important to some folks, but it doesn’t seem urgent to anyone. Let’s wait until we’ve had some C API discussions among those core devs who make it to Brno in October.

Separately, do we have documentation somewhere of all the various C API layers? I feel it would maybe belong in Extending and Embedding the Python Interpreter — Python 3.11.5 documentation but I don’t see anything in the index that seems to cover this topic.

I’ve found at least the following categories – did I miss or misrepresent any?

  • Internal API – Lives in Include/internal/pycore_*.h, requires defining Py_BUILD_CORE before including. Is typically included by adding Include/internal to the C compiler’s include search path.
  • CPython API – Some kind of public API that’s specific to CPython. Lives in Include/cpython. Don’t include these directly, you get them automatically when you include the corresponding file in Include unless you define Py_LIMITED_API. (Most of these are automatically included by Python.h, but a few aren’t.)
  • Limited API – What you get from #include "Python.h" if you first define Py_LIMITED_API. This specifically excludes everything in Include/cpython.
  • Stable ABI (note “B” for “Binary”) – ABI that CPython promises to be stable across releases (there must be an expiration but it’s unclear what that is if we never move to Python 4.x). It consists of the Limited API. There are versions you can select by setting Py_LIMITED_API to specific values. See also PEP 384 and PEP 652.
  • Private API – Legacy APIs that are currently exported by the limited and/or CPython API but are not intended for public use nor have guarantees about ABI stability. These can be recognized by a name starting with _ (usually _Py) and being undocumented, but occasionally an apparent private API is treated as a public API because we have found that it’s commonly used by 3rd party packages and we don’t want to break them.
  • Unstable API – A recent development, see PEP 689. These are public APIs that are useful for some category of 3rd party tools (like debuggers, profilers) but that for various reasons we don’t want to have to guarantee across minor versions. Their name must start with PyUnstable_. Users of these APIs may have to rewrite their code for each new minor release (though we will try not to change these unnecessarily).
5 Likes