This was originally posted at C API: Add `Py_GetBuildInType` and `Py_GetBuildInTypeBorrowed` · Issue #148643 · python/cpython · GitHub
Currently, the C API exports all build-in types as a static data export. This limits how alternative python implementations can implement static types.
I propose that we export a Py_GetBuildInTypeBorrowed function that works similar to the Py_GetConstantBorrowed function. And use these in the type check macro’s when compiling for the limited api.
Based on a bit of GitHub stalking, it looks like you want this for RustPython. Just mentioning that because it helps clarify that it’s a real problem with an actual Python implementation.
Yes you are right. I got here because I’m trying to implement the C-API in RustPython.
Why does PyObject_GetAttrString(PyEval_GetBuiltins(), "name") not suffice here?
That would be fine, but requires a custom version of the header files, which is not a big deal. But maybe we want CPython to move away from those statics as well.
Well, yes, we don’t want static types at all - they’re only still there because the existing data exports have to keep working. We’d much prefer everyone not use them.
The two functions I mentioned are in the stable ABI already… why do you need custom headers?
I’m implementing the C-API in RustPython. Which is not able to export the types as statics. Your solution can work but this requires the update the type-check macros from #define PyBool_Check(x) Py_IS_TYPE((x), &PyBool_Type) to `#define PyBool_Check(x) Py_IS_TYPE((x), PyObject_GetAttrString(PyEval_GetBuiltins(), "bool")) in the header files.
Oh I see, you’re on the other side of the API.
I would say just don’t implement the type checks as macros. We somewhat regret that now, and you’re giving us more reasons to regret it 
Probably all we’d want to change is to make the type check macros into real functions, rather than making part of the data access into a real function.
So you would like to add a PyTypeObject *Py_GetBuiltInType(unsigned int type_id) function which would use identifiers like Py_BUILTIN_TYPE_BOOL. It would set an exception and return NULL for unknown identifier.
Would it be possible to reimplement the weird API of Python built-in types using Py_GetBuiltInType(), such as #define PyBool_Type (*Py_GetBuiltInType(Py_BUILTIN_TYPE_BOOL))? Weird because PyBool_Type type is PyTypeObject instead of PyTypeObject* (pointer). According to a quick test, it seems like work!
Now the question is if CPython would benefit from such function and if we should update existing code to replace PyBool_Type with Py_GetBuiltInType(Py_BUILTIN_TYPE_BOOL). If RustPython can implement the C API using #define PyBool_Type (*Py_GetBuiltInType(Py_BUILTIN_TYPE_BOOL)), I don’t think that it’s worth it to migrate away from PyBool_Type. I’m not sure that adding such function to CPython would be useful neither.
Can you start with a private function _Py_GetBuiltInType() in RustPython?
Note: In practice, you should use borrowed references to avoid reference leaks.
dir(builtins) lists 163 names. 10 can be ignored ('_', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'copyright', 'credits', 'license', 'help'). There are different kinds: exceptions, functions and types.
grep -h 'PyAPI_DATA(PyTypeObject) ' Include/*.h Include/cpython/*.h finds 85 types (including 4 private types, _Py prefix).
Changing the type-check macros into exported functions seem to be the best alternative to me. This should give everyone the most flexibility.
What is the way forward here. Do we want to convert the type check macro’s to function calls? I can draft a MR if desired.
I think this is too late for 3.15, so, personally, I won’t prioritize it before feature freeze (planned for May 5).
A draft is the next step. These should only be functions in 3.16 limited API; I’d recommend getting one function through review before tackling the rest.
You might want to open a C API WG issue too.
Is BuildIn a typo for Builtin?
Oh well. That’s a relaxing situation. I will open a PR in the near future.
I’ve created an issue on the C API WG at Convert type check macro's to functions on the limited api · Issue #105 · capi-workgroup/decisions · GitHub
Only RustPython seems to have an issue with accessing a built-in type (&PyBool_Type) in #define PyBool_Check(x) Py_IS_TYPE((x), &PyBool_Type) macro. You can modify the C API implementation in RustPython, but not modify CPython. I get the motivation to modify CPython.
Note: PyPy exposes PyAPI_DATA(PyTypeObject) PyBool_Type; in its C API and it uses #define PyBool_Check(x) (Py_TYPE(x) == &PyBool_Type) macro.
@bschoenmaeckers: What do you think? Can you fix the issue in RustPython?
Oops. I meant: I don’t get the motivation to modify CPython.
My August thinking on this:
Built-in types are constants. Data exports. They’re not that different from Py_True.
We’ve already added a function to get constants in limited API, Py_GetConstant. For the reasoning, see this thread.
So, a proposal:
Let’s go through all the data exports in the stable ABI (except non-deprecated & internal), and add IDs for use with Py_GetConstant. That would be IDs for:
Py_*Type (except *Iter*, PyModuleDef_Type, PyMemberDescr_Type etc.) (~40)PyExc_* (~70)And then for #108, also expose PyMethod_Type this way.
Does that sound reasonable?
This constrains CPython as well: if PyBool_Type is a process-global C constant, it must be shared by all interpreters, and so it can never be a heap type. And so, we can’t get rid of static types.
RustPython is not able to implement these static type exports. Because RustPython’s PyObject layout requires data to be allocated in front of the pointer (prefix allocation). So we are not able to export static pointers to the right offset into to PyType_Type struct. There is however some minimal support in linux linkers to do this, but it’s not really supported in other platforms like Windows.
PyPy provides its own C API in /usr/include/pypy3.11/, with boolobject.h for example. But I cannot find C header files in RustPython. Does it mean that a C extension is built with CPython header files and then loaded with RustPython which implements the C API ABI?
I found crates/capi/src/boolobject.rs in RustPython:
define_py_check!(fn PyBool_Check, types.bool_type);
In CPython, PyBool_Check() is implemented as a macro:
#define PyBool_Check(x) Py_IS_TYPE((x), &PyBool_Type)
So once you build the C extension, the machine code becomes ob->ob_type == &PyBool_Type. There is no PyBool_Check() function at the ABI level.