Cross-module base type discovery with new module patterns


Not 100% sure this is the right forum since this mostly seems to be about the Python language rather than the C API, but feel free to relocate.

With the new module patterns (PEPs 489, 573, 630), what is the intended way to get a PyTypeObject* to use as a base type when the type is in another module? Today with static types, there’s usually just an extern PyTypeObject …; and the linker figures things out. However, now that types objects are dynamically created (heap types) and (presumably; I’ve not actually seen docs on where best to store the things) stored in module state structures, what is the recommended way of accessing base types?

Is it best to import the module, access the type via GetAttr on the module and cast the result to PyTypeObject* (hoping that someone doesn’t do basemod.basetype = 4 or more advanced impersonations…)? Or should I rummage through the other module’s state structure and access it that way? Is there any guidance on how to do this?



Cc @encukou since I know you’re involved in the PEPs listed above.


This touches on one of the remaining issues with the “new patterns”: type checking. We have a viable design, but it’ll not make it into Python 3.12.
Ideally, you’d GetAttr and then check the class you got has the memory layout you need, so impersonators make you fail gracefully. Currently the API for a proper check is missing, but to prevent honest mistakes you could do PyType_Check + verify that tp_basicsize is what you expect.

And if the C API is not just classes/objects, it might be worth it to export a PyCapsule – though that doesn’t prevent “impersonation” either.

Ok. Given that, is it best to just add the PyTypeObject* objects as attributes to the module and let that hold them or do they need to be stashed and managed in module state somehow?

Yes, using Python attributes is easiest.

There are many ways for users to shoot themselves in the foot. The goal should not be that “impersonations” are impossible, but that they fail with an exception rather than memory corruption.