Add a new way to get the list of subclasses of a type

Ideas
1

Now the What’s New In Python 3.12 — Python 3.13.12 documentation said:

  • This internal-only PyTypeObject.tp_subclasses may now not be a valid object pointer. Its type was changed to void* to reflect this. We mention this in case someone happens to be accessing the internal-only field directly.To get a list of subclasses, call the Python method __subclasses__() (using PyObject_CallMethod(), for example).

The question is: if the c-level metaclass needs to control the behavior of the type instance with its subclasses in tp_getattro, this way is wrong because PyObject_CallMethod calls the tp_getattro while tp_getattro also called PyObject_CallMethod so it will be an infinite recursion.

I think that adding the capi to get the list of subclasses directly is needed. This function should do that without any python attribute getting (just tp_*). The tag is:

PyObject* PyType_GetSubClasses(PyObject* cls);

Also a macro to ensure that user can use it with PyTypeObject*:

#define PyType_GetSubClasses(cls) PyType_GetSubClasses((PyObject*)cls)

If the cls is not a type object, it set TypeError and return NULL, or it will visit the tp_flags and tp_subclasses to return the final list. This way bypasses type attributes __subclasses__ access altogether and more fast than the way using PyObject_CallMethod.

2

You can call PyObject_GenericGetAttr to bypass an overridden tp_getattro and prevent the infinite recursion.

3

The final question is: the __subclasses__ may changed:

>>> class MyClass:
...     __subclasses__ = None
... 
>>> MyClass.__subclasses__()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    MyClass.__subclasses__()
    ~~~~~~~~~~~~~~~~~~~~~~^^
TypeError: 'NoneType' object is not callable

By this way the capi code cannot get subclasses the list of this type by PyObject_GenericGetAttr. So I think that use a function just visit the slots is better (just like the other magic method. Many other slots won’t use the way that first find the attribute then call, it visit just type slots and get it directly).

4

In general, Python follows the “consenting adults” principle. If you modify a dunder to break the Python data model, you will get breakage. Don’t do that.

By all means, dig into the obscure bits of Python to solve your problem, but if it’s just to demonstrate that you can break things that way, that’s well known and not considered a problem.

2 Likes
5

The “consenting adults” principle applies to semantic misuse, not interpreter safety.
Breaking the data model may lead to incorrect behavior or exceptions, but it should not result in SystemError or segfaults.

“consenting adults” often means that the result is just not what you think, but the result is still in a range and stable in same environment.

The issue is not that overriding __subclasses__ is possible. The issue is that the recommended C API approach relies on Python-level attribute lookup to retrieve interpreter-level metadata.

6

Hm? You’d just get a TypeError if __subclasses__ isn’t callable.

Anyways, if you want to bypass a type’s __subclasses__ method (which is dangerous in its own way), you don’t need a new C API function; you can just call type.__subclasses__(your_class).

1 Like