A way to disable virtual subclassing with ABC classes

Ideas
1

There is a long standing issue with ABC classes in CPython (both the C and pure Python implementation), that causes memory and performance issues when you have a large number of subclasses.

Looking into the _py_abc implementation, I came to the same conclusions as this comment, and I think the current implementation is correct/necessary to handle every case with virtual subclasses, i.e. classes registered using register() (see my comment).

So it seems like the only thing we could try is optimizing the current implementation while keeping the existing logic, which seems relatively hard.

In Pydantic, our BaseModel class has a metaclass that inherits from ABCMeta to support abstract models, and we had a number of reports from users defining many subclasses. As it seems really unusual or even unnecessary for Pydantic to support virtual subclasses, I propose adding a way to disable registering subclasses, e.g. by introducing a new metaclass to the abc module. For the pure Python implementation, that would look like:

diff --git a/Lib/_py_abc.py b/Lib/_py_abc.py
index c870ae9048b..f8e494b1185 100644
--- a/Lib/_py_abc.py
+++ b/Lib/_py_abc.py
@@ -11,7 +11,23 @@ def get_cache_token():
     return ABCMeta._abc_invalidation_counter
 
 
-class ABCMeta(type):
+class BareABCMeta(type):
+    def __new__(mcls, name, bases, namespace, /, **kwargs):
+        cls = super().__new__(mcls, name, bases, namespace, **kwargs)
+        # Compute set of abstract method names
+        abstracts = {name
+                     for name, value in namespace.items()
+                     if getattr(value, "__isabstractmethod__", False)}
+        for base in bases:
+            for name in getattr(base, "__abstractmethods__", set()):
+                value = getattr(cls, name, None)
+                if getattr(value, "__isabstractmethod__", False):
+                    abstracts.add(name)
+        cls.__abstractmethods__ = frozenset(abstracts)
+        return cls
+
+
+class ABCMeta(BareABCMeta):
     """Metaclass for defining Abstract Base Classes (ABCs).
 
     Use this metaclass to create an ABC.  An ABC can be subclassed
@@ -34,16 +50,6 @@ class ABCMeta(type):
 
     def __new__(mcls, name, bases, namespace, /, **kwargs):
         cls = super().__new__(mcls, name, bases, namespace, **kwargs)
-        # Compute set of abstract method names
-        abstracts = {name
-                     for name, value in namespace.items()
-                     if getattr(value, "__isabstractmethod__", False)}
-        for base in bases:
-            for name in getattr(base, "__abstractmethods__", set()):
-                value = getattr(cls, name, None)
-                if getattr(value, "__isabstractmethod__", False):
-                    abstracts.add(name)
-        cls.__abstractmethods__ = frozenset(abstracts)
         # Set up inheritance registry
         cls._abc_registry = WeakSet()
         cls._abc_cache = WeakSet()

(Or, alternatively, by adding a allow_virtual_subclasses metaclass argument).

This way, the BareABCMeta class (name TBD) doesn’t have to override __subclasscheck__ and we get way better performance.

1 Like