How does "TypeError: __class__ set to <class 'A'> defining 'A' as <class 'A'>" occur?

Python Help
1

Following my rant regarding how dataclasses.dataclass being a class decorator rather than a metaclass-powered class is preventing __init_subclass__ from accessing fields of a dataclass in a response to @MegaIng, I set out to implement exactly that, a metaclass that is really a thin wrapper to the dataclass decorator with all of its capabilities while calling __init_subclass__ only after the new class has been transformed by dataclass.

The goal was largely achieved by:

  1. inserting a dummy base class with a no-op __init_subclass__ to block any __init_subclass__ of the rest of the base classes from being executed during the creation of the new class, and,
  2. deleting __init_subclass__ of the dummy base class after the new class is dataclass-transformed, so that,
  3. calling __init_subclass__ of the super class of the new class can then follow the intended MRO
from dataclasses import dataclass, fields
from typing import dataclass_transform

@dataclass_transform()
class DataclassMeta(type):
    def __new__(metacls, name, bases, namespace, **kwargs):
        class InitSubclassBlocker:
            def __init_subclass__(cls, **kwargs):
                pass
        configured_dataclass = dataclass(**kwargs)
        for key in ('slots', 'frozen'): # TODO: pop all known dataclass keywords
            kwargs.pop(key, None)
        cls = configured_dataclass(super().__new__(
            metacls, name, (InitSubclassBlocker,) + bases, namespace, **kwargs))
        del InitSubclassBlocker.__init_subclass__
        super(cls, cls).__init_subclass__(**kwargs)
        return cls

class Dataclass(metaclass=DataclassMeta):
    pass

so that I can have a base class that processes dataclass fields when subclassed:

class PrintFields(Dataclass):
    def __init_subclass__(cls, **kwargs):
        for field in fields(cls):
            print(field.name, field.type)
        super().__init_subclass__(**kwargs)

class Foo(PrintFields):
    foo: str

which correctly outputs:

foo <class 'str'>

But then I tried enabling the slots option for PrintFields:

class PrintFields(Dataclass, slots=True):
    def __init_subclass__(cls, **kwargs):
        for field in fields(cls):
            print(field.name, field.type)
        super().__init_subclass__(**kwargs)

And got:

Traceback (most recent call last):
  File "/ATO/code", line 21, in <module>
    class PrintFields(Dataclass, slots=True):
TypeError: __class__ set to <class '__main__.PrintFields'> defining 'PrintFields' as <class '__main__.PrintFields'>

Demo here

I’ve never seen this error before, and a look at the CPython source that produces this error message tells me that it has something to do with __classcell__. With a bit of a search for that keyword, I found an open CPython issue (originated from this issue) regarding this symptom, but am still unable to comprehend from the responses the actual cause of this issue, and how I can possibly work around it.

Any insight would be appreciated.

2

__classcell__ has to do with the internals of super()'s zero-argument feature. For that to work, it needs to fetch self and the class the method was defined in. self can be grabbed by looking through the caller’s local variables, but the defining class needs help. What happens is if the name super is referenced in a method, the compiler will add a closure variable named __class__, set to the class. Where __classcell__ comes into play is that when executing the class body, a new cell object is created and stored under that name. The methods being defined fetch that to put in their closures, then type.__new__ sets it to the class after that’s actually created.

So the problem with using slots is that currently that has to be set at class creation. So dataclass has to go create a new class entirely. This means we ended up with two classes with different names - the undecorated one in the cell object, the decorated one returned from your metaclass. This error detected the mismatch and complained.

I think the solution is for your code to check for the __classcell__ in the namespace, and if present make sure you update the contents to be the final class object.

1 Like
3

Thanks for the detailed explanations of what __classcell__ does.

Actually after I posted the question I found a good solution to this issue on StackOverflow:

And knowing that the new class when slots=True is created by dataclasses._add_slots:

I simply use a closure to inject __classcell__ into the namespace:

def inject_classcell(classcell):
    def _add_slots(cls, is_frozen, weakref_slot):
        ...
        # And finally create the class.
        qualname = getattr(cls, '__qualname__', None)
        if classcell:
            cls_dict['__classcell__'] = classcell
        cls = type(cls)(cls.__name__, cls.__bases__, cls_dict)
        if qualname is not None:
            cls.__qualname__ = qualname
        ...
    return _add_slots

and patch dataclasses._add_slots with the injected version:

from unittest.mock import patch

@dataclass_transform()
class DataclassMeta(type):
    def __new__(metacls, name, bases, namespace, **kwargs):
        ...
        cls = super().__new__(
            metacls, name, (InitSubclassBlocker,) + bases, namespace, **kwargs)
        with patch('dataclasses._add_slots',
                inject_classcell(namespace.get('__classcell__'))):
            cls = configured_dataclass(cls)
        del InitSubclassBlocker.__init_subclass__
        super(cls, cls).__init_subclass__(**kwargs)
        return cls

And the code would work.

Demo here

Obviously patching dataclasses._add_slot with copied code like that is super ugly and unmaintainable, but I don’t see a better solution short of asking for an enhancement to dataclasses._add_slot in the Ideas forum.

Suggestions are welcome.

4

Since dataclass._add_slots gets its namespace from the __dict__ of the current class:

I solved the problem by propagating __classcell__ as a class attribute for dataclass to create a namespace with, and deleting it afterwards:

_dataclass_keywords = set((code := dataclass.__code__).co_varnames[
    code.co_argcount:code.co_argcount + code.co_kwonlyargcount])

@dataclass_transform()
class DataclassMeta(type):
    def __new__(metacls, name, bases, namespace, **kwargs):
        class InitSubclassBlocker:
            def __init_subclass__(cls, **kwargs):
                pass
        configured_dataclass = dataclass(**{key: kwargs.pop(key)
            for key in kwargs.keys() & _dataclass_keywords})
        cls = super().__new__(
            metacls, name, (InitSubclassBlocker,) + bases, namespace, **kwargs)
        if classcell := namespace.get('__classcell__'):
            cls.__classcell__ = classcell
        cls = configured_dataclass(cls)
        if hasattr(cls, '__classcell__'):
            del cls.__classcell__
        del InitSubclassBlocker.__init_subclass__
        super(cls, cls).__init_subclass__(**kwargs)
        return cls

Demo here