Confusion with class attributes

Python Help
1

I’m working on a decorator which makes a new class whose instances wrap instances of the decorated class. Like this:

@binclass
class FullBoxBody2:
    field : type_here

The new class gets its .__name__ set to that of the wrapped class, and the wrapped class gets its .__name__ set to f'{cls.__name__}__original'.

This isn’t working. It looks good at some point right after I make the new class, then just afterwards they both claim that .__name__ is "FullBoxBody2__original". I think this behaviour is related to some other attribute lookup troubles I’m trying to debug.

Anyway, I made this test example:

Python 3.12.7 (v3.12.7:0b05ead877f, Sep 30 2024, 23:18:00) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...   __name__ = 'X'
...
>>> C.__name__
'C'
>>> C.__dict__
mappingproxy({'__module__': '__main__', '__name__': 'X', '__dict__': <attribute '__dict__' of 'C' objects>, '__weakref__': <attribute '__weakref__' of 'C' objects>, '__doc__': None})
>>> getattr(C,'__name__')
'C'
>>> C.__dict__['__name__']
'X'

Notice in particular that the mapping proxy still has a '__name__':'X' entry.

I do not understand what’s going on here. Can someone explain?

The Python docs say that a class __dict__ is a read only mapping, but if that were true I’d have expected an exception from my attempts to set .__name__.

Instead, somehow, I presume that .__name__ is being set somewhere and later lookups of the .__name__ are finding that common somewhere.

2

The answer to what you have shown in code lies with type.__dict__['__name__']. This is essentially a slot descriptor like you get when using __slots__: __name__ is not stored in __dict__ but at an offset inside of the C level struct.

This means you can’t just reassign it from within the class body of C - this will end up in the __dict__, as you noticed, but there is no code that moves it from there into the relevant slot.

Also notice how by default there is no __name__ in C.__dict__ and that assigning C.__name__ = 'foo' doesn’t create one.

If you are still experiencing issues when assigning to __name__ from outside the class body, please show that code.

3

I’ll post a fuller example. My tiny example above is sort of to show how little I understand here.

Before getting into the fuller example, some questions still outstanding to me:

  • if the class __dict__ is read only, how is it that I can do a cls.__name__ = 'new name here' at all?
  • that clearly sets something, because looking up .__name__ later finds the new value, but not just on cls - also on the wrapper class

In the real class I’m not making a __name__ class attribute directly in the class definition, I’m going, essentially:

name0 = cls.__name__
..............
class BinClass(cls, AbstractBinary):
    _baseclass = cls
..............
BinClass.__name__ = name0
cls.__name__ = f'{name0}__original}

where BinClass is the new wrapper class I’m making.

So. At the bottom of the decorator is this code:

  cls.name0 = name0
  cls.__name__ = f'{name0}__original'
  assert BinClass._dataclass is dcls
  assert BinClass._dataclass.__name__.startswith(cls.name0)
  BinClass._check()
  BinClass.__name__ = name0
  assert BinClass.__name__ != 'BinClass'
  X("@binclass: returning %s %r", BinClass, BinClass.__name__)
  X("  BinClass %d:%s", id(BinClass), BinClass.__name__)
  X("  cls      %d:%s", id(cls), cls.__name__)
  X("  dcls     %d:%s", id(dcls), dcls.__name__)
  assert BinClass._baseclass is cls
  assert BinClass._dataclass is dcls
  global LastBinClass
  LastBinClass = BinClass
  return BinClass

There’s some extraneous stuff there, but best to show everything. Notes:

  • cls is the wrapped class
  • dcls is a dataclass I’ve made from the cls.__annotations
  • BinClass is the wrapper class; it subclasses cls in order to let cls provide some methods
  • X() is a debugging function, essentially a print
  • LastBinClass is a global I set up entirely so that I can inspect it from another module after calling the decorator - just for debugging

The output from the X() calls is like this:

@binclass: returning <class 'cs.binary.binclass.<locals>.BinClass'> 'FullBoxBody2'
  BinClass 140252895848048:FullBoxBody2
  cls      140252895825072:FullBoxBody2__original
  dcls     140252895826704:FullBoxBody2__dataclass

Here the .__name__ attributes seem correct.

Now the fun begins.

I’m calling the @binclass decorator from inside a decorator in another module, shown below. The additional decorator is so that I can supplant a registration of the wrapped class with the new wrapper class. Thus:

@boxbodyclass
class FullBoxBody2(BoxBody):
  ''' A common extension of a basic `BoxBody`, with a version and flags field.
      ISO14496 section 4.2.
  '''
  version: UInt8
  flags0: UInt8
  flags1: UInt8
  flags2: UInt8

The second decorator is this:

def boxbodyclass(cls):
  ''' A decorator for `@binclass` style `BoxBody` subclasses
      which reregisters the new binclass in the
      `BoxBody.SUBCLASSES_BY_BOXTYPE` mapping.
  '''
  if not issubclass(cls, BoxBody):
    raise TypeError(f'@boxbodyclass: {cls=} is not a subclass of BoxBody')
  cls0 = cls
  cls = binclass(cls0)
  X("@boxbodyclass:")
  import cs.binary
  LastBinClass = cs.binary.LastBinClass
  X("  LastBinClass %d:%s", id(LastBinClass), LastBinClass.__name__)
  X("  cls0         %d:%s", id(cls0), cls0.__name__)
  X("  cls          %d:%s", id(cls), cls.__name__)
  BoxBody._register_subclass_boxtypes(cls, cls0)
  breakpoint()
  return cls

The output here looks like this:

@boxbodyclass:
  LastBinClass 140252895848048:FullBoxBody2__original
  cls0         140252895825072:FullBoxBody2__original
  cls          140252895848048:FullBoxBody2__original

Suddenly the .__name__ attributes all have the same value.

I find this very confusing.

4

Imagine it something like this:

type_names: dict[FakeType, str] = {}

class FakeType:
    @property
    def name(self):
        return type_name[self]
    @name.setter
    def name(self, value):
        type_name[self] = value

where type_names is a C level variable you cannot access any other way. __name__ is essentially such a property. The above code has an attribute name, despite there not being a name in FakeType().__dict__.

Please produce a small, isolated, single file example reproducing the issue you are having. You are leaving out to much stuff to identify the problem.

1 Like
5

The dict being read-only just means that you can’t change it directly, by writing something like:

C.__dict__['something'] = ...

That will raise a TypeError. But, you can still write:

C.something = ...

and get back the new value by writing:

C.__dict__['something']
1 Like
6

Well, as I should have expected, I’m an idiot; I’ve been chasing this for a few days now, on and off, to no avail.

I was going to snarkily reply that if I had a small single file example I might have debugged it already. But that I’d go and try to make one.

So, less than 20 seconds into starting the single file, gathering up my stuff to prune it down to the minimum I realised that there’s a third decorator in play here, one I use so routinely that I’d almost forgotten it was there. It’s a decorator I use on my decorators, which has the boilerplate to let a decorator be used like this:

@mydeco
def func(...):

and also like:

@mydeco(this_way=True)
def func(...):

so my decorators are all declared this way:

@decorator
def binclass(cls, this_way=False, ...): # whatever

One of the things @decorator provides is to conveniently rename the decorated object with the original’s name and __doc__ etc. Like functools.update_wrapper does, and in fact that’s its preferred route, with some fallback for older Pythons.

So, the code in there looked like this:

    if decorated is not func:
      # We got a wrapper function back, pretty up the returned wrapper.
      # Try functools.update_wrapper, otherwise do stuff by hand.
      try:
        from functools import update_wrapper
        update_wrapper(decorated, func)
      except (AttributeError, ImportError):
        ... apply func.__name__ etc to decorated by hand ...

You may recall that my fuller example above basic went:

  • set __name__ on my new class and the original, and print them out, which shows the intended values
  • the outer decorator went cls0=cls; cls=binclass(cls0) and printed the __name__s and theywere all the same

And I couldn’t see any mechanism between the 2 places to make this change.

Well, there’s the mechanism right there, above, in my third decorator.

The basic problem is that it looks up func.__name__ after return from the decorator, and thus takes my f'{name0}__original name… and sticks it on the new class!

The fix in @decorator is to gather these things before calling the decorator. And thus to kick functools.update_wrapper to the kerb, because it inherently gathers these things afterwards, not before.

Thank you for prodding me to try to make a self contained example.

1 Like