Why do classes have a plain dictionary, rather than a mappingproxy, or some sort of immutable dict?

Core Development
1

The dictionary of a class is a normal dictionary, but we protect it with a mappingproxy:

>>> class C: pass
>>> type(C.__dict__)
<class 'mappingproxy'>

Unfortunately this is fragile, and adds much complexity elsewhere.

E.g. We have to dance around the normal behavior of descriptors, otherwise C.__dict__["__dict__"].__get__(C) would return the underlying dictionary. See subtype_dict() and subtype_setdict() in typeobject.c.

There is a way to access an underlying mapping in MappingProxyType · Issue #88004 · python/cpython · GitHub is arguably a security issue, yet was closed as “won’t fix”.

If the class’s dict were an immutable dict, then these problems would (mostly) go away.

So, why is it this way? Historical reasons, or is there something else?

Is the class of a class’s dict part of the API, or an implementation detail?
In other words, is it OK if type(C.__dict__) != mappingproxy?

2

At least in the Python2 days it was not uncommon to add additional methods to classes dynamically, e.g. for parsing purposes.

For Python2 I also had a tool which “froze” sub-classes to gain more speed. The tool took the class and walked the inheritance tree, adding all the methods it found to the sub-class’ .__dict__. This enhanced method lookup speed somewhat and made a difference esp. for mix-in class setups.

3

Nothing has changed there.
The problem is that Python prohibits direct mutation of a class’s __dict__, but class dictionaries are mutable dicts.

class C: pass
C.meth = lambda self: ...  #OK
C.__dict__["meth"] = lambda self: ...  #Fails

Because C’s dictionary is a dict we have to jump through hoops to prevent C.__dict__["meth"] = ...

4

Well, in Python2, you can directly assign to the class’ dict (there’s no mappingproxy trying to prevent this). This is no longer possible in Python3.

However, since setattr(C, "meth", meth) still works, missing direct write access to C.__dict__ can be worked around in Python3.

5

I hadn’t realized that this was a Python 3 feature. That goes someway to answering my question, thanks.

6

this is not quite correct, it’s only true for old-style classes. New-style classes always had immutable mappingproxies as their dicts, I think:

$ python
Python 2.7.18 (tags/2.7:8d21aa21f2, May  5 2021, 11:49:47) 
[GCC 10.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A: pass
... 
>>> A.__dict__['a'] = 1
>>> class A(object): pass
... 
>>> A.__dict__['a'] = 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'dictproxy' object does not support item assignment

PyPy experimented at some point (maybe 2008?) with making new-style class dicts be of type dict, but have those dicts still be immutable. Unfortunately we found out that there is code in libraries that inspects the types of .__dict__ and has special logic to handle mappingproxy (I don’t remember which libraries those were though). So we had to switch to also implementing mappingproxies to be compatible.

3 Likes
7

IIRC, the class dict is protected from direct item assignment due to the way slots for special methods are updated in Python 3. For example:

import ctypes

_PyObject_GetDictPtr = ctypes.pythonapi._PyObject_GetDictPtr
_PyObject_GetDictPtr.restype = ctypes.POINTER(ctypes.py_object)
_PyObject_GetDictPtr.argtypes = ctypes.py_object,

class C:
    pass

d = _PyObject_GetDictPtr(C)[0]

Assigning a special method directly to the dict doesn’t actually update the corresponding slot:

>>> d['__len__'] = lambda s: 42
>>> len(C())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'C' has no len()

It has to be assigned as an attribute:

>>> C.__len__ = C.__dict__['__len__']
>>> len(C())
42
3 Likes
8

Yes, that might be a problem. But hopefully less now, that Python 2 is in the past.
If we do change class dicts to be a new type, then types.MappingProxyType needs to be redefined as type({}.keys().mapping) which is fine for code that wants to create read-only views on dicts, but not so good for introspection tools looking for class dicts.

9

@markshannon Do you have a specific proposal, or was this just a fact-finding mission and is your question answered? It is clear that the proxy exists to disallow direct mutation of the underlying dict, but to allow indirect mutation as a side effect of setting an attribute.

If we somehow were to change type(cls.__dict__) I don’t think that would be a hugely breaking change – I’m sure there’s some code out there that depends on this (the type is published as types.MappingProxyType) but I don’t think it would be a show-stopper.

A possible improvement could be to get rid of the proxy and instead install some kind of mutation callback on the underlying dict that does the work that is currently done by the class setattr operation. I’m guessing that the proposed dict callback scheme would make this viable.

I don’t know if we’d have to implement some deprecation warning – I wouldn’t even know what we could warn about – maybe isinstance(x, types.MappingProxyType)? But what alternative would people have until the implementation is changed, other than ignore the deprecation warning?