Meaning of "(im)mutable type"

I am having some trouble understanding this slippery term when I encounter it in CPython source and in messages.

I do not have a problem seeing through statements like “list is a mutable type” when people really mean “instances of list are mutable”. When it comes to type objects, it’s not clear whether there is one concept or two going by this name. Consider:

>>> class L(list): pass
... 
>>> L.__len__ = lambda self: 42
>>> len(L(range(3)))
42
>>> list.__len__ = lambda self: 42
Traceback (most recent call last):
  File "<pyshell#10>", line 1, in <module>
    list.__len__ = lambda self: 42
TypeError: cannot set '__len__' attribute of immutable type 'list'

So L (the type) is mutable in the sense that I can change the behaviour of L objects by changing L itself, and list doesn’t let me do that.

Now consider:

>>> class L2(list): pass
... 
>>> len(x:=L2(range(3)))
3
>>> x.__class__ = L
>>> len(x)
42

So instances of L2 allow their type to be replaced, leading to a change of behaviour to that of their new type. There are limits to that:

>>> x.__class__ = list
Traceback (most recent call last):
  File "<pyshell#24>", line 1, in <module>
    x.__class__ = list
TypeError: __class__ assignment only supported for mutable types or ModuleType subclasses

These prohibitions are enforced in practice by the flag Py_TPFLAGS_IMMUTABLETYPE.

  1. We are allowed to assign L.__len__, because this flag is clear in L, and not allowed in list because it is set.
  2. We are allowed to replace L2 in x.__class__ with L, because the flag is unset in both, but not with list where it is set.

Call the first “mutable” (the type admits modifications) and the second “replaceable” (the types in a certain group can be swapped on instances). I can imagine a replacement group of types, some or all of which were mutable.

Are these necessarily the same concept, for some reason I’m not seeing?

For the reason that types are objects. :wink:

When you write

class L(list): pass

the name L refers to an object, which is the class. Because it’s a class (was created with the class keyword), it’s a type (an instance of type). It is roughly equivalent to:

L = type('L', [list], {})

type, naturally, has a unique property: it is an instance of itself.

I know, but I do not think this explains the co-incidence of the mutability of types and their replacability on their instances. A type is not mutated by assignment to __class__ in an instance.

Ah, I think that error message is a bit misleading. It makes sense to be able to swap around the __class__ of instances of user-defined classes, because they’re all fundamentally based on the same memory layout, with a __dict__ and everything. But list is a builtin type, implemented in C, which requires a specifically different memory layout under the hood. If you could tell Python that x is a list and then try to index into it, Python would expect to be able to look directly in some C-structured data, which would be rather Bad when the data isn’t actually structured that way.

The memory layout can’t be the reason directly: The memory layout of a subclass of list is compatible with list (otherwise the list methods wouldn’t work after all). Yes, potentially there are extra offsets but with __slots__ = () in the class definition they can be eliminated, meaning the only difference in the binary representation of L() and [] is that one points to list and the other to L: Why I am not allowed to change that?

x is a list alright.

>>> x = L2(range(4))
>>> isinstance(x, list)
True
>>> x[2]
2

And yes, the reason I shouldn’t be able to assign its __class__ to list is the presence of __dict__ in x and not in [].

On second thoughts (edit): I decided it would be sort of ok to assign x.__class__ = list: it would just make the instance attributes inaccessible. But it would be weird, and it would not be possible (just from the type) to recognise that it is actually safe to assign x.__class__ back to L2 and not so with some other list.

I don’t have the knowledge to answer, but perhaps this comment above the condition that leads to that error message can clarify. This issue seems to be the origin of the condition for allowing setting __class__.

From what I understood, what you call “replaceable”, which I am assuming is being able to set its __class__ or being able to set as the value of that, is not the same as the negation of being “immutable” (given by the flag Py_TPFLAGS_IMMUTABLETYPE). The former also allows the case of being a subtype of ModuleType. Another thing that I got from the comment linked above is that this implementation-defined notion of “replaceable” could be made wider. The current choice being a conservative choice.

1 Like

There is a lot in the bpo issue, thanks. Bookmarked for later. I found this Use Py_TPFLAGS_IMMUTABLETYPE in __class__ assignments check · Issue #88139 · python/cpython · GitHub which references it, but didn’t track the discussion back.

I’ve studied the code and the comment and note the PyModule_Type exclusion, which it didn’t seem worth complicating the question with. A quick look does not identify for me any subtypes of module or modules where Py_TPFLAGS_IMMUTABLETYPE is set so that the exception is necessary, but maybe it is set in code “by policy”.