Python as a dynamically typed language offers this powerful feature of efficiently altering the behaviors of an existing object by simply changing its class to a compatible one through the means of setting the object’s __class__ attribute.
Unfortunately, because of a lack of guidelines and framework for typing an object that may change class, people are discouraged from actually using the feature, as evident in the dismissive sentiment in the following sub-topic (full quotes inserted for better context):
So let’s make it type-safe.
My suggestions:
There should be guidelines for static type checkers to treat setting the __class__ attribute of an object in a fashion similar to type narrowing, where the inferred type of an object in the subsequent code path changes.
Type checkers by default should allow an object to set its __class__ only to a subtype of its declared/inferred type, unless,
A class is defined with a new generic MayBecome[T1, T2, ...], in which case an object of the class may set its __class__ to any of T1, T2, etc., or one of their subtypes.
So in the contrived example above, it may be rewritten as:
class Child(Human, MayBecome[Parent]):
def grow_up(self):
self.__class__ = Parent
or it can be made more accommodating by specifying a base class:
class Child(Human, MayBecome[Human]):
def grow_up(self):
self.__class__ = Parent # OK because Parent is a subclass of Human
while this is not OK:
class Child(Human):
def grow_up(self):
self.__class__ = Parent # error: Parent is not a subclass of Child
I think a new special form MayBecome is too much. This is a rarely used feature. I would much prefer if type checkers used the actual rules for when assignment is valid (AFAIK, compatible layout, very closely related to the disjoint_bases PEP) and then either
assume correctness and trust the user (always necessary to some degree)
verify that all attributes needed for new_class are defined on the object with compatible[1] types.
But a type checker can’t know when grow_up is called, it depends on the runtime path, and hence type checker can’t know the type. You need to guard it with an if:
from typing import Optional, Never
class Child:
def grow_up(self):
# Adult isn't defined - hence suppress type error.
self.__class__ = Adult # type: ignore
def is_grown_up(self) -> Optional['Adult']: # Can't use 'Adult' | None and Adult isn't defined yet! Hence Optional.
return self if isinstance(self, Adult) else None
class Adult(Child):
def __new__(cls) -> Never:
raise ValueError("Adults cannot be instantiated directly. Create a Child and let them grow up!")
# New fields, not in Child, cannot be directly added, they need to be added via setters and getters.
@property
def voted(self) -> bool:
return self._voted if hasattr(self, '_voted') else False # Need to guard against no call to setter yet.
@voted.setter
def voted(self, value: bool):
self._voted = value
if __name__ == "__main__":
human = Child()
print(f"Human is a {human}.")
human.grow_up()
if adult := human.is_grown_up():
print(f"Human has grown up and is now an {adult} and they {adult.voted = }.")
Which prints:
Human is a <__main__.Child object at 0x1027457f0>.
Human has grown up and is now an <__main__.Adult object at 0x1027457f0> and they adult.voted = False.
Showing that class has changed without creating a new object, same ID, and that you can access a field added to the new class.
Maybe the feature is rarely used because of the lack of typing support, though it’s admittedly hard to figure out the percentage of code out there that could be potentially improved by switching to this pattern from one of the alternatives.
Very true. Those are good rules that a type checker can follow to determine if a type is compatible with an object so there may not be a need for a new special form for manual declaration of compatible types for the feature to be type-safe.
True. There’s only so much a static type checker can reasonably infer without actually running the code so a type guard is still needed for objects that may or may not change class depending on run-time inputs.
I think there’s a subtle difference between this and narrowing. Currently what a type checker tracks is the types of names (both what types can be assigned to them and the type(s) of the object they currently reference). Narrowing modifies the inferred type of a name. But what this idea would need is to track the types of objects themselves. This is inherently more complex because names are local to whatever scope they’re in and you can have fairly straightforward reasoning about their contents. But objects themselves are more or less untrackable. A type checker can’t see where an object comes from and where it may be manipulated in a way that changes its type.
the type checker can’t tell if it was called as grow_together(child, child) or grow_together(child, other_child). So it’s not possible to know whether the type of second is supposed to have changed or not.
Similarly, any code that calls a function can’t tell whether that function internally sets __class__. We also cannot know which local or global names might be pointing to objects that are also referenced by some name in that function call. So after any function call, a safe type checker would have to infer the type of every name in all scopes to be object.
I think that code that modifies __class__ just is too dynamic to be tracked by type checkers. The best way to deal with this is to either only reference objects whose class might change by some common protocol/parent and to guard access to class specific features with isinstance checks.
I think it’s good that you found examples and support for this “feature”, but I suspect that the standard library uses this feature mainly for an extreme attempt at computational efficiency. I don’t think this is good code. As I said in the other thread, it breaks assertions:
def f(a: A):
assert isinstance(a, SubA)
g()
assert isinstance(a, SubA) # May no longer be true!!
And for the same reason, it violates static type checker assumptions. Trying to broaden a type with MayBecome is much worse than just writing the code in a more ordinary way.
My $0.02c: in my decade of writing Python professionally I have never used this, and if I saw it in a PR I would strongly push back on it. I think the limited efforts that we as a community expend on pushing typing forward would be much better used evolving the typing spec in other, better directions.
One of the most fundamental rules of type theory is that types don’t change. This is possible to do at runtime, but is impossible to safely model in type-systems. Because if __class__ would be made mutable, then it must be invariant, meaning that subtypes are no longer assignable to their supertypes.
There’s generally not a good reason to use this mechanism in modern code.
This would essentially make it so anything that “may become” another type must be a persistent union with that other type no matter what other knowledge is gained, and invalidates things like narrowing (which is at least partially specified via TypeIs).
if it’s important to you that this is modeled for any existing use, you can accomplish it with a structural type that represents what is always safe to assume, but it’s somewhat awkward.
I wouldn’t recommend any new use of this pattern, even if it does exist in the standard library and is allowed, even if it wasn’t difficult to type accurately.
In the Java world this is called Hot Swapping and has a number of uses and is officially supported via both the Debug Interface and by Java Agents. Debuging live code would be an obvious use case. Java Agents are useful when you have permanently up code, some sort of Server, and want to update a class without a re-boot.