I have two similar uses of a generic class, one passes, the other one fails.
It seems the assignment makes the difference. But I don’t understand why nor where this is documented.
from typing import Generic, TypeVar
K = TypeVar('K')
class Animal:
...
class Dog(Animal):
...
class Box(Generic[K]):
def __init__(self, animal: K):
self.animal = animal
...
def feed(box: Box[Animal]): ...
# Type checks on mypy and pyright
feed(Box(Dog()))
# Fails to type check on mypy and pyright
mybox = Box(Dog())
feed(mybox)
# Mypy: Argument 1 to "feed" has incompatible type "Box[Dog]"; expected "Box[Animal]"
# Pyright: Argument of type "Box[Dog]" cannot be assigned to parameter "box" of type "Box[Animal]" in function "feed"
# "Box[Dog]" is incompatible with "Box[Animal]"
# Type parameter "K@Box" is invariant, but "Dog" is not the same as "Animal"
this is a good problem. Would it be because in the first line, you are passing in a type and in the second you’re passing an instance of a type? As you know, classes are considered a type in their own right. So, if we type the following:
type(str) # type string for example
we get:
type
If we create a class:
class Tester:
pass
obj = Tester()
type(obj)
>>> __main__.Tester
The object obj is not a type. It is an object of a type of a class that you instantiated.
In other words, they are not equivalent in that obj is not a type as is the case for a class with inheritance (or without) which is.
is not an instance. It is a class with inheritance.
mybox is an instance of that class.
They are two different types of argument. For example, you can create additional objects (i.e., instances) with the class Box(Dog()) but you cannot create additional objects with mydog.
This is an interesting example. I think what you see is an interaction between type inference and variance.
Here a typechecker will infer the type of mybox to be Box[Dog].
If you change that line to this, it will typecheck:
mybox: Box[Animal] = Box(Dog())
The reason is that TypeVars are invariant by default. That is, a function accepting Animal will not also accept Dog, unless you declare K to be covariant. This is what the error you get from pyright tells you:
In this case, it seems that typecheckers are able to infer the type of the expression Box(Dog()) to be Box[Animal] based on how it is used, which is why you don’t get the error.
Another place you will encounter this sometimes is when annotating the built in list type with an inheritance hierarchy like you have here, because list is treated as invariant (while abcs/protocols like Sequence and Collection aren’t).
The Wikipedia article on Covariance and Contravariance is pretty thorough, but personally I can never keep them straight in my head and often get them reversed. If anyone has found more intuitive explanations somewhere do share!
Thanks for the input, I think understand the root cause now and can add a bit to the replies.
The real reason
mybox = Box(Dog())
feed(mybox)
is not safe is because feed may mutate the input box. For instance, it could do
def feed(box: Box[Animal]):
box.animal = Animal()
mybox would be typed as Box[Dog] but after feed it would really be a Box[Animal], this may lead to unexpected errors since not all animals will have an e.g. bark method. This is not a problem in the first example, because we are not keeping a reference to the object. Note that the call itself feed(Box(Dog())) is safe since after all a dog is an Animal.
It’s a bit misleading to conclude that the type checker is “being more lenient”.
First, it’s important to understand that the type of an object (including its generic type arguments) is established at the time it is constructed. The expression Box(Dog()) constructs a new Box object, but its type can be evaluated (safely) as Box[Dog], Box[Animal], Box[object] or any number of other compatible types. The important thing is that its type is fixed after construction. The operations you can safely perform on that object will be different based on whether its type is Box[Dog], Box[Animal], etc.
By default, type checkers will assume that Box(Dog()) should evaluate to Box[Dog]. However, there are ways to influence the type checker’s evaluation behavior by providing additional context. This is sometimes referred to as bidirectional type inference.
Let’s look at how this applies to the expression [Dog()].
# No context is provided in this case, so the
# list expression is inferred to be `list[Dog]`.
d1 = [Dog()]
# Here additional context is provided in the form
# of a declared type for variable `d2`, so the
# list expression is inferred to be `list[Animal]`.
d2: list[Animal] = [Dog()]
# Similarly, `d3` has a declared type, so the
# list expression is inferred to be `list[Dog | Cat]`.
d3: list[Dog | Cat] = [Dog()]
# Here additional context is provided in the form
# of a type annotation on the `elems` parameter
# in the `dostuff` function (from your sample above),
# so the list expression is inferred to be `list[Animal]`.
dostuff([Dog()])
Thanks, that makes sense. However, if im allowed to nitpick a bit, from the pyright docs it seems we may derive the inferred type from the whole scope, not just the assignment expression (the example with the two branches of the if statement). So one could also think that based on usage, mybox should be inferred as Box[Animal]. I guess one needs to know that pyright does not do that, which is probably the safer thing to do.
If anyone has found more intuitive explanations somewhere do share!