Variance in lists of generics

Python Help
1

Just when I think I understand (co|in)variance…

I have a type hierarchy similar to this:

from typing import Any, Generic, TypeVar


class A:
    ...


class B:
    ...


AB = A | B

T_AB = TypeVar('T_AB', bound=AB)


class G(Generic[T_AB]):
    ...


g1 = G[A]()
g2 = G[B]()
g3 = G[str]()  # type error, as expected

Now I want a list of G.

First question, why is this allowed? Is it by design, or just something the type checkers don’t check?

g_any: list[G[Any]] = []  # no type error?

I settled on that because I initially used this, but I get an error:

# Type parameter "T_AB@G" is invariant, but "B" is not the same as "AB"
g_ab: list[G[AB]] = [g1, g2]

I know I can ‘get around it’ with

T_AB = TypeVar('T_AB', bound=AB, covariant=True)

but I presume there are some semantics I should guarantee if I’m doing to assert that?

2

This is by design, Any is a gradual type and bidirectionally compatible[1], it thus circumvents variance rules when used to bind a TypeVar. list[Any] is also compatible with any list, even though list is invariant. It’s equivalent to omitting the type parameter entirely, so list and list[Any] mean the same thing.

Correct, covariance means instances of your Generic are not allowed to accept any values of T_AB[2] they are only allowed to produce them[3], contravariance is the opposite case and invariance is the mixed case.

A little example to show why this is necessary, using Sequence which is covariant:

x: list[str] = ["foo", "bar", "baz"]
y: Sequence[object] = x  # OK: "str" is a subclass of "object"

# Now let's assume we extended Sequence with an "append" method
# without fixing its variance:
y.append(5)  # OK: "int" is a subclass of "object"

This will break the promise of x only containing str without raising a type error, because as far as the type checker knows everything is fine.

PEP-695 introduced the infer_variance parameter to TypeVar which will do the variance calculation for you, if you’re worried about getting it wrong. Prior to Python 3.12 you can use it by importing TypeVar from typing_extensions rather than from typing, although I’m not sure if all type checkers support infer_variance at this point.

Some type checkers will warn you if there are methods/attributes that contradict your declared variance. mypy does it only for Protocol currently, but I think pyright warns in more places, assuming the corresponding checks are enabled.

Another way to get around variance in your example is to move the Union outside, i.e. G[A] | G[B] instead of G[A | B], this way, even if both the containers are invariant, you can still append either of the two G variants to the list, since they’re both a member of the union.

  1. i.e. it’s considered both a sub and superclass of any type ↩︎

  2. outside of __new__/__init__ or some classmethod ↩︎

  3. so you are only allowed to use it in return types and all the public attributes that store it need to be read-only ↩︎

1 Like