Non-uniqueness of TypeVar on Python versions <3.12 causes resolution issues

Typing
1

In Python 3.12, the new Generic / TypeVar syntax allows us to do this:

class Foo[T, U]:
    type: T
    info: U

class Bar[T](Foo[str, T]):
    test: T

Which, for older versions is functionally equivalent to this:

from typing import Generic, TypeVar

FooT = TypeVar('FooT')
FooU = TypeVar('FooU')

class Foo(Generic[FooT, FooU]):
    type: FooT
    info: FooU

BarT = TypeVar('BarT')

class Bar(Foo[str, BarT], Generic[BarT]):
    test: BarT

Key point being that each TypeVar is unique to the class.

However, in older versions it is much more common to reuse the same TypeVars for everything, as even some code in PEP 484 showcase doing.

from typing import Generic, TypeVar, get_type_hints

T = TypeVar('T')
U = TypeVar('U')

class Foo(Generic[T, U]):
    type: T
    info: U


class Bar(Foo[str, T], Generic[T]):
    test: T

This however, poses a risk at runtime, which has incidentally been solved by the new syntax. When resolving the types of Bar, test: T will resolve to the same TypeVar as type: T, when we expect type: T of Foo (later becoming str) and info: T of Bar.

I encountered this while investigating a bug with Pydantic not finding a TypeVar with a default (testing out PEP 696) that had its value overridden by an inherited generic alias. The reason was because it was not resolving TypeVar values all the way up, only on the first level (e.g. Bar[bool] meant all T = bool, but no type for U). Initially, I had written a utility function (gist) to find the corresponding overriding values given a set of TypeVars that are being used as a type hint for attributes of the generic class. It works given most conditions, and should work reliably on Python 3.12 as the TypeVars are unique.

But this solution does not reliably work on versions older than that, and I was wondering what could be done to solve this.

Typecheckers (namely pyright and mypy that I’ve tested) on the other hand are able to solve this across all versions.

Let me know if I somehow missed out a key bit that would fix this.

2

I think you’ve simply found a bug in pydantic. For example, this test passes in pyanalyze (which is mostly doing runtime inference in this case):

from typing import Generic, TypeVar, get_type_hints
from typing_extensions import assert_type

T = TypeVar("T")
U = TypeVar("U")


class Foo(Generic[T, U]):
    type: T
    info: U


class Bar(Foo[str, T], Generic[T]):
    test: T


def func(x: Bar[int]) -> None:
    assert_type(x.type, str)
    assert_type(x.info, int)
    assert_type(x.test, int)

This is implemented by going over the base classes and figuring out the type parameters separately for each base class. That’s what type checkers do statically, and it’s also possible with runtime introspection.

2 Likes
4

Hi! Thanks for the reply. What I realised while investigating further was that __annotations__ only shows annotations of that class, and not of inherited attributes. Somehow forgot about that behaviour :sob: This makes it much easier to associate TypeVars to classes with older style / non-PEP 695 generics.