When MyType[T] is a generic type alias, MyType[int] is a TypeAliasType structurally but not nominally. Why?

Typing
1

I’ve stumbled over this situation recently; isinstance claims that a Generic Type Alias is not a TypeAliasType when subscripted, though in practice it actually is by virtue of having all the members of a TypeAliasType.

>>> from typing import Annotated
>>> type MyInt = Annotated[int, "test"]
>>> type MyType[T] = Annotated[T, "test"]

>>> type(MyInt)
<class ‘typing.TypeAliasType’>
>>> type(MyType)
<class ‘typing.TypeAliasType’>
>>> type(MyType[int])
<class ‘types.GenericAlias’> # <----- Not a TypeAliasType
>>> type(list[int])
<class ‘types.GenericAlias’>

>>> from typing import TypeAliasType
>>> isinstance(MyInt, TypeAliasType)
True
>>> isinstance(MyType, TypeAliasType)
True
>>> isinstance(MyType[int], TypeAliasType) # <----- Not a TypeAliasType
False
>>> isinstance(list[int], TypeAliasType)
False

>>> MyInt.__value__
typing.Annotated[int, 'test']
>>> MyType.__value__
typing.Annotated[T, 'test']
>>> MyType[int].__value__
typing.Annotated[T, 'test'] # <----- Actually a TypeAliasType?
>>> list[int].__value__
Traceback (most recent call last):
  File "<python-input-28>", line 1, in <module>
    list[int].__value__
AttributeError: type object 'list' has no attribute '__value__'. Did you mean: '__le__'?

That means in practice, when analysing a Generic Type Alias, I need to work around the type-checker, because the only way to check whether something is actually a Generic Type Alias is to do this:

if isinstance(t, GenericAlias) and hasattr(t, "__value__"):
    t = cast(TypeAliasType, t)
    # Now I can work with t.__value__ or other members

I get that MyType[int] is a syntactically a GenericAlias, but semantically it’s not. It’s also in a weird limbo state because MyType[int].__value__ returns typing.Annotated[T, 'test'], meaning something like type(get_args(t.__value__)[0])) returns typing.TypeVar, which indicates that t is generic and must be made concrete by passing a type, which is wrong. It would be better if MyType[int].__value__ already returned the concrete value typing.Annotated[int, 'test'].

I’m wondering if this genuinely a bug or if I’m maybe misunderstanding something about the intent of the type system here?

2

This is a little difficult to understand because of some terminology.
I’m not sure what it means to be “syntactically a GenericAlias” or to be a GenericAlias “semantically”.

It looks like it might be made more clear by talking about “nominal” and “structural” types as these terms are normally used in the type system.

I thought that maybe you’re saying that MyType[int] is a nominally a GenericAlias, but structurally it’s not. But that’s not true because structurally, every python object is a GenericAlias because, according to the documentation, GenericAlias doesn’t have any members. A type doesn’t tell us that a value doesn’t have members - it only tells us that it does have members.

So it looks like you’re trying to say this:

MyType[int] works like a TypeAliasType structurally. So why isn’t it a TypeAliasType nominally?

I think that would be a good question.

1 Like
3

Thank you Doug!

Indeed, that is exactly my question! I’ll change the title to make this clearer. EDIT: Ah, it seems editing older posts is disabled. That makes sense.

4

It seems like I can edit the title. Maybe it is a new user forum control. What do you want to change the title to?

1 Like
5

“When MyType[T] is a generic type alias, MyType[int] is a TypeAliasType structurally but not nominally. Why?”

6

The title is now updated. It is indeed an intruiging question, but not one that I am qualified to answer :sweat_smile:

1 Like
7

The behaviour you’re seeing is a somewhat confusing result of how generic types and type aliases are implemented. So let’s start by just looking at generic types on their own:

If you just write a: int then you’re telling your type checker that you want a to only ever refer to int objects. And when you introspect this annotation at runtime you’ll see that the annotations dictionary contains the builtin int type object that we’re all so familiar with. So far, so expected. But what actually happens when you write b: list[int]? A type checker can just look at the AST (or whatever else it’s using internally) and see that your intention is that b should contain lists that only contain int objects. So we want the runtime annotations dict to contain some object that tells us that. It can’t just be the list object since then we can’t know that it should be lists of ints specifically.

This is what GenericAlias is for. It’s a wrapper object that more or less contains two fields: __origin__ which points to the generic class that was parametrized and __args__ pointing to the type parameters. So our annotations dict can now map "b" to an object GenericAlias(__origin__=list, __args__=(int,)) and we can introspect all of the information of the annotation at runtime.

But now the question becomes “If the object created by MyType[int] isn’t actually a TypeAliasType, why does it behave so much like one?”. The answer to that is that GenericAlias basically forwards almost all behaviour to the __origin__. That is, if you write list[int](...) the behaviour you’ll see is basically the same as with list(...) because GenericAlias just forwards the call to list. And the same is true for MyType[int].some_cool_attribute. The idea behind this is that we often need to combine the type hinting subcripting with accessing some attribute or method on a generic class. It’s just much more convenient if list[int] behaves mostly like list.

So now to the type alias part of the question, namely __value__. That’s one of the attributes that TypeAliasType has and it points to the value of the type expression you’re aliasing. So the reason you’re seeing it on MyType[int] isn’t because it’s a feature of GenericAlias, but because the attribute access is being forwarded to the TypeAliasType object. This explains why MyType[int].__value__ is Annotated[T, "test"], you’re accessing an attribute on an object that has no knowledge of the fact that you parametrized the generic.

Another annoyance of all of this is that different Python versions and even different type objects within in the same version use slightly different variants of these wrapper classes. So it’s best practice not to check for them directly, but use the helper functions in the typing module. AFAIK there’s also no builtin way to get the Annotated[int, "test"] object that you want directly. You’ll have to manually perform the type variable substitutions yourself while you introspect the type annotation.

So instead of writing

if isinstance(t, GenericAlias) and hasattr(t, "__value__"):
    t = cast(TypeAliasType, t)
    # Now I can work with t.__value__ or other members

what you probably want is something more like

from typing import get_origin, get_args

if get_args(t):
    # now we know that `t` is some flavour of specialised generic
    t = get_origin(t)
    args = get_args(t)
else:
    args = ()

if isinstance(t, TypeAliasType):
    params = getattr(t, "__type_params__", ())  # this attribute stores the tuple of type variables
    if len(params) != len(args):
        raise ValueError("type alias isn't fully specialised")
    param_map = dict(zip(params, args))
    t = t.__value__
else:
    assert len(args) == 0
    param_map = {}

# now you can introspect `t` however you need and whenever you encounter a `TypeVariable` you look up its meaning in `param_map`
8

Thank you so much for the write-up, that has helped me understand this a lot better! Though it seems to me that resolving a fully specialized generic type alias to a fully specialized type is something that would be both quite useful and difficult to implement. The simple code you’ve shown is far from sufficient to cover anything but the most trivial cases. Which is fine if you only have a small set of types to expect, but for my usecase that really isn’t sufficient.

The easiest I could come up with is this (sans proper handling of errors as well as ParamSpec and TypeVarTuple):

def resolve_generic_type_alias(
    t: GenericAlias, param_dict: dict[str, Any] = EMPTY_DICT
) -> type | TypeAliasType:
    type_params = cast(type, t).__type_params__
    type_args = get_args(t)

    # Alias is already fully specialized
    if not type_params and not any(isinstance(arg, TypeVar) for arg in type_args):
        return cast(type, t)

    origin = get_origin(t)

    # Keep a running mapping of type variables to be able to resolve
    # variable parameters in nested GenericAliases
    param_dict |= dict(zip((p.__name__ for p in type_params), type_args, strict=False))

    # Resolve TypeAlias if necessary (type-checker thinks this can never happen)
    if isinstance(origin, TypeAliasType):  # pyright: ignore[reportUnnecessaryIsInstance]
        type_args = get_args(origin.__value__)
        origin = get_origin(origin.__value__)

    resolved_args: list[Any] = []
    for alias_arg in type_args:
        # Resolve nested generic aliases
        if isinstance(alias_arg, GenericAlias):
            resolved_args.append(resolve_generic_type_alias(alias_arg, param_dict))
            continue

        # If it's not a TypeVar, alias_arg is a concrete value or type
        # that doesn't need to be resolved any further
        # (this is where ParamSpec and TypeVarTuple would have to be handled)
        if not isinstance(alias_arg, TypeVar):
            resolved_args.append(alias_arg)
            continue

        # Find the concrete value or type that has been set for the TypeVar
        resolved_param = next(
            param for name, param in param_dict.items() if name == alias_arg.__name__
        )
        resolved_args.append(resolved_param)

    # Actually specialize the type
    final_args = tuple(resolved_args)
    return origin[final_args]

This function allows me to get the fully specialized type as an actual type object I can then inspect further:

>>> from resolver import resolve_generic_type_alias
>>> type MyType[V, K] = Annotated[dict[K, V], 2, list[V], 4]
>>> x = MyType[int, str]
>>> print(resolve_generic_type_alias(x))
typing.Annotated[dict[str, int], 2, list[int], 4]

I understand that it’s not really feasible for GenericAlias.__value__ to just lazily resolve to this (and that would be a breaking change), but I do feel that something like GenericAlias.resolved_value() or something in that vein that is guaranteed to be up-to-date with all typing constructs in the language should be in the standard library.

I would assume type-checkers must implement a function similar to this already, but I haven’t yet found an actual implementation.

9

Most static type checkers do not look at the actual runtime type objects; indeed, most major Python type checkers are not written in Python. (The exception is my pycroscope tool, but it is not widely used.)

Runtime type checkers do have to deal with this sort of issue. They will generally have code similar to what you posted. I am generally resistant to adding too much code in the standard library that can be implemented in user code instead: a third-party library is more easily updated and doesn’t burden the CPython core team.

10

Ah, that does make sense.

Indeed, I’m kind-of building a runtime type-checker for my application. It’s an extension to pydantic for immutable data structures with cheap creation of modified copies via implicit sharing/copy-on-write semantics and for that I have to verify that the types of all attributes of a class are immutable, else the entire concept breaks down.

My thought was that because the semantics of runtime type objects can be amended and changed, having the functionality in stdlib would guarantee that it stays consistent, but I concede that this is probably too niche to justify the additional maintenance burden for the core team.

11

If you’re interested, I have some ongoing work related to this as part of the typing-inspection library. I’m planning on picking it up again soon.

1 Like
12

Very interesting! From what I can tell, introspection.inspect_annotation(t, unpack_type_aliases="eager") is doing what I’m looking for, correct?

What do you mean by “ongoing work”? Are there edge-cases you’re not covering yet?

13

For inspect_annotations(), it only expands PEP 695 type aliases if it contains Annotated or type qualifiers (e.g. Required/NotRequired). And the logic is not quite right, e.g.:

type A[T1, T2] = Annotated[dict[T2, T1], ...]

introspection.inspect_annotation(A[int, str], unpack_type_aliases='eager')
# InspectedAnnotation(type=dict[int, str], qualifiers=set(), metadata=[Ellipsis])

But the goal is to properly reuse Add type hint parsing utilities by Viicos · Pull Request #39 · pydantic/typing-inspection · GitHub.