Run-time behaviour of `TypeAliasType

Typing
1

There was a heated discusson in another thread regarding a difference in run-time behaviour of typing.TypeAlias and the new TypeAliasType introduced in PEP 695.
In short, we disagreed on whether TypeAliasType should be a “transparent reference” to a “concrete type”.
“Concrete type” here means a type that you can subclass, instantiate/construct, and use with isinstance and issubclass checks[1].
“Transparent reference” here means being able to use a TypeAliasType as a “concrete type” if it is an alias for a “concrete type”.

typing.TypeAlias run-time behaviour

If typing.TypeAlias is used to to mark a “concrete type” as an alias, it will not change the run-time behaviour of the alias, as the typing.TypeAlias only affects static type checkers:

>>> import typing
>>> OldAlias: typing.TypeAlias = str
>>> class SomeSubclass(str):
...     ...
... 
>>> issubclass(OldAlias, str)
True
>>> isinstance("abc", OldAlias)
True
>>> test = OldAlias()

TypeAliasType run-time behaviour

TypeAliasType is, afaict, only meant to be used by static type checkers, so it cannot be used as a “concrete type”:

>>> type NewAlias = str
>>> class SomeSubclass(NewAlias):
...     ...
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: typealias() takes exactly 2 positional arguments (3 given)
>>> issubclass(NewAlias, str)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: issubclass() arg 1 must be a class
>>> isinstance("abc", NewAlias)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
>>> test = NewAlias()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'typing.TypeAliasType' object is not callable

What’s the problem?

The previous thread had two conflicting opinions that were both concluded to be valid, which are:

  • TypeAliasType is meant for type checker use and its design should not be restricted by run-time behaviour.
  • TypeAliasType is a replacement for the old typing.TypeAlias and should work the same at run-time.

IMO, neither PEP 695 nor the typing spec clarifies which of these are correct.
Thus, It’s probably for the best if the typing spec clarifies this.

What choices do we have?

I currently see three options, but there are probably more than I can think of at the moment.
I will label these options as A, B, and C.

  • Option A is to not restrict TypeAliasType and continue with the deprecation of typing.TypeAlias with eventual removal[2].
    This option would not affect the type system nor complicate the design of TypeAliasType, but it would force users of run-time type tools to choose between TypeAliasType or doing normal assignment without a way to explicitly mark an identifier as a type alias.

  • Option B is to not restrict TypeAliasType and undeprecate typing.TypeAlias.
    This would not affect the type system but still leave users of run-time type tools with the ability to mark an identifier as a type alias.

  • C would be to extend TypeAliasType to be used as “transparent references” of “concrete types” and continue with the deprecation of typing.TypeAlias with eventual removal.
    This would unify run-time and static-time type aliases but complicating the design of TypeAliasType.

My opinion

While I am not particularly well versed in the theory of types, from my perspective as a user of type annotations both for type checking and documentation, I’d prefer Option C.
A type alias is just another name for a type and thus should be able to be used as that type.
Also, typing.TypeAlias is a nice way to document an identifier and as things currently stand this way to document aliases is discouraged due to it being soft deprecated.

There is precedence in to add similar functionality where it makes sense, like how type unions support isinstance checks despite being a special typing construct[3]:

>>> UnionType = int | str
>>> isinstance(1, UnionType)
True
>>> issubclass(int, UnionType)
True

Looking at this, I would argue that we have already recognized that there needs to be some run-time support for special typing constructs, and adding it to TypeAliasType makes a lot of sense to me.

  1. I will admit that there are probably more cases where one might want TypeAliasType and “concrete types” to behave the same but these were the ones brought up in the other thread. ↩︎

  2. I know it’s currently soft deprecated (even though the documentation currently doesn’t use that term) but ISTM that if we’re serious about replacing typing.TypeAlias it should be scheduled for removal at some point. ↩︎

  3. However, unions cannot support instantiation or subclassing since it’s ambiguous which choice should be used. ↩︎

3 Likes
2

I’ll summarize my stance from the previous thread:

typing.TypeAlias behaving like a regular assignment is entirely incidental[1] and I don’t think it was ever intended to be used to mark aliases to nominal types, since that case is entirely unambiguous.

Its main purpose was to resolve ambiguities in analyzing type annotations, since a regular assignment will leave no trace behind, especially once you refer to a type alias inside another type alias, so you can’t tell there was a type alias in between. This is especially relevant for scoping of type parameters, once you try some type analysis at runtime, rather than just statically. So in many ways the runtime behavior of typing.TypeAlias is actually insufficient if you truly want to be able to make use of some typing features at runtime.

Making TypeAliasType fully transparent is impossible, because at that point it would just be a regular assignment again. I think making TypeAliasType work with isinstance/issubclass can make some amount of sense, but I don’t think it makes sense to add complexity to subclassing logic in order to support the incredibly small set of type aliases that are actually subclassable.

The other reason I’m not a fan of using typing.TypeAlias to annotate an alias to a nominal type, is because it hides errors as soon as you decide to replace that alias with a string forward reference. The type checker will pretend it will have the runtime type in that case, whereas in a regular assignment it would detect that the variable contains a string and cannot be subclassed:

Safe = "str"
Unsafe: TypeAlias = "str"

class Foo(Safe): ...  # type error

class Bar(Unsafe): ...  # type checker assumes this is fine

Runtime use of TypeAliasType is still perfectly viable and in fact it’s more powerful than typing.TypeAlias. If you need to treat a TypeAliasType like a typing.TypeAlias you have the __value__ attribute which contains the value of the type alias, so it’s still possible to do the following[2]:

type StrList = list[str]

class Foo(StrList.__value__):
    pass

  1. it behaves this way because that’s all it is ↩︎

  2. although I wouldn’t encourage this pattern ↩︎

2 Likes
3

For this post static type checking behavior refers to pyright as I think only it supports pep 695 at moment. For code I commonly work with due to usage of libraries similar to pydantic/cattrs/typeguard any type alias has a high chance of being used at runtime. Isinstance being more commonly needed while subclassing being done occasionally. usually for generic issues like stub is generic but runtime is not generic where you need alias to do subclass. This example is even recommended by mypy documentation here, Annotation issues at runtime - mypy 1.8.0 documentation (using classes generic in stub but not runtime).

Let’s examine proposed solution of using A.value a bit more.

First this works at runtime but not at static type checking time for pyright. Pyright does not follow value and treat it like underlying type. Pyright even gives an error on that line of code saying expected type found property expression instead.

Next it’s not just a few places like subclasses. Libraries like pydantic/cattrs inspect runtime type annotations of classes. Many of classes in my codebase use similar library. Several hundred classes. Any of their field/constructor attributes can be runtime type inspected with get_type_hints and then have isinstance/similar applied. All of those libraries now either need to have special logic for these new aliases or users have very unobvious footgun bug. And similar to the prior example runtime behavior may be happy if you use value but if you use it bunch type checkers will likely be confused.

Instead if I had to use new style aliases I would likely end up using TYPE_CHECKING and have it be alias for static type system but value for runtime type system. That does not feel fun to explain to my teammates. So in practice if we decide that new style type aliases should not interact well with runtime similar to old style aliases I will likely just strongly discourage new style aliases being used at all. The runtime usage is often not obvious given codebase that heavily uses runtime tools.

Lastly pyupgrade is a tool that tries to rewrite code to avoid older styles/deprecated classes. Support for rewriting type aliases was considered here, --py312-plus: `type ...` aliases · Issue #836 · asottile/pyupgrade · GitHub, but given up after realizing backwards compatibility with runtime behaviors was poor and likely to lead to a bunch of crashes.

1 Like
4

First of all, thanks for writing up this summary! I had been watching that other thread unfold and I think you did a good job of capturing the major points.

The last part of this statement is where I experience the most dissonance regarding the use of type aliases.

I think something important was not discussed in the original thread in enough depth, and perhaps we can open it up here:

Users want to mark an assignment, which will be used as a type at runtime, as a type alias. But why? That is, why do they want to annotate this assignment as a type alias rather than annotating it as a type?

The following assignments are valid today, and each is slightly different:

type x = str
y: type = str
z: TypeAlias = str

y and z are identical at runtime, x is not.

It seems to me that y: type = str has been underappreciated as an option here. It protects against badness:

w: type = int | str  # error! UnionType assigned to type!
2 Likes
5

If you use A: type = str, and use A as an annotation on function type checkers will give you an error as they now thing A is variable instead of an alias. For type checkers alias that are safe to use as annotations vs variables are distinct concepts. In fact here if you write just A = str then hopefully all type checkers assume you intended alias valid as annotation. But if you write A: type it’s assumed to be variable and you can no longer use it in annotation contexts (at least testing with pyright).

For more complex aliases and boundary here is vague as type checkers use different heuristics when you don’t specify explicit alias, it becomes harder to tell is this intended to be valid annotation or variable. That’s main purpose of using TypeAlias.

Currently on my phone but you can try it in pyright playground and see errors pop up when you attempt to use A: type = str in most annotation contexts (function’s type hint or attribute type hint).

Edit: In general there are two “worlds” involved here. What does code do at runtime vs at static type checking type. Type checkers usually do not run your code and have need for identifying aliases separate from other code. From my view ideally what works well for static type checker should also interact well at runtime (isinstance, issubclass, and subclassing).

Edit 2: Your union example is also interesting. Is int | str a type is I think not well defined in type system today especially when TypeForm, the annotation for any thing valid as type annotation is proto-pep at moment. So whether type checker only allows things that are true subclasses of type as a type vs other special forms is inconsistent and varies even across recent type checker versions.

1 Like
6

Thanks for pointing that out! It also fails in mypy – I had only been checking the subclassing case, which works fine with it (as it should).

Reading more about how mypy treats something of type Type, it seems that using : type annotations isn’t feasible. Things like this are allowed:

x: type = int
if foo():
    x = str

which is simply not going to fly for static analysis.

I still think that the question around why users want their runtime assignment annotated as a TypeAlias has bearing.

My presumption is that they want the safety it offers:

x: TypeAlias = 1  # fails

So perhaps the real desire here is for something which must be a valid type at runtime. Therefore, it should forbid strings and other non-runtime-type values.

e.g. Imagine:

x: RuntimeTypeAlias = str
y: RuntimeTypeAlias = str | int  # fails!
7

That’s not my main motivation for using it. I mostly use it so type checkers understand the type should be kept as valid for annotations. Later when I’m home I’ll try to find some more complex annotations where missing explicit alias causes type checker to be confused and have wrong behavior. What is considered complex may be different per type checker so if you want something compatible with multiple that motivates most aliases to be explicit.

Also many of aliases I use are not valid runtime types in sense you are looking at. Union/special forms like literal are often used in my aliases and intended. Most type aliases I have tend to be used in get_type_hints/annotation/isinstance contexts. Even strings are often fine in contexts like annotation or using typeguard/pydantic which have logic to resolve forward refs. Unions are not “type” but are valid in isinstance already. This again touches on topic that type vs type alias vs type form are similar but separate concepts. Especially when type form concept useful for runtime introspection is not yet possible to annotate explicitly today until that pep continues. I think trying to add another concept here is somewhat orthogonal to should pep 695 aliases behave more consistently at runtime with pep 613 aliases and backwards compatibility/upgrade implications.

Edit: A pep 613 alias is something valid in annotation context which is broader then most other notions of type and runtime usability varies depending on a lot of details. I’d prefer pep 695 alias be similarly as broad as 613. Producing errors otherwise is likely to be false positive prone for codebases that do interact with them heavily given boundaries today are very messy. If you really wanted boundaries you would likely end up adding many forms of type (isinstance compatible vs subclass compatible vs other rules that vary by runtime library). Many runtime type libraries like typeguard have their own random rules of what’s supported that boils down to how complex is it to handle (generics with non specialized type variables tend to be difficult at runtime). Restricting to basic set will give a bunch of false positive errors and distinguishing which is safe sounds very complex.

1 Like
8

IMO, trying to make TypeAliasType behave more like the type it wraps is not a good idea. It can’t be fully transparent without throwing away the benefits the syntax provides, and adding some stuff that works and other that doesn’t is not a good idea. Here a few things that people could potentially want to do with a class object:

  • use as annotation
  • subclass
  • isinstance/issubclass
  • instantiate
  • access class methods
  • read class attributes
  • write class attributes
  • access magic methods and attributes (e.g. __init__, __dict__, …)
  • check with ==
  • get it’s metaclass with type(cls)
  • check with is

IMO, supporting some of the runtime behavior, but not others is just going to invite confused bug reports. So Option C is IMO a terrible idea, since it can never truly work.

1 Like
9

It’s still early days for PEP695 support in type checkers. I would consider this a pyright bug. Runtime use of TypeAliasType.__value__ should probably behave exactly the same as a runtime use of variable annotated with typing.TypeAlias would.

I’ll be the first to criticize the current state of runtime type introspection. There’s many special cases you have to keep in mind and the current tools like typing.get_type_hints do not help you enough to deal with all of them. PEP 649 is promising and a step in the right direction, but I would like to see much better introspection than that still.

Maybe there would be some interest amongst library authors of libraries such as Pydantic and SQLAlchemy to consolidate their type introspection code into common utility functions and publish it as a separate package on pypi to demonstrate what kind of functions (and what special cases they need to be able to handle) would be useful to have in the standard library. Maybe such a project already exists and I couldn’t find it?

That being said I don’t view any of this as a true criticism of TypeAliasType, but rather as a criticism on the available tooling for runtime use of type expressions. I think the main pain point right now is that any time we add new typing concepts, your hand written runtime analysis code needs to extended to deal with that new possibility. In the case of typing.get_type_hints they could have made the decision to automatically resolve any TypeAliasType in the chain, to increase chance of backwards compatibility, but then what if you wanted to know about them? Add a new parameter to the function? But is that really the default behavior that is the most intuitive?

There’s many open questions when it comes to runtime introspection of typing constructs, without obvious answers we’ll just have to keep experimenting and improving things with future PEPs.

10

Hello. I’m the author of a library that uses runtime type introspection (cattrs, mentioned in the thread already). I can provide my perspective.

I really like the new type aliases because they’re distinct from their base types at runtime. The ability to treat an type Iso8601 = datetime runtime alias to datetime.datetime, or a Float32 runtime alias to float is really valuable for certain types of libraries.

That said, I’m not sure how much of a problem it’d be for cattrs to enable most, if not all of the uses mentioned by @MegaIng as long as there’s a way to introspect the type and determine that it is, in fact, a type alias. (In older Pythons and typing.TypeAlias this isn’t possible because they’re literally the same object.) It would also be great if functools.singledispatch were made to work with type aliases treating them as a subtype, and not the exact type.

I think it’d be weird if Float32 is float returned true though.

11

All is literally impossible because of the last two, and even most of the others would be very weird.

Also, from your description it sounds like what you actually want is NewType with runtime support, not an alias.

12

Well, it might be possible if there was a types.get_type_alias_base() or something that would do the work on a lower level, I guess. I’m not saying this is a good idea (or a bad one, for that matter).

13 
type MyStr = str

assert MyStr is str
assert get_alias_name(MyStr) == get_alias_name(MyStr)
assert get_alias_name(MyStr) != get_alias_name(str)

I don’t see how any sane semantics for get_alias_name could be implemented so that it can make a distinction between MyStr and str, despite the object those two names refer to being the exact same.

Or do you mean something else? If so, please explain more concretely than “or something”.

14

First, let me take a step back to why we designed the type statement the way it is today. The motivation of PEP 695, which added this construct, was to provide clear scoping semantics for generic type aliases: you can now write type SomeGeneric[T] = ... and have the scope of T be clear and unambiguous. While implementing the PEP, I noticed that the new syntax would also enable us to obviate the need for quoted annotations in forward references. That is why in type X = <something>, we now do not evaluate <something> until you explicitly ask for the value. This also made it trivial to create recursive type aliases (the initial version of PEP 695 had an odd special case to support recursive aliases).

These new type aliases thus have two advantages over the old system: they provide a clear scope for type parameters, and they are lazily evaluated so you don’t have to worry about quoting annotations. Also, if a runtime typing library wants to show the name of the alias to users, they now can.

You can do this already: use isinstance(Iso8601, typing.TypeAliasType) to see that the object is a type-based type alias, then access Iso8601.__value__ to see its value:

>>> type X = int
>>> import typing
>>> isinstance(X, typing.TypeAliasType)
True
>>> X.__value__
<class 'int'>

Note though that the value is lazily evaluated (to enable forward references and recursive types), so you have to account for errors that happen during evaluation:

>>> type Boom = 1/0
>>> Boom.__value__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in Boom
ZeroDivisionError: division by zero

(Obviously such a type is not meaningful, but a robust library needs to deal with this case and provide a good error message to users.)

Others in this thread brought up other libraries (pydantic, SQLAlchemy) that do runtime type introspection. They will simply need some updates to support the new type statement.

@MegaIng above has a useful list above of other operations you can perform on classes but not on type aliases. If we wanted to, we could add special-casing to support at least some of those (e.g., we could add TypeAliasType.__mro_entries__ to support subclassing, or override __getattr__ to access class attributes), but there are two limitations:

  • We’d have to implicitly evaluate the __value__ of the type alias during this operation. Now, for example, subclassing could cause a NameError to be thrown inside the code for the type alias. This is confusing for users.
  • Many operations will only be valid on some kinds of type aliases. For example, it’s not meaningful to subclass a union. Again, this might be confusing for users of type aliases.
4 Likes
15

Thanks for the explanation, it’s exactly what I wanted to see.

I still disagree that these special cases shouldn’t be supported (we do have support for isinstance and issubclass checks for UnionAlias after all), but if we’re not going to support them I think it should be explicitly stated as part of the typing spec, and the error messages for these actions should also be improved as they are kind of confusing to me at least.

>>> isinstance("abc", NewAlias)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union

It looks weird that a type alias is not a type.
I don’t think this is error any less confusing than the two cases you listed.
I’ve been using Python for 7 years and would consider myself an advanced user, and while I would be able to diagnose the error fairly quickly the messages aren’t of much help atm.
My conclusion is that both cases are fairly confusing and we have to choose what confusion we want, but I will admit that you’ll probably find the current error earlier and that possibly makes the debugging experience nicer.

Once again, to reiterate, I’m happy with any conclusion we draw as long as this is document carefully. Even if I disagree with the decision made.

16

I don’t have a strong opinion on adding isinstance() support for type aliases. You’re definitely right that the error message you posted is confusing. I do think there’s some potential for new confusion if we add the support, but it may be a net positive on balance.

Just a minor point: None of this should go in the typing spec. The typing spec is about type checker behavior, and what we’re discussing here is the behavior of the runtime. The runtime is documented in the language reference.

1 Like
17

Sorry, I assumed the typing spec was about run-time behaviour as well. Thanks for the correction.

18

I would say that type checkers should preferably be extended to add support to raise errors if incompatible stuff is used, i.e. currently TypeAliasType in isinstance.

1 Like
19

The two main runtime features I’m interested in for type alias are,

  1. Isinstance/issubclass: This mostly happens in a few specific functions/utilities/libraries and doing __value__ first in those places is pretty doable. This is not something most developers would even have to notice.
  2. Subclassing: Here it would be nice to have clear way that works both at runtime and for static type checkers. This does not go through library/utility though so it’d be nice to have mro_entries.

The rest of possible features on that list of ways to mimic runtime I’ve rarely come across a need for.

A bit weirdly even though I think 2 is much less common then 1, I think 1 is easier to support in a way that average developer doesn’t notice while 2 would be more helpful to have runtime support vs documenting a pattern of use __value__.

Edit: For clarity, I’d be content with documented recommendation for number 2 of use value. My preference is for mro_entries but any clear way that supports both runtime and type checkers is reasonable. Still needs type checker support though if we take the value route.

2 Likes
20

If I were to send a PR to CPython to make isinstance() work with TypeAliasType, would anyone here object to that? Does anyone think this will make the Python experience worse?

1 Like