Idea: Simpler and More Expressive Type Annotations

Typing
41

I’m not sure I like this in its current form.

Calling get_annotations(obj, format=Format.AST) gives a dict[str, ast.expr] but your create_annotate_from_asts function requires each expression also have an associated Mapping but it’s not clear where this is supposed to come from or what exactly it should be?

create_annotate_from_asts(annotations: dict[str, tuple[ast.expr, Mapping[str, object]]]) -> Callable[[Format], dict[str, object]]

Presumably it’s related to the context in which names in the ast expression should be evaluated?

I’m also not sure how I would be intended to use this if I need to add objects that have not been obtained through get_annotations to the new annotations? Things like the return None value for dataclasses’ __init__ or a dynamically created TypedDict for an as_dict method[1].

I’m actually curious also to how memory usage would compare to capturing the original source text of the annotations. I assume that this wasn’t done originally for memory use reasons too, but it does have the unfortunate side effect that call_annotate_function(cls.__annotate__, format=Format.STRING) is significantly[2] slower than call_annotate_function(cls.__annotate__, format=Format.VALUE).

Speaking of STRING, I don’t like that this implementation means that the AST format can construct a more accurate STRING than the STRING format itself.

There appears to be a bug in the reference implementation with how the AST is constructed in the reference as op in a BinOp should be an instance of the operation and not the class but with a little editing…

from annotationlib import get_annotations, Format
import ast

class Example:
    a: 1 | 2 | 3

a_anno = get_annotations(Example, format=Format.AST)['a']

# ast.unparse will fail as this is currently the class and not an instance
print(f"{a_anno.op = }")

# Editing in post
a_anno.op = ast.BitOr()
a_anno.left.op = ast.BitOr()

print(ast.unparse(a_anno))
print(get_annotations(Example, format=Format.STRING)['a'])

Output:

a_anno.op = <class 'ast.BitOr'>
1 | 2 | 3
3

  1. In the current implementation, dynamically created TypedDict instances themselves can’t support this format. ↩︎

  2. About 50x slower in a direct test on call_annotate_function, noticeable as about a 40% performance hit on constructing dataclass-like classes when using STRING instead of VALUE. ↩︎

42

It was my initial idea yes, <...> would create an inline type expression, and this PEP only introduces inline TypedDicts for now (but to avoid “reserving” the <...> syntax only to TypedDicts, I proposed having potential extensions for e.g. for tuples with <(...)>). Then @Jelle proposed in th thread that <...> would be an explicit way of exposing the AST.

I just read the draft PEP, thanks for the work put into it. Here is some feedback:

  • Although this is not the main goal of the PEP, I feel like (as mentioned a couple times in this thread) it would make more sense to have the more complex syntax (inline TDs, conditional type expressions) as motivating examples. Some of the current ones in the PEP can also lead to confusion (e.g. dict[int, str] → {str: int} will be too similar to the potential inline TD syntax – {'key': int}; getting rid of Literal for string keys clashes with string annotations).
  • I’m not too sure about having a separate Format.AST value. Up until now, users only had to call get_type_hints() without having to worry about the syntax being used. With the proposed changes, what would be the use case to call get_type_hints() with Format.VALUE, given that it could potentially raise any kind of runtime error (e.g. if we assume bare literal are now allowed, 1 | 'some_string' would raise if Format.VALUE gets used)? I feel like the default get_type_hints() (and maybe get_annotations()) behavior should be to return the “constructed” annotations directly (e.g. 1 | 'some_string' gets converted to Literal[1, 'some_string']).
  • With the previous point in mind, to be able to properly evaluate type annotations, get_type_hints() would need unconditionally[1] to convert the annotation to AST types, to then build the proper typing constructs from it. Memory usage was mentioned in some other feedback, but I’m also curious to know how performance would be affected as well.

  1. Even for existing syntax (e.g. list[int]) that wouldn’t require the intermediary AST step. ↩︎

43

I’m still working on getting all the details right and rewriting the PEP, I’ll report back when I have more details ready. But I can share my current thoughts on the things you mentioned:

In my mind the more advanced future features also are the primary motivation of this change. I just avoided them since I didn’t want to throw in the cognitive load of entirely new features (and their motivation/use cases) with the syntax stuff of this proposal. But from all the feedback so far, that does seem to be the correct approach and I can totally get why. I’ll rewrite large parts of the motivation of the PEP to use better examples.

However, I disagree on the second point. We can’t (from how I understand the backwards compatibility guarantees) really change the behaviour of __annotate__, typing.get_type_hints or annotationlib.get_annotations. While 1 | 2 (or some fancy typed dict literal) isn’t currently legal from the typing spec’s POV, it is an entirely valid annotation in the Python runtime. So even if most people use annotations for type hints, we can’t just assume that every annotation is a legal type annotation and change the behaviour of all annotations.

That’s why I went to the workaround of the AST format, it lets users opt-in to typing specific syntax and then we can “break” non-typing annotations. I also really dislike my initial proposal of adding a weird keyword argument to typing.get_type_hints, but we can’t just change its behaviour either. We also cannot change annotationlib.get_annotations, both because of backwards compatibility and because we want to be able to backport annotations via typing_extensions. I would like to also really avoid a situation where we have three (or even more) different “please give me the stuff after the colons” functions.

My current thinking is that the best solution is to add a new typing.get_type_annotations (or whatever other new name, everyone please feel free to bikeshed here) helper that explicitly uses the AST format to create typing-specific annotations and to deprecate typing.get_type_hints. The intermediate period will be annoying since we have two very similar functions in the same module and one of them will be essentially useless. But we do have the advantage that most usages of these helpers occur in libraries rather than user code directly. The people that tend to write those libraries also tend to be more plugged in to new features and changes, so we have a greater chance of them seeing which function is the “correct” one.

Once the deprecation period is over, we’ll have a nice split of responsibilities: if you want to just get whatever arbitrary annotations exist in an object, you use annotationlib.get_annotations, but if you want type annotations specifically, you use typing.get_type_annotations (or the typing_extensions backport). This difference in use cases and behaviour will also be nicely reflected in their names and module placements.

That is exactly the behaviour I’m proposing. That is, calling get_type_annotations(func, Format.VALUE) doesn’t actually ever call func.__annotate__(Format.VALUE) but rather func.__annotate__(Format.AST) and resolves it using typing specific semantics. The updated PEP will explain this much better than it currently does.

I haven’t done benchmarking yet, but I do expect the performance to be worse than with the current Format.VALUE. Unfortunately, that’s more or less guaranteed since you can’t really beat having everything precompiled into bytecode and evaluated by the interpreter itself. If performance ends up being an issue, we could also think about not using ast.AST objects directly in order to skip their construction and/or write the evaluating code in C, but those have their own issues too.

On the topic of memory usage, I suspect that the impact will be reasonably small. The AST is stored as tuples which almost exclusively contain statically allocated strings and integers, so it really shouldn’t be more than a handful of pointers per annotation. There’s also a lot of room to optimize this more, my current implementation just dumps the AST nodes but a lot of their fields are redundant and/or represented inefficiently.

3 Likes
44

The Python-level AST objects are relatively expensive. Dino opened an issue about improving this (Have internal immutable AST nodes · Issue #140514 · python/cpython · GitHub) but not sure when or if that will land.

It might make sense to have a separate third-party library that evaluates annotations in C (or Rust?) for speed, so we can keep the standard library implementation relatively simple but users like Pydantic that care about speed can use a faster version. Such a library could also allow backporting new syntax to earlier versions of CPython.

I wouldn’t be so sure. Memory usage was a major motivation for PEP 563 and 649: people were noticing significant memory usage from typing objects, and those weren’t huge either.

1 Like
45

I’ve done some optimizations and initial benchmarking, with unfortunately fairly mixed results.

The AST is no longer stored as nested tuples but a bytes object and a tuple (which contains objects that must be allocated regardless) for each annotation. So the main overhead now is cut down to two objects and a small amount of data. In principle, the data being stored should always be significantly less than the size of the __annotate__ code object and my initial testing confirms this. This could also be optimized further if there still is a need for that.

Unfortunately, constructing the AST nodes is very expensive indeed. In my testing any reasonably realistic __annotate__ is executed fast enough that basically all of the runtime is in either annotationlib.get_annotations or typing.get_type_annotations. In the VALUE format we see the biggest difference, get_annotations doesn’t really have to do anything, so get_type_annotations is between 50 and 200 times slower.

Most of that time (about 80%) is spent constructing the AST, which means that even if libraries implement their own bespoke AST to type annotation conversion, it’s still a lot slower. This also means that annotations that require a lot of AST nodes are significantly slower than simpler ones. E.g. a: int is significantly faster than a: Some[Deeply[Nested[Annotation[...]]]].

For the other formats, the picture is much nicer but still very mediocre. The STRING format is bit slower in all cases, but I suspect that this could be brought to par if the AST creation logic was implemented in C. The FORWARDREF format is significantly slower, but I’ve also not implemented the proper logic for it fully, so most of its overhead is just the AST creation again.

I’m not sure how big these performance concerns are in real world use cases. The slow down is massive, but in general I’d have assumed that so little of actual run time is spent in introspecting annotations that I thought it didn’t matter much. But since these concerns have already been raised, they should be addressed sooner rather than later.

One way would be the Issue Jelle linked above, but I’m not sure if I have the experience to really work on that. We could also have the AST format not return ast.AST objects but some faster representation, but that’s not a great solution. If someone has any ideas, I’d be happy to hear them!

4 Likes
46

I wonder if starting from the AST format is working backwards?

From brief testing the output seems to be what you’d extract from compile[1] on string annotations if the strings weren’t ‘lossy’.

import ast
from annotationlib import get_annotations, Format

class Example:
    a: int
    b: list[int]
    c: 1 | 2 | 3

ast_annos = get_annotations(Example, format=Format.AST)
str_annos = get_annotations(Example, format=Format.STRING)

for attr in ['a', 'b', 'c']:
    from_str = compile(str_annos[attr], "<string>", "eval", flags=ast.PyCF_ONLY_AST).body
    print(f"{attr}")
    print(f"From AST:    {ast_annos[attr]}")
    print(f"From STRING: {from_str}\n")
The output for 'a' and 'b' is the same, while 'c' differs as `STRING` currently simplifies the annotation to '3' 
a
From AST:    Name(id='int', ctx=Load())
From STRING: Name(id='int', ctx=Load())

b
From AST:    Subscript(value=Name(id='list', ctx=Load()), slice=Name(id='int', ctx=Load()), ctx=Load())
From STRING: Subscript(value=Name(id='list', ctx=Load()), slice=Name(id='int', ctx=Load()), ctx=Load())

c
From AST:    BinOp(left=BinOp(left=Constant(value=1, kind=None), op=BitOr(), right=Constant(value=2, kind=None)), op=BitOr(), right=Constant(value=3, kind=None))
From STRING: Constant(value=3, kind=None)

Is it possible to instead make non-lossy[2] STRING annotations and then build any AST-like annotations from those?

PEP-649 listed the idea of keeping the original strings in rejected ideas but as a ‘not now’ rather than an outright rejection. Perhaps that’s worth revisiting? More complete string annotations would also support tools that wish to use AST parsers written in other languages (such as the one ruff uses).

Slight digression, but I noticed in one of the original discussions there was going to be a FORWARDREF format that was essentially what would have been what is now needed for making a better (simpler!) dataclasses __init__.__annotate__ function. What is now the FORWARDREF format was at one point the HYBRID format, which I actually feel is a more accurate name. :frowning:

  1. with appropriate arguments ↩︎

  2. at least to the level that __future__ annotations were non-lossy ↩︎

2 Likes
47

From a data point of view, the string and the AST are largely equivalent. We can recover the AST from the string via parsing and the string from the AST (ignoring formatting) via ast.unparse. The big difference is that if you store the strings then you more or less force every user of the annotation to parse the string when they want to introspect them. Either via compile or with some bespoke application-specific solution like the current dataclass implementation.

In particular, if the long term plan of typing annotation specific semantics is implemented, any consumer of type annotations will be forced to (implicitly) parse the string, construct the AST and then resolve it into some typing object. You’re only getting around the expensive AST construction step if you’re using a library with its own parser and AST representation.

I don’t think its a good idea to say that we have some exciting new typing features, but if you actually want to use them you have to first implement your own Python parser. If some specific library wants to get the absolute maximum speed they can get, that’s fine and may very well be worth the effort. But in my opinion the standard library options should always be good enough for the general use case.

48

The idea is mainly that the internally generated __annotate__ function change would be to support better STRING annotations, rather than supporting AST annotations directly. AST annotations can then be built on top of this, in the same way that FORWARDREF and STRING annotations are currently built on top of VALUE_WITH_FAKE_GLOBALS.

By building on top of STRING annotations, this can be backported and can be made to work more generally. __future__ annotations, or annotate functions that only support VALUE won’t make retrieving AST annotations fail if done this way.

Here is a fork with a sketch implementation of both Format.AST and Format.DEFERRED contained entirely within annotationlib. It’s a quick sketch, so don’t expect tests to pass or for any new tests.

It also includes a version of how I would expect make_annotate_function to roughly work. DeferredReference is kind of a ‘fake’ ForwardRef object for things that have already been evaluated. There’s probably a cleaner way to handle this, but for the demonstration this is enough.

These are based on the current implementation of STRING (not through string conversion but extracted from the _Stringifier objects where possible), so details lost in STRING are also lost in these formats.

from annotationlib import get_annotations, Format, make_annotate_function, call_annotate_function

# 'c' annotation will be converted to 3 - the result of 1 | 2
def f(a: int, b: str, c: 1 | 2) -> None: pass

print(get_annotations(f, format=Format.AST))
# {'a': Name(id='int', ctx=Load()), 'b': Name(id='str', ctx=Load()), 'c': Constant(value=3, kind=None), 'return': Constant(value=None, kind=None)}

# Even if the function only supports `VALUE`, this still works
def anno(format):
    if format == Format.VALUE:
        return {'a': list[str]}
    raise NotImplementedError(format)

print(call_annotate_function(anno, format=Format.AST))
# {'a': Subscript(value=Name(id='list', ctx=Load()), slice=Name(id='str', ctx=Load()), ctx=Load())}
DEFERRED / make_annotate_function format demo 
from annotationlib import get_annotations, Format, make_annotate_function, call_annotate_function

def f(a: int, b: str, c: 1 | 2) -> None: pass

# DEFERRED annotations are always ForwardRef or DeferredReference objects
# DeferredReference exists for objects that are already evaluated or constants
# Yes it probably needs a better name or folding into ForwardRef

print(get_annotations(f, format=Format.DEFERRED))
# {'a': ForwardRef('int'), 'b': ForwardRef('str'), 'c': DeferredReference(3), 'return': DeferredReference(None)}

# Add an additional annotation to the annotations dict
annos = get_annotations(f, format=Format.DEFERRED)
annos['d'] = list[int]
new_annotate = make_annotate_function(annos)

print(call_annotate_function(new_annotate, format=Format.VALUE))
# {'a': <class 'int'>, 'b': <class 'str'>, 'c': 3, 'return': None, 'd': list[int]}

print(call_annotate_function(new_annotate, format=Format.STRING))
# {'a': 'int', 'b': 'str', 'c': '3', 'return': 'None', 'd': 'list[int]'}

print(call_annotate_function(new_annotate, format=Format.AST))
# {'a': Name(id='int', ctx=Load()), 'b': Name(id='str', ctx=Load()), 'c': Constant(value=3, kind=None), 'return': Constant(value=None, kind=None), 'd': Subscript(value=Name(id='list', ctx=Load()), slice=Name(id='int', ctx=Load()), ctx=Load())}

print(call_annotate_function(new_annotate, format=Format.DEFERRED))
# {'a': ForwardRef('int'), 'b': ForwardRef('str'), 'c': DeferredReference(3), 'return': DeferredReference(None), 'd': DeferredReference(list[int])}

Unfortunately, despite having your fork working earlier this morning, after going back and forth between versions and cleaning/rebuilding, it now crashes with an Assertion when I try to open the REPL?

python: Python/compile.c:1479: _PyAST_Compile: Assertion `co || PyErr_Occurred()' failed.

Not sure what’s happened there, I can still build from Main and tried to clean the folder between builds.

1 Like
49

From an implementation and performance pointof view, I think we need to examine how often runtime introspection actually happens.

Even with validation libraries built on typing annotations in heavy use at my day job, less than 5% of objects that are annotated (not even counting any stdlib objects that don’t have annotations) end up introspected for annotations at runtime.

Whatever does happen, should be as low cost for the majority of use as reasonably possible. If we ever land in a position where I’m saying we shouldn’t ever get rid of from __future__ import annotations for performance reasons, something has failed

If we’re looking to make sure performance is optimal, there’s a very real discussion to be had about storing the smallest size for the least work required format internally, and not doing anything with them at all in the object itself. Move anything more expensive to only be done when introspection is requested, and have that and any cached state of that belong to introspection.

I think such a function is something that looks like it would be useful in general, but that we have a better way. I wouldn’t want cast to have to use a function like this, that adds a function call to something people are already trying to optimize out, and have even proposed syntax in the past to try and remove the overhead of.

By contrast, It wouldn’t be difficult to propose a string special form that solves this.

Elevator pitch for something I’ve considered here:

typing.StringTypingExpression[T] is a generic special form compatible with any string, and has no runtime behavior. in a type-checking context, the generic is synthetic, and extracts a type from a string expression. When used as a parameter, typecheckers may require that the string is a literal or that all possible values of the string as used in such functions are statically determinable.

This makes any function like cast look like the below instead, and generalizes the special case out of the functions that need it:

def cast(type: typing.StringTypingExpression[T], val: Any) -> T
    return val
50

Having thought a bit more about the previous sketch implementation, I’ve actually moved more against having Format.AST as a stand-alone format.

This isn’t that I don’t see transforming the AST as useful, but that I don’t see it as useful without linking it with the context in order to be able to evaluate it later. Related to what Jelle suggested in his first response (emphasis mine):

I’ve gone for strings and the context needed to resolve these strings rather than using the AST directly. This is largely to avoid the cost of rebuilding the AST if it is not needed - unfortunately it seems that it’s not possible to extract the AST from _Stringifier in all cases (list literals, for instance bypass it entirely).

The new DeferredAnnotation provides a helper .as_ast property instead that parses and returns the AST, and an .evaluation_context property that gives an object that can be used to evaluate the AST (or string, or code object).

Sketch proof of concept fork

Here’s a demo of transforming annotations with literals and lists into Literal[...] and list[...].

Rough AST Transformer code 
import ast

class AnnotationTransformer(ast.NodeTransformer):
    """
    This is used to convert a literal list into a list[...]
    and any Constants other than None into `Literal`s
    """
    # Probably missed some things, not an expert in AST traversal
    def visit_List(self, node):
        if node.elts:
            return ast.Subscript(
                value=ast.Name(id="list"),
                slice=ast.Tuple(
                    elts=[self.visit(n) for n in node.elts]
                ),
            )
        else:
            return ast.Name(id="list")

    def visit_Constant(self, node):
        if node.value is None:
            return self.generic_visit(node)

        return ast.Subscript(
            value=ast.Name(id="Literal"),
            slice=self.generic_visit(node),
        )


import typing
from annotationlib import get_annotations, Format
transformer = AnnotationTransformer()

class NewListStyle:
    a: [int, str, True, 42, [complex, False]]  # Include constants
    b: [foo, str]  # include a forward reference

annos = get_annotations(NewListStyle, format=Format.DEFERRED)

print(f"{annos = }")

transformed_a = transform_annotations(annos['a'].as_ast)
transformed_b = transform_annotations(annos['b'].as_ast)

# Get the context to use to evaluate the annotations - same for both in this case
context = annos['a'].evaluation_context

# Need to add "Literal" to locals in order to evaluate as it's not a builtin
extra_names = {"Literal": typing.Literal}
evaluated_a = context.evaluate_ast(transformed_a, use_forwardref=False, extra_names=extra_names)
print(f"{evaluated_a = }")

# use forwardref
evaluated_b_fr = context.evaluate_ast(transformed_b, use_forwardref=True)

print(f"{evaluated_b_fr = }")

# now define foo - evaluation is now complete
foo = float

evaluated_b = context.evaluate_ast(transformed_b, use_forwardref=True)
print(f"{evaluated_b = }")

Output:

annos = {'a': DeferredAnnotation('[int, str, True, 42, [complex, False]]'), 'b': DeferredAnnotation('[foo, str]')}

evaluated_a = list[int, str, typing.Literal[True], typing.Literal[42], list[complex, typing.Literal[False]]]

evaluated_b_fr = list[ForwardRef('foo', is_class=True, owner=<class '__main__.NewListStyle'>), str]

evaluated_b = list[float, str]

There are some exceptions where annotations can’t be transformed and evaluated. For the use case I have in reconstructing __annotate__ functions it needs to be possible to create annotations from regular objects. .evaluation_context currently returns None to indicate the AST can’t be evaluated in these cases, but this only affects those specific names.

1 Like
51

Maybe I’m mising something, but I’m pretty sure that we can already do that without having to have some new bespoke object for this. Annotations can basically reference three kinds of variables: globals, locals from enclosing functions and class variables if the annotations are methods of that class. We already have a reference to the globals in __annotate__.__globals. If an annotation contains references to enclosing locals, the compiler will see them as cell variables in the __annotate__ function. This means that __annotate__.__code__.co_freevars contains their names and __annotate__.__closure__ contains references to their values. Finally, annotate functions that reference class variables automatically have an additional __classdict__ cell variable that references the containing class’s dictionary. So we can reconstruct the outer namespace as the __annotate__ function sees it without any new mechanisms.

And to Address Jelle’s concern about not being able to evaluate variables that are defined within the annotation: Say an annotation is something like a: list[expr vor var in SomeList], then evaluating expr properly does require knowledge about what var means in its context. But that doesn’t come from the enclosing context of the annotation, it comes from the typing specific semantics of the comprehension expression. If we evaluate [var + 1 for var in some_list] in a regular expression context, we also don’t need to know what var refers to in the scope that contains the expression, that binding would even be shadowed by the comprehension scope. So all that we need to do to be able to evaluate expr in the above example is to construct the ast to typing object evaluation function in such a way that it properly creates and tracks such internal scopes.

Lastly, I also don’t really see how we can do any better than namespaces I listed above without significantly altering Python semantics. The only names that we can’t resolve with that approach are locals from enclosing functions that aren’t already referenced within the annotation. Note that this can indeed happen via stringified annotations. But the references to the objects they contain simply do not exist anymore by the time we’re evaluating the annotate function. Say we have an annotation like this:

def outer():
    SomeAlias = int
    def inner(a: "SomeAlias"):
        ...
    return inner

outer().__annotate__(...)

In this example, SomeAlias isn’t a closure variable since it’s not referenced in any function contained within outer. So it only exists in the stack frame for the final outer() call. But that’s already gone by the time the __annotate__() call happens. The only way for some new name resolution object to be able to resolve it would be to turn every (or at least those that might be referenced in an annotation) local variable into (effectively) a cell variable.

52

Yes, I agree with all your points. The performance impact of annotations should only happen if users actually introspect them. Because of that I’m focusing on implementations that have as minimal a memory overhead as possible.

Regarding your points about type expressions in non-annotation contexts, that’s pretty much exactly my thoughts too! For functions like cast a simple stringified annotation is enough. We also now have TypeForm, which already covers stringified type forms, so I think we don’t even need a new special form to support this. The reason I’m proposing a string evaluation function is for use cases that actually use the type form at runtime. So e.g. if you want to use an expression with the potential new typing syntax as a base class or for some runtime type checker. Then you or the library needs some way of resolving that string to a type, which should be provided in the standard library.

53

For example, to create dataclasses __init__.__annotate__ properly[1] it is necessary to combine annotations from different __annotate__ functions into one new function that will no longer have the original context. By attaching the context to the annotation itself, a new __annotate__ function can provide these and they can all be evaluated correctly. The new annotate function won’t have that context (names could clash between annotations merged from objects in different modules for one).

  1. The current implementation is overly complex currently and can’t handle VALUE annotations if there’s an unused annotation it can’t evaluate. These deferred objects simplify implementing annotation functions like this. ↩︎

54

Ah ok, I misunderstood your point then. In my implementation I’ve solved this by modifying the annotation ASTs such that a reference to a variable var is replaced by some __namespace_ref__.var. The new __annotate__ then has a __globals__ that contains a __namespace_ref__ for each original AST that references that AST’s namespace.

We could also attach the namespaces to the ASTs directly, but that only takes care of the AST format (or however else you want to represent it). How would your approach synthesise the VALUE format? With the transformed AST we can just compile it and it works as expected in the new globals dict. Wouldn’t your approach require some fairly bespoke custom codegen?

55

No, my approach is essentially the same approach ForwardRef.evaluate already takes for Format.VALUE. In fact it’s mostly that code moved into a separate function so it’s reusable. Same for Format.FORWARDREF. The generated __annotate__ function doesn’t support VALUE_WITH_FAKE_GLOBALS, it supports all of the other formats directly.

Edit:
All of the logic is in annotationlib.py of the fork. I’m not going to claim it’s clean - discovering that a literal list skipped the stringifier broke my initial idea fairly late on. There’s lingering code that operates on a ForwardRef as the object inside DeferredAnnotation but that shouldn’t really be the case any more.

56

Since I maintain a library that consumes annotations at runtime I’m trying to follow along with this thread to understand what’s being proposed, but some of the discussion here is over my head.

From what I know, if my library sees a string like `{A: B}` there’s no general way to resolve it to a dict[A, B]. The string itself would have to somehow have a reference to the actual globals/locals in use when it was “created”, right?

1 Like
57

I’m not sure what exactly you’re referring to here. The proposed evaluate_type function would take the string and give you the resolved type object in whatever format you’re requesting. So for example, if you call evaluate_type("{int: str}", format=Format.VALUE) you’d get the dict[int, str] object (a GenericAlias instance).

However if you’re referring to type aliases, then yes that cannot be resolved without further help. I.e. in this

type A = int
eval_type("{A: str}", format=Format.VALUE)

you’d get an error because A cannot be resolved in the builtin namespace. the eval_type or whatever library function is built on top of it would have to be passed the globals/locals of the calling site explicity. I’ve assumed that libraries of runtime annotations already encouraged this since stringified type forms already are a widespread solution for things like forward references.

Also note that this is almost exclusively an issue for stringified type forms that exist outside of function/class annotations. For those we can get most of the actually used globals/locals via the __annotate__ functions dunder attributes.

Would this be an issue for your library? I don’t want to propose changes that make it harder to use annotations at runtime, but from previous discussion the only real way to solve this would be to introduce entirely new syntax, which would be a much bigger hurdle.

58

I’m not talking about aliases - A and B are just some types (maybe dataclasses, but doesn’t matter), defined somewhere. When my library sees the string "{A: B}", there’s no way to resolve it. You’d have to know what A and B refer to exactly in the context where the string was created. I don’t think even stack crawling can help since the string may have come from somewhere else.

To clarify, I’m the maintainer of cattrs. The main interface is something like this:

from cattrs import structure

structure(json.loads(b"{}"), MyClass)

This will continue to work fine. But these, from what I gather, won’t:

# Structure a tuple containing MyClass
structure(json.loads(b"[1, {}]"), "(int, MyClass)")  # What is 'MyClass'?

# Structure a parametrized generic class
structure(json.loads(b"{}"), "MyClass[{int: float}]")

At least in cattrs, stringified annotations work in the context where they can be resolved; dataclass, attrs, namedtuple, typeddict, etc. attributes. But not at the top-level, as the second argument to structure.

To me the only workable solution I’ve seen so far would be allowing something like this:

structure[MyClass[{int: float}]](json.loads("b{}"))

where the interpreter could figure out that the MyClass[{int: float}] expression is a type annotation, and not a “normal” expression. Although it’s not clear to me how the interpreter would know this.

59

My understanding is that part of the goal with PEP-649/749 was precisely to get away from needing these solutions that operate directly on strings. annotationlib is supposed to handle all of the scoping for retrieved annotations.

If you’re trying to generate your own annotation from a string though, EvaluationContext and DeferredAnnotation are also able to help with that. Here’s an example eval_type that transforms dicts based on this.

Using the DEFERRED fork - I only just added the .transform method and fixed some things.

Long AST transformer code 
import ast

class TypeTransformer(ast.NodeTransformer):
    # Probably missed some things, not an expert in AST traversal
    def visit_List(self, node):
        if node.elts:
            return ast.Subscript(
                value=ast.Name(id="list"),
                slice=ast.Tuple(
                    elts=[self.visit(n) for n in node.elts]
                ),
            )
        else:
            return ast.Name(id="list")

    def visit_Constant(self, node):
        if node.value is None:
            return self.generic_visit(node)

        return ast.Subscript(
            value=ast.Attribute(ast.Name(id="typing"), attr="Literal"),
            slice=self.generic_visit(node),
        )

    def visit_Dict(self, node):
        if node.keys:
            return ast.Subscript(
                value=ast.Name(id="dict"),
                slice=ast.Tuple(
                    elts=[
                        self.visit(node.keys[0]),
                        self.visit(node.values[0]),
                    ]
                ),
            )
        else:
            return ast.Name(id="dict")

    # BinOp is necessary to make `undefined | undefined2` a union
    # It is not necessary for types that can be evaluated
    def visit_BinOp(self, node):
        if isinstance(node.op, ast.BitOr):
            return ast.Subscript(
                value=ast.Attribute(ast.Name(id="typing"), attr="Union"),
                slice=ast.Tuple(elts=[self.visit(node.left), self.visit(node.right)])
            )
        return self.generic_visit(node)


import ast
from annotationlib import DeferredAnnotation, EvaluationContext, Format

type_transformer = TypeTransformer()

def eval_type(t: str, context: EvaluationContext, format: Format = Format.VALUE):
    anno_t = DeferredAnnotation(t, evaluation_context=context)
    new_annotation = anno_t.transform(dict_transformer)
    return new_annotation.evaluate(format=format)

# As this is generated and not retrieved, we need to create a context
context = EvaluationContext(globals=globals(), locals=locals())

print(eval_type("{A: str}", context=context, format=Format.FORWARDREF))
# dict[ForwardRef('A'), str]

type A = int
print(eval_type("{A: str}", context=context, format=Format.VALUE))
# dict[__main__.A, str]

More generally a get_type_annotations function that transforms annotations is also fairly short[1]:

import typing
from annotationlib import get_annotations, Format

type_transformer = TypeTransformer()

def get_type_annotations(obj, *, format=Format.VALUE):
    extra_names = {"typing": typing}

    deferred_annos = get_annotations(obj, format=Format.DEFERRED)
    annos = {}

    for k, v in deferred_annos.items():
        # Only transform annotations that have a context
        # otherwise this can break evaluation
        if v.evaluation_context:
            annos[k] = v.transform(type_transformer).evaluate(format, extra_names)
        else:
            annos[k] = v.evaluate(format)

    return annos

class Example:
    a: [int, str]
    b: typing.Union[1, 2, 3]
    c: {str: bool}
    d: undefined | undefined2

annos = get_type_annotations(Example, format=Format.FORWARDREF)

print(annos['a'])  # list[int, str]
print(annos['b'])  # Literal[1] | Literal[2] | Literal[3]
print(annos['c'])  # dict[str, bool]
print(annos['d'])  # ForwardRef('undefined') | ForwardRef('undefined2')

  1. once someone has written the transformer at least ↩︎

60

The related attrs (which I think you’re also involved in maintaining) itself is actually one of the primary motivating cases for DEFERRED[1]. Unlike dataclasses it could potentially make use of backported annotationlib logic to avoid having to jump through the same hoops.

Currently you’re writing __annotations__ for your __init__ functions, which means ForwardRef unfortunately appears in your STRING and VALUE annotations. The string annotations for __init__ show up in help(cls) which is kind of ugly (try this on a dataclass in 3.14.0 and it’s arguably worse).

from attrs import define

@define
class Example:
    a: list[Example]

help(Example)

Output:

Help on class Example in module __main__:

class Example(builtins.object)
 |  Example(
 |      a: list[ForwardRef('Example', is_class=True, owner=<class '__main__.Example'>)]
 |  ) -> None
...

  1. in that it has the same challenges dataclasses has ↩︎