So this didn’t get much attention but it’s still something that will ease some issues with the new annotations and I’d like to have it both for dataclasses and my own dataclass-like.
I ended up implementing this as part of the other discussion on annotation transformations, but it really covers a separate issue. I also have a branch reworking dataclasses on top of this new format to demonstrate the problems it solves.
I hope that this demonstrates why this format is useful because I’d like to know if it’s worth following this up with a proposal or if there’s just no interest as it first appeared.
Just the annotationlib changes
The demo/reference implementation of the annotationlib changes with no dataclass modifications is currently here: GitHub - DavidCEllis/cpython at deferred-annotations
This adds 2 classes and one function to annotationlib, along with the new enum value for Format.DEFERRED. These changes could be made available in a backport to support 3.14 if necessary.
Calling get_annotations or call_annotate_function with Format.DEFERRED will always return a dict with DeferredAnnotation objects as values given a sensible, working annotate function or __annotations__. This is unlike Format.FORWARDREF which will attempt to resolve as much as possible.[1]
from pprint import pp
from annotationlib import get_annotations, Format
Vector = list[float]
class Example:
a: int
b: Vector
c: undefined
d: list[undefined]
forwardrefs = get_annotations(Example, format=Format.FORWARDREF)
deferred = get_annotations(Example, format=Format.DEFERRED)
print("ForwardRef:")
pp(forwardrefs)
print("\nDeferred:")
pp(deferred)
Output:
ForwardRef:
{'a': <class 'int'>,
'b': list[float],
'c': ForwardRef('undefined', is_class=True, owner=<class '__main__.Example'>),
'd': list[ForwardRef('undefined', is_class=True, owner=<class '__main__.Example'>)]}
Deferred:
{'a': DeferredAnnotation('int'),
'b': DeferredAnnotation('Vector'),
'c': DeferredAnnotation('undefined'),
'd': DeferredAnnotation('list[undefined]')}
DeferredAnnotation objects internals are private, partly as they can be several different types but also to allow for future optimizations if there is a better representation in a future Python release. It is also possible to construct them from objects that have already been evaluated for make_annotate_function in order to support cases such as make_dataclass where annotations need to be created from evaluated objects.
from annotationlib import (
call_annotate_function, make_annotate_function, Format, ForwardRef
)
annotate = make_annotate_function({"a": str, "b": ForwardRef("Any", module="typing")})
print(call_annotate_function(annotate, format=Format.FORWARDREF))
print(call_annotate_function(annotate, format=Format.STRING))
print(call_annotate_function(annotate, format=Format.DEFERRED))
import typing
print(call_annotate_function(annotate, format=Format.VALUE))
Output:
{'a': <class 'str'>, 'b': ForwardRef('Any', module='typing')}
{'a': 'str', 'b': 'Any'}
{'a': DeferredAnnotation('str'), 'b': DeferredAnnotation('Any')}
{'a': <class 'str'>, 'b': typing.Any}
Dataclasses example
As an example of their utility, here is another branch that also changes dataclasses to use Format.DEFERRED: GitHub - DavidCEllis/cpython at deferred-annotations-dataclasses
This makes dataclasses use get_annotations(cls, format=Format.DEFERRED), replacing the use of Format.FORWARDREF. It also removes the two custom annotate functions that were created (one for __init__, one for make_dataclass), replacing them with new standard ones from annotationlib.make_annotate_function.
This fixes and improves a number of things:
- References to the class itself will resolve in
field.type- issue.field.typeis made into a settable property with an internalfield._typewhich may be aDeferredAnnotation.
from dataclasses import dataclass, fields
@dataclass
class Example:
examples: list[Example]
for f in fields(Example):
print(f"{f.name}: {f.type}")
3.14.2
examples: list[ForwardRef('Example', is_class=True, owner=<class '__main__.Example'>)]
deferred-annotations-dataclasses
examples: list[__main__.Example]
- The annotate function for
__init__no longer fails if there is an unresolvable non-init annotation.
import inspect
from dataclasses import dataclass, field
@dataclass
class Example:
not_in_init: list[undefined] = field(init=False, default=None)
in_init: int
print(inspect.signature(Example))
3.14.2
...
not_in_init: list[undefined] = field(init=False, default=None)
^^^^^^^^^
NameError: name 'undefined' is not defined
deferred-annotations-dataclasses
(in_init: int) -> None
- Annotations removed from the class that were present when
__init__was generated are kept in the generated__init__.__annotate__function and hence the function signature.
from dataclasses import dataclass
import inspect
@dataclass
class C:
"doc" # prevent inspect.signature from running early
x: int
C.__annotate__ = lambda _: {}
print(inspect.signature(C))
3.14.2
(x) -> None
deferred-annotations-dataclasses
(x: int) -> None
Note on a potential change to untyped `make_dataclass` behaviour
In the current deferred implementation, untyped fields made with make_dataclass no longer import typing when get_annotations is called. As such, get_annotations on such a generated class will fail with VALUE annotations if typing is not imported. This could be changed if desired, but I think it’s consistent with how PEP-649 annotations work in general.
Note: Don’t run this in the REPL, as the REPL imports typing itself.
from dataclasses import make_dataclass, fields
from annotationlib import get_annotations, Format
C = make_dataclass('C', ["x"])
print("ForwardRef")
print(get_annotations(C, format=Format.FORWARDREF))
x_field = fields(C)[0]
print(f"{x_field.type = }")
print("\nValue")
try:
print(get_annotations(C))
except NameError as e:
print(repr(e))
print("Retry with typing imported")
import typing
print(get_annotations(C))
print(f"{x_field.type = }")
else:
print("With typing imported")
import typing
print(f"{x_field.type = }")
3.14.2 - Note that even trying to use Format.FORWARDREF imports typing, as get_annotations will first try to get __annotations__ which uses VALUE annotations.
ForwardRef
{'x': typing.Any}
x_field.type = ForwardRef('Any', module='typing')
Value
{'x': typing.Any}
With typing imported
x_field.type = ForwardRef('Any', module='typing')
deferred-annotations-dataclasses
ForwardRef
{'x': ForwardRef('Any', module='typing')}
x_field.type = ForwardRef('Any', module='typing')
Value
NameError("name 'Any' is not defined")
Retry with typing imported
{'x': typing.Any}
x_field.type = typing.Any
Note that these will not attempt to resolve stringified
__future__annotations, matching howget_annotationswill also not attempt to resolve such annotations. ↩︎