PEP 835: Shorthand syntax for Annotated type metadata

PEPs
142

PEP 835 Update: Spec Alignment, Format.TYPE, and Rendered Draft

Thank you all for the extensive stress-testing and feedback over the last few weeks. The discussion here has been very useful to refine this PEP.

I’ve just pushed a major update to the draft in my personal fork. I’m waiting a bit before upstreaming the changes to gather more feedback. You can read the fully rendered version here, as well as the diff.

Summary of Changes

  • Terminology & Spec Alignment: Consolidated terminology to provide clear definitions for the concepts used throughout the PEP (Type Expression, Type Metadata, Non-Typing Annotation, Symbol Decorator) and align better with the typing spec. This clarifies that @ is strictly scoped to Type Expressions.

  • Forward References & None: Clarified that the newly introduced Format.TYPE handles None @Metadata structurally. This eliminates the need to modify NoneType.__matmul__, keeping standard runtime execution perfectly safe. The “Supporting None” open issue has been removed.

  • Future Work: Added extensive examples of how this syntax perfectly maps to potential future Symbol Decorators (e.g., annotating parameters/fields directly). Also added a JSR-308 inspired proposal for PEP 746, demonstrating how intersection types (&) could enforce target-based constraints for metadata.

  • Structure & Formatting: Reordered all sections to strictly comply with the PEP 1 template and added the mandatory “Security Implications” section.

  • Rejected Ideas: Expanded to address the alternative syntaxes discussed in this thread (e.g., <@, brackets, soft keywords) and moved the __rmatmul__ rationale here.

A Note on Thread Context and Feedback

This thread has been fantastic for catching edge cases. Just a heads up: to keep the document focused, I’ve centered it strictly on the canonical design rather than listing every alternative we’ve covered.

If you spot a critical technical flaw I’ve missed, please feel free to DM me. I also want to leave my inbox open to maintainers of type checkers (Mypy, Pyright, ty, pyrefly…) and libraries (Pydantic, FastAPI, cattrs, msgspec…) or anyone who might prefer to discuss implementation details outside the busy public thread.

Thanks again to everyone who has helped shape this proposal!

5 Likes
143

The addition of Format.TYPE seems like a pretty big deal and could almost be its own PEP, I feel.

I see that the existing Format.FORWARDREF can already parse list[undefined] as list[ForwardRef('undefined')]. I’m actually wondering why it doesn’t also already parse int | undefined as int | ForwardRef('undefined')? It can abstract the __getitem__ but not the __or__, but why not? Would it be backwards compatible to make Format.FORWARDREF behave like the proposed Format.TYPE?

3 Likes
144

It can resolve list[undefined] to list[ForwardRef('undefined')] because list.__class_getitem__ is defined on arbitrary objects.

>>> list.__class_getitem__(object())
list[<object object at 0x10bf60810>]

This isn’t the case for type.__or__, which would be needed for list | undefined to resolve.

>>> type.__or__(list, object())
NotImplemented

You can use a metaclass that does support __or__ for arbitrary objects to demonstrate this.

>>> from annotationlib import call_evaluate_function, Format
>>> from typing import Union
>>> class MetaUnionize(type):
...     def __or__(self, other):
...         return Union[self, other]
... class Unionize(metaclass=MetaUnionize):
...     pass
...
>>> type t = Unionize | unknown
>>> call_evaluate_function(t.evaluate_value, format=Format.FORWARDREF)
__main__.Unionize | ForwardRef('unknown')

Note that I still think adding a new Format.TYPE is a mistake, most of the reasons I’ve had for this earlier haven’t changed.

5 Likes
145

Coming back to this after an attempt to get permission for us to make public the current affected tooling…

The key problem with the latest proposal is that it breaks an existing behavior relationship with annotations.

At work, we have our own internal typechecker and our own typeshed. This came about after years of advocating fixing certain issues in the type system that had caused us issues, and basically being told it won’t ever be fixed. There’s not much motivation to make this typechecker public, and I was told we’d just patch or drop any libraries that we rely on that use the new annotation format if this is accepted.

However, the existing relationship between what is alowed in an annotation, and what the standardized type system accepts has allowed us the freedom to opt out of the standard for static typing.

This breaks that relationship. Currently, if someone tries to use a tool that expects a “type system annotation”, but gets something that is a valid python annotation while not being a valid standard type system annotation one of two things happens:

  • If they inspect the actual annotation and parse it statically: The tool itself recognizes the annotation isn’t what it expects to be there, and has visibility to error.

  • If they inspect the resulting value of the annotation at runtime: This may or may not work, but again, the visibility is there if it doesn’t. In our case, the tools works fine, because we’ve gone out of our way to make sure all of the conveniences implemented (like allowing unwrapped literal numbers) result in types that are shaped correctly at runtime to work with runtime inspection. M[1, 2] @ M[2, 3] injects some literals while validating there’s a shared dimension.[1] Runtime inspection of evaluating the annotation works, because we’re able to implement compatibility for tools that get the value.

In both of those cases, one side or the other gets to ensure compatibility with expectations.

The new format proposal breaks that. It sticks a middleman into the equation that assumes without validating that the valid python annotation is also a valid type system annotation. It doesn’t error in the case of a validly implemented operator it doesnt understand, it reinterprets that operator.

So at runtime, the actual value would be a valid python annotation, and might be implemented for compatability with tools that expect to be able to retrieve them at runtime.

Statically, it would be invalid to tools expecting a standard type system annotation.

And at runtime, if fetched via annotationlib with the proposed format, it would be something that does the wrong thing for both sides of the equation, while informing neither of them of it.

  1. It does more than this, but I don’t want to cross the line on what I can share or explain the more generalized tensor contractions or how we handle typevariables. ↩︎

1 Like