Thanks for the suggestion!
The main blocker for list syntax like [int, ...] is runtime semantics. At runtime, [int, Metadata] evaluates to a standard list instance. Because a list is not a type, adopting this would break runtime type checkers (like Pydantic or typeguard) which expect type hints to evaluate to valid type constructs.
(Note: I’ve also just updated the Pre-PEP draft to explicitly address this in the Rejected Ideas section).
That assumes though that [int, …] would just evaluate to a list at runtime, not Annotated? The proposal is not to replace Annotated with lists, but to use the bracket syntax to produce Annotated.
Currently, all type checkers reject [int, …] as an invalid annotation, so it doesn’t currently hold any semantic meaning or fulfill a purpose. I don’t see an issue then just having that produce Annotated at runtime?
A couple of other questions / remarks:
Performance
Making the syntax built-in eases runtime metadata use by removing
typingmodule import overhead.
Does that actually make a difference? This would only matter for code that both uses these annotations, but does not use any library which consumes them or, if it uses such a library, if that library does not use the typing module. I am not aware of any library using runtime type introspection that does not use the typing module, and I’d wager that almost all code making use of these annotations will be using typing. Wouldn’t a lazy import be the better solution here?
Usage examples / applicability
The usage examples, libraries cited that could benefit from this, and compatibility checks are all from the Pydantic/FastAPI ecosystem. Since usage of Annotated is much more widespread than this though, I would find an analysis and consideration of usages / benefits / downsides in the wider typing ecosystem quite relevant.
Developer Ergonomics and Ecosystem Alignment
Cites
This ergonomic barrier was notably evident in the withdrawal of PEP 727 (Documentation Metadata). The extreme verbosity of the syntax in function signatures was a primary factor in its community pushback.
But I don’t think that’s entirely true; The verbosity did not come from a lack of shorthand syntax for Annotated, but from the proposed Doc functionality itself.
Compare some of the actual snippets from FastAPI’s source code:
def __init__(
self: AppType,
*,
debug: Annotated[
bool,
Doc(
"""
Boolean indicating if debug tracebacks should be returned on server
errors.
Read more in the
[Starlette docs for Applications](https://www.starlette.dev/applications/#instantiating-the-application).
"""
),
] = False,
vs. how it would look with a shorthand syntax:
def __init__(
self: AppType,
*,
debug: bool @ (
Doc(
"""
Boolean indicating if debug tracebacks should be returned on server
errors.
Read more in the
[Starlette docs for Applications](https://www.starlette.dev/applications/#instantiating-the-application).
"""
),
) = False,
The verbosity is almost identical. It’s the injection of docstrings into type annotations and the function signature itself that causes it, not Annotated, so I don’t see this as a strong argument in favour of the proposed shorthand (or any shorthand for that matter).
I also don’t see any points made towards “ecosystem alignment”. I’m not sure what exactly that is referencing.
That doesn’t seem like a major problem; tools often have to be updated to deal with new typing syntax.
The bigger problems to me would be that this syntax doesn’t compose well, because lists of types can already have a meaning in the type system (ParamSpec specializations). Also, typing syntax is already very heavy on brackets, so adding yet another bracket syntax doesn’t feel compelling.
While im not against the current proposed syntax, the list-based one would be exactly equivalent to removing “Annotated” relative to the existing code it would be replacing. So i dont know that its any more or less bracket heavy than code is today
Eliminating a source of brackets could be considered a pro of the current proposal, but imo its not a negative of the suggested alternative.
With that said, ambiguity relative to existing things that use bare lists is a fair point
The main reason I lean toward the @ operator over lists ([int, annotation]) or tuples comes down to runtime semantics.
In our patched python you can do:
>>> type(int @ ...)
<class 'typing.Annotated'>
This is consistent with how the other operators behave (e.g.: list[int] or int | float).
Reusing the @ operator requires less conceptual overhead (and changes in type checkers/libraries) than changing the core semantics of lists.
I think we’re striking the right balance: it “graduates” annotated from typing.Annotated (in the same way that Optional, Union, Generic, TypeAlias and Generic have) but sticks with the current abstractions.
The goal is to better support “type-directed” programming. This trend is evident across both modern “type-first” libraries like FastAPI, Pydantic, Typer, and Strawberry, as well as established ecosystems [1] like SQLAlchemy 2.0, Pandera (dataframes), and jaxtyping (tensors/arrays). All of these are now built on top of Annotated.
Despite being pervasive, Annotated is largely hidden because of its verbosity. For example, Pydantic exposes custom aliases like PositiveInt for Annotated[int, Gt(0)]. This limits the usefulness of shared standards like annotated_types and falls down as soon as you need to express a constraint that lacks a built-in alias (e.g. Ge(0)).
A concise syntax removes this friction. A single int @ Ge(0) constraint can cleanly thread across tools:
a notable exception is embedded compilers like Numba, Cython, and PyTorch that rely heavily on type-attached metadata but have not adopted Annotated ↩︎
Thanks for the PEP! I have some thoughts on it:
The
@operator is implemented by addingnb_matrix_multiplyto the metatype (type) and to several typing-related types. The operator is supported for any left-hand operand that currently supports the|operator for making a union.
Does this include type(None)? The reference implementation adds nb_matrix_multiply to it, but calling NoneType a “typing-related type” is a bit of a stretch.
This proposal introduces a new format,
Format.FORWARDREF_STRUCTURAL.
Could you put that in the Specification section?
To me it looks that this proposal could be a PEP by itself, as a solution to the increasing friction between regular syntax & typing annotations. It’s quite surprising to find it as a Backwards Compatibility note.
Notably, if FORWARDREF_STRUCTURAL becomes the state of the art, we … don’t really need __matmul__ on the runtime types anymore.
The shim is scheduled for removal in Python 3.18.
The standard deprecation period is 5 years; is there a reason to rush this?
Were chained colons considered?
class Project(BaseModel):
name: str: Field(title="Project Name"): Len(1)
url: HttpUrl: Field(description="The project homepage")
stars: int: Field(ge=0) = 0
Would it make sense to standardize naked strings as docstrings/descriptions? Frameworks could define how exactly they treat these, but assigning a default meaning might help across projects.
class Project(BaseModel):
name: str @ "Project Name" @ Len(1)
url: HttpUrl @ Field(title="Project Homepage") @ "The project homepage, including the scheme (like http://)"
stars: int @ Field(ge=0) = 0
Just read the PEP, nice work!
I’d love to see examples of usage that span multiple lines, if you believe that’s worth showing. Typically, would we wrap metadata in parentheses?
int @ (
Hello()
@ World(
"""
very long string
"""
)
@ Bye()
)
Looks like the Python syntax highlighter here doesn’t like it: it sees World and Bye as decorators. This isn’t something new (operator already exists) so maybe the PEP shouldn’t concern itself about it.
I suppose the rejected idea in PEP 727 applies here too:
this would create a predefined meaning for any plain string inside of
Annotated, and any tool that was using plain strings in them for any other purpose, which is currently allowed, would now be invalid
This doesn’t apply to new syntax though. There will be no ambiguity if we defined a meaning for bare strings for a new Annotated syntax.
As a code reviewer, the chained colons look like a typo or syntax error to mii. I definitely prefer the proposed @ operator haha. I’m sure eventually I could get used to the colons but it seems like it would be a lot more friction
Not all types are inside annotations, and that will remain the case for the foreseeable future. (Places where you might need a type outside an annotation include in NewType definitions; in generic parameters to base classes; in calls to cast() and more generally in places where TypeForm is used at runtime; in the argument to NewType(); in type definitions for the functional form of TypedDict and NamedTuple; and in old-style type aliases.)