PEP 7xx: Dataclasses - Annotated support for field as metainformation

PEPs
3

Thanks for the PEP!

In general, I don’t find the motivation you’ve given for this change convincing, currently. It seems to me that it would be tricky to achieve the correct behaviour at runtime in all cases, and would require a lot of new special-casing from type checkers in order for it to work correctly. The maintenance costs would be significant, but this proposal would not give users the ability to do anything that they cannot currently already do. While it is true that there are many examples in Python where there is more than one way to do the same thing, it is nonetheless better, where possible, to stick to the Zen of Python:

There should be one-- and preferably only one --obvious way to do it.

I also think it’s worth noting that the current way of doing it is actually less verbose than your proposed way of doing it.

Specific points of feedback follow:

1. Nested Annotated types

In the typing documentation, it states (following the specification originally laid out in PEP-593):

Nested Annotated types are flattened. The order of the metadata elements starts with the innermost annotation:

assert Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[
    int, ValueRange(3, 10), ctype("char")
]

This snippet currently fails with NameError with your reference implementation:

from __future__ import annotations

from dataclasses import dataclass, field
from typing import Annotated

@dataclass
class Foo:
    x: Annotated[Annotated[int, field(init=False, repr=False)], "not y"]
Traceback with your reference implementation 
Traceback (most recent call last):
  File "C:\Users\alexw\coding\cpython\test2.py", line 6, in <module>
    @dataclass
     ^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\dataclasses.py", line 1329, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\dataclasses.py", line 1319, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\dataclasses.py", line 1053, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\alexw\coding\cpython\Lib\dataclasses.py", line 805, in _get_field
    ann_a_type, *ann_args = eval(eval_str)
                            ^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
NameError: name 'Annotated' is not defined

Would nested Annotated annotations be supported under your proposal? If not, you should state as such in your PEP. (You should probably also not be raising NameError, whether or not it’s supported, as well :slight_smile: )

2. Compatibility with type checkers

You state in your PEP draft:

This PEP introduces also no compabitility issues with type checkers:

  • The special cases introduced to handle the “field-default-value” syntax in dataclasses are not affected.
  • Type checking the “field-annotated” syntax is 100% standard and done as for Annotated annotations in non-dataclass classes.

This is not true. Type checkers have to implement a substantial amount of special-casing in order to understand dataclasses.field(). If this proposal were accepted, they would have to implement a separate suite of special-casing in order to parse the second argument to Annotated. You can see in this mypy-playground demo that mypy correctly understands the signature of Foo.__init__, but does not understand the signature of Bar.__init__:

from dataclasses import dataclass, field
from typing import Annotated

@dataclass
class Foo:
    x: int = field(init=False)
    
@dataclass
class Bar:
    x: Annotated[int, field(init=False)]

# no error (mypy understands the current way of creating init=False fields)
Foo()

# false-positive mypy error 'Missing positional argument "x" in call to "Bar"'
# (mypy would need to introduce new special-casing in order to understand your proposal)
Bar()

More broadly, the typing documentation (again, following the specification laid out in PEP 593) promises that static type checkers will always ignore any extra metadata given to Annotated. This PEP would mean that static type checkers would have to start looking at, and have to try to parse, the second argument to Annotated in the context of dataclasses. This would be a significant shift, which should be called out in your PEP.

3. Interaction with ClassVar

In gh-90669, it is proposed to allow Annotated to wrap ClassVar in dataclasses, e.g.:

@dataclass
class Foo:
    x: Annotated[ClassVar[int], "this is the x field metadata"]

That issue/PR has yet to be decided on. Would you allow this with your proposal? What would be the behaviour if somebody did something like this, which would seem to be invalid?

@dataclass
class Foo:
    x: Annotated[ClassVar[int], field(repr=False)]

By the way, this also fails with NameError with your reference implementation:

from __future__ import annotations

from dataclasses import dataclass, field
from typing import Annotated, ClassVar

@dataclass
class Foo:
    x: Annotated[ClassVar[int], "foooo"]

4. Tests

Your reference implementation contains no tests, so it is hard to see which scenarios you have considered and which you haven’t.

7 Likes