Dataclasses - make use of Annotated

Ideas
21

Yes, this would certainly require a PEP.

3 Likes
22

For what it’s worth, Typer also supports Annotated:

1 Like
23

I don’t think this should go into dataclasses, but I can see why some people think it would be a good idea.

There are essentially two kinds of users of dataclasses, one that uses it to create simple record classes, and one that uses it to remove boilerplate. I am in the first camp and have found that the second way of using dataclasses is not that good.

When I say simple record classes, I mean classes that typically only contain simple types, other record types, and no container types. So I think dataclasses is good for this

from dataclasses import dataclass

@dataclass
class Foo:
    bar: int
    baz: str

@dataclass
class FooBar:
    index: int
    foo: Foo

But if one of the classes must contain a container or any other complex class I’d prefer to just hammer out the boilerplate, because in my experience that typically means the class contains (or will eventually contain) somewhat complex initialization logic. And if I have to write a __post_init__, I might as well write the __init__ and take normal control of class initialization. Keeping dataclasses simple makes it easier to keep track of the responsibility of the classes in my code: dataclasses is just a clump of data and everything else models more complex behaviour.

I also have no issue with type checkers having to special-case dataclasses. Fitting a static type system onto Python is like fitting a round peg into a square hole. Somethings aren’t modeled nicely and adding complexity to dataclasses seems like the wrong thing to do.

3 Likes
24

There is something which I may be missing in your post.

  • There are no traces of any field call in your examples.
  • The proposal has nothing to do with __init__ and/or __post_init__ which you mention
  • The proposal is not about whether the type hint is complex or not (a dataclass is a container) but as how per-field specific information is conveyed.
25

Thanks for the tip.

26

Contributing a new PEP seemed like overdoing it at this point in time, but using Python’s EAFP (Easier to Ask Forgiveness Than Permission) a PEP Draft is available at:

27

My point is that, in my experience, when you add “complex” fields (like containers) to a dataclass, it’s often going to be rewritten as a regular class at some point anyway, so you might as well do it immediately. That’s why you never save a field call in my example, I never use it!

I mentioned __post_init__ because that’s what you have to use to initialize a dataclass with complex initialization behaviour, and my point was that then you might as well just write the boilerplate and get it over with.

While this proposal itself doesn’t talk about the complexity of the type hint, accepting it is an implicit endorsement of using dataclasses as a boilerplate-removal tool (because it promotes the use of field), even when the class has complex behaviour. I personally don’t like this and would prefer dataclasses to be used only as simple record classes. I think we should leave the more complex behaviour for the more complex tools, like attrs and pydantic.

28

Fair, but no the point. field and its return value Field do exist, are used and are not going anywhere.

The proposal is about modernizing how they are used, because they have been used, they are going to be used and they will be used.

Don’t take it personally, but if you don’t like those parts of the dataclasses and the endorsements (implicit or explcit), my suggestion is to open a topic in which you advocate for the removal of the functionality including the removal of __post_init__

dataclasses were a formalization of many of the things I did for my own projects/classes and I found it fantastic to have them added to the stdlib, hence my proposal to improve them further.

1 Like
29

I definitely don’t want to remove existing behaviour, but that doesn’t mean I cannot be against adding new behaviour to “modernize” a feature I don’t particularly care for. That said, I won’t cry if this gets added (tho if I were a maintainer I might’d). It also adds another non-obvious way of doing what is incredibly simple already, which might confuse people. Although that might just be me. I’m also not a fan of binding the (optional) type system more strongly to the language, even stdlib modules (one should not feel forced to use types).

Anyway, I wish you luck finding a sponsor for the PEP-to-be

1 Like
30

Bes of luck indeed.

If a sponsor wants to show up … (no need to hold your peace forever)

Best regards and thanks to all for contributing to the thread, whether you agree/disagree with the proposal.

2 Likes
31

FWIW this makes total sense to me.

This not only simplifies the jobs of type checkers, it also clarifies one big problem that I’ve had with dataclasses:

when you use the x: int = field(...) syntax, it looks like that variable has a default value. But it may not.

Since dataclasses don’t let you create fields with default values before fields without one, the presence and absence of an = sign would have been a very obvious indicator of where the required arguments end, and non required arguments start.

Except that due to field() syntax this is not always true. With the Annotated syntax it will be true again. So a definite +1 from me.

TL;DR

  • Makes dataclasses more readable
  • Uses the “appropriate” language feature for the appropriate task (i don’t see how this is different from adding, say, the decorator syntax. readability counts)
  • Simplifies the type checkers
32

I can understand why people aren’t a massive fan of the current syntax, but as I and Eric Traut said in the other thread, this wouldn’t simplify things for type checkers. Unless and until the current way of doing things is removed – which would be hugely disruptive at this point – type checkers would simply have to add even more special-casing so that they could account for both ways of doing things. This would also be the first time we would be asking type checkers to look at the metadata provided with an Annotated type, an additional complication: currently we promise that type checkers can always treat Annotated[T, <metadata>] identically to how they treat T.

5 Likes
33

this wouldn’t simplify things for type checkers

You’re right, I didn’t consider the field properties.

This now feels like the change doesn’t belong in the current state of the dataclasses package, more like a rethinking of the package as a whole.

1 Like