This PEP enhances TypedDict - an existing kind of type annotation for describing dictionaries with a known set of named keys - with the ability to directly mark individual keys as Required[] or as NotRequired[], rather than requiring the indirect use of inheritance plus a total=False attribute to achieve the same result. See full PEP text quoted below.
Consensus on the PEP has been reached in typing-sig. Implementations for mypy, pyright, and pyanalyze exist.
This RFC has been previously posted to python-dev and I am now posting it here as well for additional visibility, by suggestion of the PEP Sponsor ( @guido ).
My understanding of the PEP process as described in PEP 1 is that content review of new PEPs should be requested of the core team prior to submitting the PEP to the Steering Council. So this is my RFC to the core team.
Best,
David Foster | Seattle, WA, USA
Contributor to TypedDict and Python’s type system
Hey @davidfstr , sorry for any confusion over the process in PEP 1 regarding posting to the various places—we’re currently working right now to clarify many of the things you ran into python/peps#2266.
Basically, the process goes something like this:
Hold pre-draft discussions somwhere(s) of your choice (good idea to link them in the PEP)
Submit the PEP
Create (if not already) a discussion thread at an appropriate venue (Typing-SIG, Python-Dev, Discourse) and link the thread in the PEP
Announce it on Python-Dev with a link to wherever the canonical Discussions-To location is (if not Python-Dev)
Submit it to the SC for consideration (which is currently also not explicitly specified in PEP 1, which is something we also want to provide at least general guidance on)
A few things here:
Posts about (non-packaging) PEPs go in the PEPs category here. I moved it for you.
I strongly advise editing your post to not include the full, raw text of the PEP here, which Discourse further mangles, as opposed to just linking the actual PEP—this makes it far easier for interested readers to read, while not making for a huge wall of text in your OP here.
Please link the canonical Discussions-To thread on Typing-SIG where official discussion and feedback has and should take place, both here and on the PEP (I submitted python/peps#2344 to do the latter)
I suggest making clear that this is only an announcement, and that discussion should take place on the canonical thread as linked here and in the PEP, to avoid fragmenting the discussion and making it harder to follow for both community members and the SC, when they review the PEP.
We could even lock this post, to avoid people missing that and replying anyway. @brettcannon , thoughts?
I think dataclasses (with deserialisation from duct) already achieves the functionality proposed. Would it be better to teach users to use that rather than asking another feature to typing?
I don’t feel to strongly, but I would like to see this as a rejected idea
Dataclasses are an alternative to TypedDicts as a whole, not to this proposal, and TypedDict has been in typing for years already. I’d agree with you that dataclasses are a better choice when designing a new API, but when annotating an existing codebase, you have to describe the APIs that already exist. And unfortunately, often these APIs will involve dictionaries with a fixed set of keys—which is why we created TypedDict. This proposal provides an enhancement to the existing TypedDict system, and I don’t think it makes much sense to discuss dataclasses in the PEP.
If you are working with data that originates from json, the most straight forward way to type hint my experience has been with TypedDict. You can have a dataclass (or similar like pydantic) as a wrapper for that json data, but I tend to just pick TypedDict for simplicity. There is a caveat I’m not sure dataclasses has a method to deserialize from a dict/json directly so you would still need to rely on an external library or have your own logic for that. dataclasses has asdict, but no from_dict.
I think for most part this PEP mainly serves to make required/not required more friendly to write with TypedDicts. It discusses how you can subclassing and total/non-total to have very similar behavior but I think,
class Foo1(TypedDict):
a: int
b: str
class Foo2(Foo1, total=False):
c: bool
is less readable and less obvious to do vs,
class Foo:
a: int
b: str
c: NotRequired[bool]
The main difference between the two I know of is Required/NotRequired when using functional form of typed dict allows keys that are not valid variable identifiers and working with json data hyphens or under symbols may reasonably appear.
This last example is not possible to write directly as a dataclass or as typeddict using subclass style. The best you can do is have conversion from invalid key to a valid key and then load it.
I don’t appear to have the ability to edit my original post. Perhaps that’s only possible a short while after posting? Or maybe I don’t have enough permissions since I only just registered for this Discord server?
Perhaps you or a server administrator would be able to edit the post on my behalf?
Its a combination of both, AFAIK—new users only get a short edit period, which gets longer as TL increases. I’m not a big fan of this Discourse limitation—I’m TL3 and even I run into limits when trying to update older posts to reflect progress or changes—but it is what it is (maybe it could be changed on the instance?)
I believe TL4 can edit others’ posts, if that option is enabled, but I’m only TL3. I am pretty sure admins can, though. Sorry to bug you again @brettcannon — could you help here?
Sorry, just got back from vacation and catching up on things. What specifically is needed? “edit the post on y behalf”, for instance, doesn’t tell me what the post should be edited to.
Thanks! Will do; I felt bad nagging you all the time since you were the only admin I knew, but I didn’t know of a way to mention all of them until now.
