Do we want an exact `TypedDict` / if so how (`@final`, `__extras__ = Never`, ...)?

Typing
1

I was catching up on some discussions about TypedDict and noticed that the topic of @final TypedDict has come up repeatedly but there seems to be no thread for directly discussing it.

In particular I noticed in this thread on subclassing that:

In my opinion the semantics are not entirely 100% specified right now. No PEP actually discusses @final, nor does the current typing spec, so the most reasonable conclusion is that as of today @final is meaningless and TypedDicts always allow structural subtyping.

But it also would be reasonable to treat @final TypedDicts as exact, given how final classes work. I actually thought that was already true (and it sounds like at least for a while Pyright and Pylance treated it this way), and as far as I can tell it doesn’t cause problems in the type system if we were to declare that @final make TypedDict into an exact structural type.

I noticed a few comments in other threads suggesting that it’s not meaningful to talk about exact types with structural typing. I believe this is not inherently true; today Python only has “open” structural types that allow structural subtyping, but “exact” structural types make sense and can coexist with open types.

Two examples I know of offhand of languages that allow both open (structural subtypes) and exact structural types:

  • Flow objects are exact by default (like the proposed @final TypedDict) but can be explicitly marked as open.
  • Ocaml objects behave similarly, closed by default but there’s syntax to allow structural subtyping

Typescript objects are open by default unlike Flow, but there is a long-standing discussion about adding an explicit Exact form. It’s not clear whether it will be added (and the discussion there is likely informative for us since open-by-default is the existing TypedDict behavior)

Since this appears to be a question of whether we want semantics to exist that haven’t been specified thus far, I thought it would make sense to have a thread where people can directly express views on this, rather than leaving to issue trackers and as a side topic in other threads.

2 Likes
2

So, is it a fair summary to state that @final TypedDict could have four possible meanings:

  1. It’s an error to even use it
  2. You can use it but it’s completely ignored
  3. It disallows subclassing, but has no effect on other assumptions by type checkers
  4. If a variable’s type is a @final TypedDict, type checkers can assume it has no keys other than those specified by the TypedDict
3 Likes
3

The @final decorator already has a well-defined meaning in the type system. When applied to a class, it means that the class cannot be subclassed.

TypedDict classes can normally be subclassed to form other TypedDict classes, so @final already has a meaning in the context of a TypedDict class. (This is meaning 3 in Guido’s list above.) Ascribing a different meaning to @final in this particular context (“closed” versus “open”) sounds problematic. It introduces a backward compatibility issue and a confusing redefinition of an existing type system concept.

If we want to introduce the notion of a “closed” TypedDict, I’d prefer to come up with some other mechanism. There is a draft PEP 728 that proposes a mechanism that I think is worth considering. It provides additional flexibility beyond simply marking a TypedDict as “closed”.

4 Likes
4

@guido I’d say your summary is consistent with my view. Since the existing spec never discusses the meaning of @final directly, any of these interpretations are potentially valid. I think most type checkers already enforce (3); at least some have interpreted it to mean (4) in the past but that may not be a good idea, and I think we’ve been moving toward labeling that to be divergencent from the spec.

@erictraut Agreed that maybe @final isn’t the best choice. For what it’s worth I think preventing subclassing is not really useful for a structural type anyway, which is why I think we could interpret it to mean exact, but an alternative might be better.

Interesting to note your point about PEP 728, I hadn’t realized this is captured there. It took me a while to find the relevant idea in the draft, for anyone reading this thread the key bit is:

As a side-effect, annotating __extra__ with :class:typing.Never for a
TypedDict type will act similarly to decorating the class definition with
@final, because existing fields’ types cannot be changed and no meaningful
extra fields can be added.

So a definition like

MyClosedTypedDict(typing.TypedDict):
    x: int
    y: float
    __extra__: typing.Never

would achieve what people sometimes try to use @final for today, denoting a typed dict for which there can be no unknown keys.

(Update: I also suggested on the draft of PEP 728 that this feature seems important, and might deserve a louder call-out than it currently has)

3 Likes