PEP 728: TypedDict with Typed Extra Items

PEPs
153

Thank you for the PEP! I’ve read it over and think it is generally very good. It specifies the feature clearly and motivates it well.

I do have a few comments. I’m happy to submit a PR to address all of these if they are uncontroversial.

Disallowing extra items specifically

This can also be resolved by defining a closed TypedDict type.

I’m confused by the “also” in this sentence. It seems to imply either that there is some previously-discussed alternative way to disallow extra items, or that there is some previous problem which is also resolved by the new closed parameter – but I don’t see either of those in the text.

Rationale

A type that allows extra items of type str on a TypedDict can be loosely described as the intersection between the TypedDict and Mapping[str, str].

I think it is more confusing than helpful to describe extra_items in this way, even “loosely”. If we have TypedDict("Foo", {"year": int}), the intersection of that type with Mapping[str, str] would require the key year to have a value which is both int and str, but this is not true of a TypedDict with extra_items.

Readers of the PEP (and the spec which will be derived from it) will either have thought carefully about type intersections, or they will not. For the former readers, this analogy suggests something that is incorrect. For the latter readers, referencing intersections is more likely to be intimidating/confusing than to aid understanding.

I think we should instead describe the semantics of extra_items directly, without any reference to intersections.

I would also remove the subsequent mention of “without introducing general intersection of types” – even if we had general type intersections, I don’t think they would serve as a usable replacement for extra_items.

(This point is correctly noted later in rejected items, but I think it should also lead us to avoid discussion of intersections entirely in the rationale.)

Supported and unsupported operations

Operations with arbitrary str keys (instead of string literals or other expressions with known string values) should generally be rejected.

This restriction does allow __getitem__ on a TypedDict with extra_items to be sound, but it’s more restrictive than necessary. If we have a TypedDict({'a': int}, extra_items=str), then __getitem__ with a str key does not have to be rejected, it could have type str | int.

I’m also curious what @erictraut thinks of including this restriction in the PEP, since my understanding is that pyright already allows this for regular TypedDict, for pragmatic reasons?

As I said earlier in this thread, I think the PEP currently still leaves too much to the imagination when it comes to the core, most common, behavior of a TypedDict – getting items from it and setting items in it. I think the intended signatures of __getitem__ and __setitem__ on a TypedDict with extra_items should be outlined clearly in the PEP. If we want to intentionally leave some flexibility for type checkers to define their own rules here (so some type checkers can opt for more soundness, and others can opt for more ease of use) I think that’s fine, but it should still be stated explicitly that there is implementation flexibility, and what the boundaries of that flexibility are. This reduces uncertainty for type checker authors, and for a future author of a spec conformance suite for this feature.

Assignability

Because 'year' is absent in Movie, extra_items is considered the corresponding key. 'year' being required violates this rule:

  • For each required key in A, the corresponding key is required in B.

I think something is not right in this example? The rule listed here has the context that we are checking whether B is assignable to A. But the code example above shows an attempt to assign MovieWithYear to Movie. So Movie is A – and year is not a required key in Movie. So the rule listed does not seem to apply to the assignment in the example.

Interaction with Mapping[str, VT]

I think it would be clarifying to show the .items() and .values() examples for MovieExtraInt instead of (or in addition to) MovieExtraStr. For the latter all types are str, so it is less clear that the known value types must be unioned with the extra-items type.

Interaction with dict[str, VT]

However, dict[str, VT] is not necessarily assignable to a TypedDict type

The word “necessarily” here suggests that in some cases it might be assignable, but I don’t think this is true? Can we just remove “necessarily” and say “is not assignable”?

Also, I think the code example here is not useful, because it shows a case where the type-checker knows the custom dict to be of type CustomDict. A more relevant example would be like this:

class CustomDict(dict[str, int]):
    pass

def f(might_not_be_a_builtin_dict: dict[str, int]):
    int_dict: IntDict = might_not_be_a_builtin_dict # Not OK

not_a_builtin_dict: CustomDict = {"num": 1}
f(not_a_builtin_dict)

5 Likes