PEP 821: Support for unpacking TypedDicts in Callable type hints

[Daraan]

This document introduces PEP 821:

PEP 821 – Support for unpacking TypedDicts in Callable type hints |...

Currently, we often need to use a more verbose Protocol definition. For example, instead of

class KwCallable(Protocol):
   def __call__(self, **kwargs: Unpack[ATypedDict]) -> Any: ...

we would like to write:

Callable[[Unpack[ATypedDict]], Any]

Additionally, this form allows positional parameters to be listed before the unpacked TypedDict:

Callable[[int, str, Unpack[ATypedDict]], Any]

It also supports substituting a ParamSpec with an unpacked TypedDict:

type C[**P] = Callable[P, Any]
C[Unpack[ATypedDict]]

This PEP is intended to follow existing semantics closely: equivalence to the corresponding Protocol definition; **kwargs typing as per PEP 692; and respect for closed and extra_items as defined in PEP 728.

---

Open Questions

1. The PEP currently proposes to also allow a short form that omits the parameter list as the first argument:

   Callable[Unpack[TypedDict], Any]
   #
   Callable[[Unpack[TypedDict]], Any]
   

   This mirrors the existing exception for ParamSpec and Concatenate, except that those must appear outside the parameter list.

   Should we allow this exception to reduce the visual clutter of the additional pair of brackets?
   On the other hand, such an exception would require a change to the definition of the parameters_expression and might increase confusion about when a parameter list is required and when it is not.

2. Should multiple TypedDicts be supported, i.e.
   Callable[[Unpack[TD1], Unpack[TD2]], Any]?

   This would allow combining and merging multiple keyword-parameter definitions, but raises the question of how to interpret them, especially when keys overlap.
   
   I currently see two ways to interpret such a parameter list:

   2.1. Type checkers treat [Unpack[TD1], Unpack[TD2]] as a subclass declaration:

   class _MergeTD(TD2, TD1): ...
   

   Later bases would appear earlier in the resolution order (to the extent that this matters), and incompatibilities would result in an error. In short, both TypedDicts must have compatible key definitions.

   2.2. A more permissive approach, similar to {**dict1, **dict2}: keys from the later TypedDict completely overwrite keys from the earlier one. In a --strict mode, they might still be required to be at least partially compatible, e.g. broadening from int (in TD1) to int | str (in TD2) is acceptable. This could also allow changes in totality.
   

   Do you think 2.1 or 2.2 introduces too much complexity for type checker implementations? Which approach would you favor, and why? Is there another approach that would be better?

   My current view is that 2.2 is closest to the runtime semantics of dict unpacking and what users might expect. 2.1 might be better expressed using a different form such as intersections or unions, which are currently not defined for TypedDicts.

3. Do any TypedDict-related special forms, such as ReadOnly, require special handling that is not currently covered by this proposal?

---

This PEP is a follow-up to my idea thread: PEP Idea: Extend spec of Callable to accept unpacked TypeDicts to specify keyword-only parameters

[mikeshardmind (Michael H)]

@Daraan

Should multiple TypedDicts be supported

I think if the answer is yes, that this needs to consistently match the runtime behavior and that

@Daraan

In a --strict mode, they might still be required to be at least partially compatible

Isn’t a direction that should be attempted to be specified.

This would mean that the last overlapping required key wins. In the case of optional keys, this needs to describe that a callable could accept the corresponding type from any collected type (a union), unless a later unpacked dict requires the key, which would reset building up the union.

[JakobStadler]

Quick question: In the section Interaction with extra_items (PEP 728), how does the acceptance of e1 work?

class ExtraTD(TypedDict, extra_items=str):
  a: int

type ExtraCallable = Callable[[Unpack[ExtraTD]], Any]

def accepts_str(**kwargs: str): ...
def accepts_object(**kwargs: object): ...
def accepts_int(**kwargs: int): ...

e1: ExtraCallable = accepts_str     # Accepted (matches extra_items type)  <-- this line
e2: ExtraCallable = accepts_object  # Accepted (object is a supertype of str)
e3: ExtraCallable = accepts_int     # Rejected (int is not a supertype of str)

e1(a=1, b="foo")   # Accepted
e1(a=1, b=2)       # Rejected (b must be str)

In my mind, a function that only accepts strings and one that mixes int and strings are incompatible.

[Jelle (Jelle Zijlstra)]

Thanks for putting this together! I think this is a nice little addition to the type system.

Here are my opinions on the open questions:

@Daraan

The PEP currently proposes to also allow a short form that omits the parameter list as the first argument:

I think we should remove this part of the proposal. We allow a similar short form for ParamSpec (where you can omit a layer of []) and I feel that special case has mostly been confusing.

@Daraan

Should multiple TypedDicts be supported, i.e.
Callable[[Unpack[TD1], Unpack[TD2]], Any]?

No. The proposal is for something that’s a short form for a callable with **kwargs, and you can’t have multiple **kwargs in a function. Users who want to combine multiple TypedDict can use inheritance and standard inheritance rules will apply.

@Daraan

Do any TypedDict-related special forms, such as ReadOnly, require special handling that is not currently covered by this proposal?

ReadOnly shouldn’t affect anything here, just like it doesn’t in PEP 692.

@JakobStadler

Quick question: In the section Interaction with extra_items (PEP 728), how does the acceptance of e1 work?

Agree, that seems incorrect.

[erictraut]

Thanks for writing the PEP. Here are some suggestions and questions:

The motivation section is quite clear that the new syntax is simply a shorthand syntax for an existing concept in the type system. That’s a reasonable motivation. The key here is that the new syntax is equivalent to the existing (more verbose) callback protocol syntax. I recommend leaning into this equivalence. By doing so, you can significantly simplify the remainder of the PEP — and in the process making it easier to review, validate, and integrate into the typing spec.

The current draft makes the concept sound much more complicated than it needs to be because it reiterates rules that are already covered elsewhere in the typing spec — handling of optional arguments, handling of extra_items and closed, interaction with ParamSpec and Concatenate, assignability rules, etc. All of these rules should fall out naturally from equivalence to the callback protocol. I recommend removing most of these details and replacing them with “it works the same way as the equivalent callback protocol”. This will not only eliminate complexity, it will also avoid the possibility of introducing conflicting or ambiguous new behaviors in the spec.

---

I’m confused by the statement “Only a ParamSpec may be substituted by an unpacked TypedDict within a Callable.” I read that statement half a dozen times and still don’t understand the intended meaning. First, the placement of the word “only” arguably makes the meaning nonsensical. Second, the phrase “substituted by” is ambiguous. Perhaps you mean “When specializing a generic type parameterized by a ParamSpec, an unpacked TypedDict can be used as a type argument". This should be true regardless of whether a Callable special form is involved. Based on my next point, I think you may want to simply delete this statement.

---

The spec says “Using a TypedDict within Concatenate is not allowed.”. I’m not sure what that means. I can interpret it in one of two ways:

1. You can’t use Concatenate in the new syntax form with an unpacked TypedDict. This means Callable[Concatenate[int, Unpack[KeywordTD]], None] is disallowed.
2. An unpacked TypedDict can’t be used to specialize a generic type that is parameterized with a ParamSpec if a Concatenate is involved in that type’s definition.

I think that interpretation 1 is OK because Callable[[int, Unpack[KeywordTD]], None] is allowed.

I think interpretation 2 is problematic. Consider the following:

class ClassA[**P]:
    def __init__(self, c: Callable[Concatenate[int, P], None]): ...

Is ClassA[Unpack[KeywordTD]] permitted here? I don’t see why it shouldn’t be. After all, the following is permitted today.

def func1(x: int, *, a: int) -> None: ...

x = ClassA(func1)
reveal_type(x) # ClassA[(*, a: int)]

---

The draft says "This feature is mostly an additive typing-only feature”. The word “mostly” here will set off alarm bells to reviewers. I don’t see any ways that it’s not additive, so I recommend deleting that word. If you see a reason to include the word, then an explanation should be provided.

---

You don’t say anything about the use of an unpacked generic TypedDict in a Callable.

class TD1[T](TypedDict):
    a: T

def func2[T](a: T, cb: Callable[Unpack[TD1[T]], None]) -> None: ... # Allowed?

PEP 692 didn’t address this topic, and it remains an ambiguity in the typing spec today. If you take my advice above and lean into the equivalence to callback protocols, then I think you can deftly avoid taking on this complex topic in your PEP. If you don’t invoke equivalence and lay out all of the detailed rules associated with the new syntax, then reviewers will be more likely to insist that you address this topic in your PEP.

---

The draft doesn’t specify whether a ParamSpec or a TypeVarTuple can be used in conjunction with an unpacked TypeVarTuple in the shorthand syntax. Here again, I think the right answer is informed by equivalence to a callback protocol. An unpacked TypeVarTuple is associated with an *args parameter, it doesn’t interfere with the **kwargs parameter. That is, an unpacked TypeVarTuple can be used along with an unpacked TypedDict in a callback protocol. But a ParamSpec is associated with both the *args and **kwargs parameter, so it interferes with **kwargs with an unpacked TypedDict. That implies the new syntax should allow a TypeVarTuple but not a ParamSpec.

class ClassB[*Tv, **P]:
    a: Callable[[Unpack[Tv], Unpack[KeywordTD]], None] # Allowed
    b: Callable[[P, Unpack[KeywordTD]], None] # Not allowed

---

Open Questions:

1. I agree with Jelle that the ParamSpec short form (where you can omit a layer of []) has not been very successful, so I don’t think that’s a pattern we should adopt here.
2. I agree with Jelle that multiple TypedDicts should not be supported. It introduces a bunch of complex validation that type checkers will need to replicate, and reporting errors (e.g. type conflicts for overlapping keys) will be difficult in this context.
3. I don’t think any TypedDict-related special forms require special handling. If they did, I would see that as a big red flag for the proposal. Here again, I think it’s important to lean in to the equivalence.

[Liz (Elizabeth King)]

@Jelle

@Daraan

Should multiple TypedDicts be supported, i.e.
Callable[[Unpack[TD1], Unpack[TD2]], Any]?

No. The proposal is for something that’s a short form for a callable with **kwargs, and you can’t have multiple **kwargs in a function. Users who want to combine multiple TypedDict can use inheritance and standard inheritance rules will apply.

Is that what that’s saying? To me this reads like a callable that can accept both TD1 and TD2 being unpacked as keyword arguments, and that seems like a reasonable thing to want.

It might not be worth the complexity to support, but I don’t think it’s clear that this is just a short form for **kwargs since callable is a protocol that describes usage rather than a concrete interface.