PEP 821: Support for unpacking TypedDicts in Callable type hints

[Daraan]

First a bit thank you to @erictraut for the thorough review, feedback and input! Likewise to @Jelle as well as all you others for the feedback

I revised the PEP to stronger lean into the equivalence with callback protocols as suggested, i.e. Callable[[Unpack[TD]], R] ≡ Protocol with __call__(self, **kwargs: Unpack[TD]) -> R. Parts of the original PEP that restated rules for optional keys, extra_items, and assignment compatibility have been removed, since they now follow directly from that equivalence.

Draft PR | RDT version for the update

---

@Liz

@Jelle

@Daraan

Should multiple TypedDicts be supported, i.e.
Callable[[Unpack[TD1], Unpack[TD2]], Any]?

No. The proposal is for something that’s a short form for a callable with **kwargs, and you can’t have multiple **kwargs in a function. Users who want to combine multiple TypedDict can use inheritance and standard inheritance rules will apply.

Is that what that’s saying? To me this reads like a callable that can accept both TD1 and TD2 being unpacked as keyword arguments, and that seems like a reasonable thing to want.

It might not be worth the complexity to support, but I don’t think it’s clear that this is just a short form for **kwargs since callable is a protocol that describes usage rather than a concrete interface.

Multiple unpacking of TypedDicts would be a new concept that has no direct Protocol equivalence that can be written with **kwargs, only explicitly, i.e.
Callable[[Unpack[{'a' : int}], Unpack[{'b': str}]], Any], would relate to __call__(self, a: int, b: str).
I would edit the first post to make this more clear but can’t anymore

FWIW the complete picture: For non overlapping keys its easy and can be replaced by an inherited dict, otherwise we would need new rules on how to merge such cases. Callable[[Unpack[{'x' : int}], Unpack[{'x': str}]], Any] → __call__(self, x: ???)
I’ve moved the multiple dict approach into rejected ideas to stick to the equivalence and laid it out there in more detail.

---

---

@JakobStadler

Quick question: In the section Interaction with extra_items (PEP 728), how does the acceptance of e1 work?

As Jelle said that was a mistake. I’ve removed the section during the revision.

@Dutcho

Other uses of Unpack[] can be abbreviated to */**. Is that also the intent here?
If so, it may be good to document in the PEP (I couldn’t find it).
If not, that may be confusing and in need of proper documentation

No, there is not intent to add this. While I agree */** (e.g. **TD in place of Unpack[TD]) is an appealing and logical shorthand, it was touched on in the original idea thread, similarly in PEP 692. It would require a grammar change; **-unpacking can currently not be used in type-hints. I see this as a future proposal that involes more work and acceptance.

---

@erictraut

The spec says “Using a TypedDict within Concatenate is not allowed.”. I’m not sure what that means. I can interpret it in one of two ways:

1. You can’t use Concatenate in the new syntax form with an unpacked TypedDict. This means Callable[Concatenate[int, Unpack[KeywordTD]], None] is disallowed.
2. An unpacked TypedDict can’t be used to specialize a generic type that is parameterized with a ParamSpec if a Concatenate is involved in that type’s definition.

I mean it closer to 1, you cannot use it in Concatenate, e.g. Concatenate[Unpack[TD], P] as wit would lead the resolution problem.

@erictraut

I’m confused by the statement “Only a ParamSpec may be substituted by an unpacked TypedDict within a Callable.” […] Perhaps you mean “When specializing a generic type parameterized by a ParamSpec, an unpacked TypedDict can be used as a type argument". This should be true regardless of whether a Callable special form is involved.

Here I want to clarify that the case that you labeled with 2) on the quote directly above is allowed, and clarify that only a ParamSpec and neither a normal TypeVar nor TypeVarTuple can be substituted by it.

So, your version “When specializing a generic type parameterized by a ParamSpec, an unpacked TypedDict can be used as a type argument" is what a mean

This should be true regardless of whether a Callable special form is involved. […] I think you may want to simply delete this statement.

I think so too: that it should be true, but as far as I see it is not supported.

For now, I moved the part into Open Questions. To me it seems to indeed remove the statement and defer the support / stick to existing spec rules regarding ParamSpec are the better way, else the scope would get broader.
What are your stances on this?

@erictraut

The draft doesn’t specify whether a ParamSpec or a TypeVarTuple can be used in conjunction with an unpacked TypeVarTuple in the shorthand syntax.

Well spotted, for now I extended it with notions on both. For the ParamSpec though, isn’t Callable[[P, <Anything>], R] invalid per the grammar already, and that Unpack[TD] may not be in Concatenate covers that it cannot be alongside ParamSpec? - see the Interaction with ParamSpec, TypeVarTuple, and Concatenate section.
As the paragraph imo covers an invalid grammar I would rather not include it; what are your opinions about it?

---

@erictraut

You don’t say anything about the use of an unpacked generic TypedDict in a Callable.

class TD1[T](TypedDict):
    a: T

def func2[T](a: T, cb: Callable[Unpack[TD1[T]], None]) -> None: ... # Allowed?

I cannot say I grasp the whole scope of this, but thinking about variance it gets weird and ambiguous on how to check compatibility with cb.
It looks interesting to discuss and define, but I believe the answer is not in this PEP