PEP 692: Using TypedDict for more precise **kwargs typing

PEPs
81

The SC is happy to say that we accept the PEP (w/ the syntax changes removed)!

And to reiterate, the SC is not wholly against the syntax change, but we do feel it deserves a separate PEP for a more thorough discussion.

8 Likes
82

The PEP mentions

Also, for the benefit of IDEs and documentation pages, functions that are part of the public API should prefer explicit keyword parameters whenever possible.

To me this indeed seems like a good reason to still prefer duplication over using this new feature. But I wonder if maybe IDEs/LSPs will be able to do the unpacking for the user, i.e. show the TypedDict fields as normal keyword parameters (completely hiding the use of **kwargs), and auto-complete the parameters.

@erictraut (hope you donā€™t mind my question), do you think this will be possible to do in the Pylance LSP, or is there something preventing it?

83

The current implementation in Pylance does not show the TypedDict fields as normal keyword parameters, nor does it auto-complete the parameters if PEP 692 is used. These are both theoretically possible to do and (IMO) worth considering. User feedback plays heavily into decisions made by the Pylance team, so if this is something youā€™d like to see, I encourage you to submit an enhancement request in the pylance-release issue tracker.

84

Iā€™m not aware of any library using this feature yet, so no need from me, but itā€™s good to know that itā€™s possible, thanks.

One thing that will definitely be lost is seeing the value of the parameter defaults in the hover/documentation, so NotRequired fields will need to be shown as param=... instead of param=100, I donā€™t think thereā€™s a way around that.

1 Like
85

I support this proposal. I want to make it easier to make a hook function and pass arbitrary context values to it, but in reality there is a type parameter that is an Enum and each enum value has a different Context dictionary.

86

I like the idea. In a future, kwargs could be annotated as immutable maps.

87

Iā€™m trying to use Unpack, but Iā€™m finding it too limited for my use case. I asked about it on StackOverflow and it was suggested that I bring it up here, because PEP 692 isnā€™t yet final.

There are two ways to specify types for **kwargs:

  • The new way, with TypedDict and Unpack. This allows only predetermined argument names, and each can have a different type.
  • The normal way, like **kwargs: T, where T is some type. This allows any argument names, and they all have to be of type T.

The issue is that there are many practical use cases where you might want to do both. You might want to specify some arguments types with TypedDict, but also allow any other argument names, and for those arguments, they should all be of some type T.

For example, suppose I have this function:

def foo(*, x: bool, **kwargs: int) -> None:
    ...

And suppose it is called from many other functions, like this:

def f(a: str, b: str, x: bool, **kwargs: int) -> None:
    print(a + b)
    foo(x=x, **kwargs)

I would like it if I did not have to put x in fā€™s function signature, and also not explicitly pass x to foo().

I wish I could do something like this with TypedDict and Unpack, where _other_ represents any other argument names that are passed in:

class FooKwargs(TypedDict):
    x: bool
    _other_: int

def foo(**kwargs: Unpack[FooKwargs]) -> None:
    ...

def f(a: str, b: str, **kwargs: Unpack[FooKwargs]) -> None:
    print(a + b)
    foo(**kwargs)

Itā€™s worth mentioning that, in a response to my SO question, @sterliakov suggested some other possible paths:

There was a sort of similar addition to dataclasses in 3.10: KW_ONLY constant that makes following attributes kw-only in constructor, so a precedent exists. It may also work as a metaclass argument (so that you do smth like class MyArgs(TypedDict, rest=int) ).

88

This has been discussed before; see Mailman 3 Any way to express that a dict is expected to have a key(s) but don't care about other keys? - Typing-sig - python.org. Itā€™s a reasonable idea, but itā€™s logically separate from PEP 692 and should be considered in a new PEP. If you are interested in pursuing this idea, Iā€™d encourage you to start working on a PEP.

3 Likes
89

Thanks, things are pretty busy for me with other commitments, but Iā€™ll think about working on a PEP in the future. But if anyone else is interested in doing it, please feel free to go ahead and do it!

90

I would like to help with writing up a PEP for the proposed __extra__ attribute idea proposed by Eric in that thread. Are you willing to be listed as a sponsor in the PEP?

91

Not right now as I have too many PEPs in flight. However, I can provide feedback on the draft and perhaps one of the other core devs is willing to serve as a sponsor.

1 Like
92

Iā€™d be happy in principle to sponsor a PEP proposing that idea @PIG208! I agree it would be a pretty useful feature

1 Like
93

Also wanted to note that I added experimental support for this feature to the pyanalyze type checker: https://github.com/quora/pyanalyze/blob/9bfc2c58467c87774a9950838402d2657b1486a0/pyanalyze/extensions.py#L590

Itā€™s spelled like this:

@has_extra_keys(str)
class TD(TypedDict):
    a: int

def f(x: TD) -> None:
    assert_type(x["a"], int)
    assert_type(x["arbitrary_key"], str)

The decorator has the advantage that it doesnā€™t require changes to TypedDict itself. Thatā€™s not as much of an issue for a standard feature, though.

1 Like
94

Following up on this thread, I believe my issue above is solved with PEP 728 and Unpack.