While that is true, it’s worth remembering that this is very unlikely to come up in practice: it happens only if you want a TypedDict key that is literally called __extra_items__.
I think the type system should make something like that possible, but I’m OK with it being a little less obvious than a more normal case.
This is a lot stronger than the text in the rejected ideas section of the PEP, which comes across much more like “here’s a couple bad things which are already true for TypedDicts anyway”. I’m not able to get a good idea for why it’s fine for all these to be a problem for anyone forced to use the functional syntax due to language constraints in the class syntax (keys with dashes in!) but a blocking concern for users of the exact same type accessing a feature that I would expect to see much less usage. Is there something specific about putting a type in a keyword argument versus dictionary value I don’t understand?
Isn’t this more a consequence of the specification and user-facing docs not having a good separately defined term and explanation of the difference between these, and something we can remedy with time, without blocking typing features that have a reason to do things this way?
The other places where this user confusion exists aren’t going away, and likely need better documentation to support anyhow (see Annotated and the need for typing.TypeForm/SpecialForm/whatever other name we go with)
I just released typing-extensions 4.10.0, which adds runtime support for this PEP.
The PEP is also supported in pyanalyze 0.12.0 and is expected to be supported in the next release of pyright.
Just spotted this in the PEP:
If there is no required key, the TypedDict type is consistent with dict[KT, VT] and vice versa
I believe this contradicts the rule that a TypedDict cannot be a subclass of dict at runtime? Which is a constraint that dict itself doesn’t have.
I think the consistency can only be in one direction.
You only quoted half the statement, the way it’s written in the PEP, bidirectional compatibility makes sense, although the use is fairly limited, since it essentially amounts to having dict[str, VT] where a few optional keys are hinted at, but in the end you get about the same amount of safety as you would otherwise. The only real use I can see there is with Unpack and **kwargs, so your editor can hint at some of the commonly expected arguments, but still allowing you to just unpack a generic dict[str, VT] into kwargs, rather than forcing you to use a literal or the TypedDict itself.
But it still doesn’t hurt to have this rule, as narrow as it is.
The rule could probably point out better how this composes with PEP 705. Since it could be a little bit more useful there, since it would allow you to essentially define a type safe Immutable[dict[str, VT_co]], which is something I have been wanting, because sometimes Mapping doesn’t match the runtime isinstance(x, dict) check that API uses.
I quoted all I need to. I don’t think you can ever safely assign from a dict to a TypedDict because the value at runtime could be a subclass.
class MyDict(dict[str, str]): pass
class TD(TypedDict, closed=True):
__extra_items__: str
d: dict[str, str] = MyDict() # ok
td: TD = d # must be illegal!
I don’t really see why this needs to be illegal, could you elaborate?
PEP 589 states (emphasis mine):
TypedDict objects are regular dictionaries at runtime, and TypedDict cannot be used with other dictionary-like or mapping-like classes, including subclasses of dict.
And elsewhere
the runtime type of a TypedDict object will always be just dict (it is never a subclass of dict).
As far as I’m aware, no PEP has ever changed that, and I didn’t see it in this one. If you’re intending to change it, it probably needs to be a big part of the upfront text and justification given
I agree that spec/docs inconsistency is probably one of causes to the user confusion around mixing type expressions and value expressions, and it is fixable in long term.
Given the current concerns about extra=type adding more confusion, I think it would be helpful/convenient to avoid this pattern for new typing features just so it doesn’t become one of the exceptions we need to address with documentation or future spec changes. I don’t see strong arguments against moving forward with closed=True.
That’s fair. The limitation makes sense for TypedDict historically because otherwise you could smuggle in new keys and values or remove keys that need to be there. But in this particular exception that PEP 728 carved out, it seems completely type safe to make it bidirectionally compatible, because there is no way to violate the contract of the TypedDict, even with a subclass, unless you ignore LSP errors, and those would not be type safe with regular dict either.
That being said, it doesn’t necessarily need to happen[1], if it turns out be too complicated for type checkers to support. It would introduce a weird edge case, that’s maybe a little bit too subtle and difficult to understand for people in order to actually be useful.
compatibility in one direction with TypedDict would already be a win ↩︎
Agreed. Just hoping to get the best possible arguments against the alternative captured for the PEP.
You will need to clearly specify under what circumstances a TypedDict can now be a subclass of dict at runtime and put that somewhere in the PEP.
Agreed. @PIG208 could you extend the “Interaction with dict[KT, VT]” section that this means that some closed TypedDict types can now be a subclass of dict at runtime.
By the way, I’m not sure if this was the only reason subclasses were banned. It seems from the PEP text that ruling out people wanting to add other methods (e.g. by mixing in protocols) was a motivation. You may be unintentionally opening a can of worms? But I have no insight beyond the text of the PEP.
Hmm. I didn’t realize that. It would be fine if it is just consistency in one direction though, right? So this would be fine:
class IntDict(TypedDict, closed=True):
num: NotRequired[int]
__extra_items__: int
not_required_num: IntDict = {"num": 1, "bar": 2}
regular_dict: dict[str, int] = not_required_num # OK
I believe so
I’ve published pyright 1.1.352, which has support for draft PEP 728. To use this functionality, you will need to set enableExperimentalFeatures to true in your pyright config.
This version incorporates the latest thinking about type consistency being valid in only one direction.
Here’s a code sample in pyright playground
If you install pyright via the community-maintained Python wrapper (published in pypi), it may take several hours before the new version is available.
If you find any issues with the PEP 728 implementation in pyright, please file bugs in the pyright issue tracker.
If you’re curious, here’s the commit for this change. Compared to other typing PEPs, this change was more complex than some but less complex than others — roughly in the middle of the spectrum.
Re-reading the quoted section, I believe the concerns were about disallowing defining additional methods on TypedDict itself, which makes complete sense, since the way this PEP was designed it was impossible for anything other than a dict literal to be used to fulfill a TypedDict, unless you willfully make a bad cast or pass in Any.
Once you use a subclass there’s no longer an easy way for type checkers to validate the presence/absence and types of individual keys.
So this was more about preventing misguided complexity in the TypedDict constructor and re-implementing Protocol for arbitrary mutable mappings. Not about preventing TypedDict runtime values to be anything other than a dict instance, since you can’t really prevent that anyways.
So I’ll hold that bidirectional compatibility seems fine and it would allow some useful patterns in conjuction with PEP 705, that were previously impossible, without relying on protocols like Mapping.
class ReadOnlyParams(TypedDict, closed=True):
__extra_keys__: ReadOnly[str | float | bool]
def takes_params(params: ReadOnlyParams) -> None: ...
params: {'foo': 'bar'} # inferred type is dict[str, str]
takes_params(params) # this works now, even if inference doesn't look ahead
This is probably not the most compelling example. This comes up more often in stubs, where you can’t change the implementation to fit the type hint.
But maybe @JukkaL has some insights as to why that might still be a bad idea today if both PEP 728 and PEP 705 were accepted.
I don’t believe the current iteration of the text allows this example, as it only allows compatibility if the types are identical.