I would not expect the extra key field name to inherit.
I see that the extra_key parameters proposal in its current form does not address the issue of potentially breaking existing codebases, but rather the need to type the reserved item when the user is aware of the breakage or when they just want to use the key.
What I’m not sure about is the impact of the default, and if it is something we can deal with by educating the users to use the extra_key parameter. An open-source code search from before suggests that the majority of __extra__ are from non-relevant typing_extensions code. We could do similar checks if we want to use a briefer name.
A concern that I had when writing was this:
class A(TypedDict, extra_key="test"):
__extra__: bool
test: ReadOnly[int | None]
class B(A):
test: int
__extra__: int # Valid type narrowing, but "__extra__: bool" from "A" is lost now.
Maybe we can define a behavior that makes sense, but it feels kind of confusing if the subclass uses a different extra key name. This is less of an issue for type consistency checks (at least for me) because we aren’t dealing with inheritance.
Here are a couple of additional ideas to throw into the mix.
The first idea involves the use of a method declaration within a TypedDict.
Currently, TypedDict definitions don’t allow any method declarations. We could specify that a __closed__ method can be declared that returns the type of the allowed ‘extra’ attributes.
class Point(TypedDict):
x: float
y: float
def __closed__(self) -> Never: ...
This idea has the benefit of not affecting backward compatibility.
The downsides are that it’s probably not what most Python developers would expect. It’s also not clear how this would work with the functional form of TypedDict.
The second idea involves the use of a new parameter to the TypedDict constructor named closed. If set to True, the TypedDict would be considered “closed” in that type checkers should assume that no additional items are allowed. If a user wants to allow additional items of a specific type, they would additionally include an __extra__: <type> attribute declaration. If no __extra__ is present but closed is set to True, it acts as though an implicit __extra__: Never is present.
class Point1(TypedDict, closed=True):
x: float
y: float
class Point2(TypedDict, closed=True):
x: float
y: float
__extra__: int
I think this construct feels pretty natural — especially in the common case where the user simply wants to specify that no other items can be present. It also preserves backward compatibility and works with the functional form. It could also be made to work with a potential future inlined version (by special-casing __extra__ in the inlined version).
In the (extremely unlikely) event that you want to create a closed TypedDict with a key named __extra__, you could do so through inheritance.
# Create a non-closed TypedDict that includes `__extra__` item
class TDWithExtra(TypedDict):
x: float
y: float
__extra__: float
# Create a closed version of the above
class ClosedTDWithExtra1(TDWithExtra, closed=True):
...
# Or a closed version with extra items
class ClosedTDWithExtra2(TDWithExtra, closed=True):
__extra__: int
4 Likes
I’m in favor of the “closed” solution, which does seem easier to implement than extra_key=.
Looks like it would make sense to inherit closed=True when subclassing and to require that it can never be overridden.
I like your closed idea best so far of what I’ve seen, although I’d go with Jelle’s suggestion of changing the reserved name from __extra__ to _ in that case[1], I think the analogy with the match statement fits.
as long as we have a way to express a
TypedDictwith a_key through inheritance, I don’t think we need to be as worried about choosing a name with more collisions ↩︎
1 Like
I don’t understand from the PEP what the problem was with class Foo(TypedDict, other_values=int). It seems unambiguous and easily maps to functional syntax. I don’t understand why it would make inheritance harder as the PEP claims, or why generic TypedDicts are brought up, or why there are corner cases around “treating a type as a value”. Could someone clarify that whole section?
See the second reply in this thread.
I think the most problematic thing about type as a value is, that it can’t be a forward reference, unless you wrap it in a string, which is the kind of thing we would like to get away from. In the functional case I think the argument is not as strong, although type checkers would have to special case the parameter in the absence of TypeForm, in order to reject values that are not valid types, which is the other thing that’s not as nice about it.
I think it’s definitely desirable to have a way to specify the type of the extra keys with an annotation with the class syntax, so it can be an implicit forward reference.
Also true for the functional form which is nevertheless the only solution to the common problem of keys that aren’t valid property names, so I don’t see why that’s important.
I haven’t seen this addressed for any proposal. Why is this especially hard for this one?
Half of this is true of every proposal, and isn’t the other half also true of functional syntax already?
Also true of the new “closed” proposal
For the functional syntax you can probably re-use the original special casing of treating the dictionary that’s passed in like the body of the class syntax[1], whereas if you add a parameter that is a type, you have to add entirely new special casing and write code to reject invalid values for that specific parameter, since the annotation for the parameter would currently have to be Any.
ignoring valid identifier names ↩︎
Raced with you 
I don’t see how demand for this new feature plus forward references is going to be high enough to justify going through contortions with a more complex syntax.
Wouldn’t that break keys that don’t follow Python naming convention?
I think closed is actually more concise in many cases, since __extra__: Never is probably the thing I want to do most often, so not having to import Never is great. I’ll gladly pay typing seven[1] extra characters in the other cases for that.
Actually on second thought it’s less than seven additional characters, since closed is shorter than extra_keys.
Trueand_:↩︎
3 Likes
You wouldn’t generate code that needs to parse using the Python grammar, you would directly construct the AST nodes you need, so no naming restrictions there.
To be clear, I really like the closed proposal, I think it makes the common case much simpler. I just want the other option properly dissected, and ideally the rejected ideas section rewritten to stand on its own without needing to reference discourse posts.
1 Like
It’s less the extra characters and more the extra mental complexity of the change of behaviour of a special reserved key in the presence of a (possibly inherited? are we allowing this to be silently inherited?) keyword parameter.
I would still prefer something verbose like __other_values__ over the snappy but magical _ for this reason, especially now it’s not needed for the Never case. I might guess what effect “other values” had on first seeing it.
I think just like with total it should only apply to the current class definition, the property that the extra keys are restricted is inherited, but if you want to override what type they are restricted to[1], then you need to set it again, just like you need to set total=False again in a subclass of a total=False TypedDict if you want the keys to once again be NotRequired by default.
given that we even want to allow that ↩︎
2 Likes
Overriding makes sense if your earlier restriction was marked ReadOnly. I think it makes sense to allow any structural subtype to make that relationship explicit using explicit inheritance.