I’ve had some time to reflect on this PEP. While I do agree that having inlined typed dictionaries would be nice to have, I’m still not convinced it’s worth introducing a third way to create a TypedDict without proposing an actual syntax extension, especially since the third way is less expressive than the other two existing ways.
It will be much harder to get a more powerful syntax extension to the language approved when there are already three existing ways to spell a TypedDict, so I’m worried about painting ourselves into a corner.
I know that it’s currently listed under rejected alternatives, but why not just add a new overload to the functional syntax, that can omit the name and allow that to be used in type expressions[1]? I know that thus far we’ve avoided call expressions inside type expressions, but since TypedDict already needs to be special cased for the call expression anyways and would need another new special case inside type expressions for the subscript syntax, I’m not convinced that it would make type expression evaluation more expensive compared to the subscript syntax.
It would also solve the open issue about PEP 728, since it would all just be the same as the functional syntax now, so if you want a closed anonymous type dictionary you just pass closed=True. It also allows total=False which is a lot less verbose if all or most of your keys are optional and generally would avoid any confusing differences in behavior between the two syntaxes, because it’s no longer a different syntax.
It might give some people the ick purely from an aesthetics perspective, to allow a single call expression to sneak into type expression syntax, but it seems preferable over the current proposal to me.
i.e. when called with only one positional argument, the first argument is the key to type mapping ↩︎
I just read the PEP, and I think it’s worth doing.
But…
The proposed syntax is too “verbose” for me, in the sense that there’s an awful lot of redundant punctuation: There’s an extra pair of curly braces, as well as a pair of string quotes around each name. It would be so much nicer if you could write
def get_movie() -> TypedDict(title=str): ...
The rejection reason for using a functional form sounds kind of weak:
call expressions are currently unsupported in such a context for various reasons (expensive to process, evaluating them is not standardized).
Are those “various reasons” really so strong? For type checkers, TypedDict is a special marker, and we can specify the grammar that follows it. (Which just “happens” to be a valid runtime expression too, and TypedDict can easily overload this argument pattern.)
However, it might be the case that calling TypedDict also happens when using the class-based syntax. I haven’t had time to look into this. Though I suspect that that doesn’t provide keyword args.
The keyword arg based syntax looks prettier to me, too.
The main challenge I see would be around the handling of fields with names that match TypedDict metaclass arguments.
If only TypedDict(total=int) is allowed, using non-default structural settings would be restricted to the named class form.
If only TypedDict(total=False) is allowed, not only would defining fields with names matching structural settings be restricted to the named class form, but adding any new structural settings would pose a backwards compatibility problem.
If both are allowed, type checkers could resolve the ambiguity based on the arg value’s type for boolean flags, but PEP 728’s extra_items arg would remain ambiguous.
The first “all inline typed dicts use the same structural settings” restriction seems the most reasonable option to me. Those settings could be different from the defaults for the named class form, but it would be easier to reason about if they were the same.
The keyword arg syntax also does not support keys that are not valid Python identifiers (either because they contain non-identifier characters or because they happen to be Python keywords). Currently, people sometimes have to use the call syntax for TypedDict instead of the class syntax for this reason.
and I kind of agree it would be unfortunate to have this PEP introduce a way to write inline TDs that would be obsoleted if and when a new syntax to do so (that would presumably cover the partial/omit/etc features). However:
Tweaking the functional syntax to be able to create inline TDs with it suffers from the same issue IMO. If a new language syntax gets introduced at some point, this alternative functional syntax gets obsoleted as well.
Because a new language syntax extension would only be usable in the Python version it is introduced, the proposed syntax in this PEP/the functional syntax without a name would be a way to get inline TDs in earlier Python versions (albeit it would have to be deprecated at some point).
So while I understand that the functional syntax already exists, point no. 1 makes it less of a compelling reason.
Regarding having call expressions inside type expressions, static type checkers maintainers may have more insights, but I believe it is expensive for type checkers to evaluate a call expression (finding the right overload, etc). However, when analyzing type expressions, they would probably just special case the TypedDict call and would actually not perform any evaluation of the call, so I don’t know if this really applies.
However, I think an argument can be made about readability. Currently, parentheses cannot appear in type expressions (apart from a couple rare situations related to empty tuples, e.g. in the tuple_type_expression). If we were to set this precedent, it would be confusing for users to know whether [...] or (...) should be used.
Using the TypedDict(key=<type>) is probably not possible to introduce, considering what was said above (not possible to use invalid identifiers/to set meta arguments such as closed) and the fact that it already existed but was removed in 3.13). We end up with two syntaxes having the same readability: TypedDict[{key: <type>}] vs TypedDict({key: <type>}).
I don’t totally agree with this. The functional syntax would still remain useful. Ending up with an overload that’s now less commonly used, is not the same as having a subscript that’s now never getting used. FWIW I don’t actually think adding the overload is necessary to begin with, apart from some very simple cases, like dictionaries with a single key, I would always prefer to give the TypedDict a name.
However, adding an overload is a valid option, if having to specify a name is a strong enough reason against using the functional syntax for inlined typed dictionaries for enough people. That’s the only reason I brought it up.
I don’t believe that this applies. It would only apply if we started allowing call expressions in general[1], so type checkers would actually need to go through the general call overload resolution path, which can get expensive with generics and a large number of overloads[2].
Neither can curly braces, so either way you’re introducing a singularity around TypedDict into the type expression syntax. I personally feel it’s easier to teach inline TypedDict if there’s only two syntaxes, rather than three, especially since TypedDict({'x': int}) != TypedDict[{'x': int}] under your current proposal if PEP 728 were accepted.
Where I agree with you however, is that, if this PEP were accepted with the functional syntax, we shouldn’t use it as an excuse to add even more call expressions into type expressions and muddy the waters about when to use which one.
In conclusion, even given similar drawbacks, I personally would always choose the more expressive, future-proof syntax[3], even if it looks slightly more out of place in a type expression.
You could e.g. imagine allowing function calls that return a TypeExpr in type expressions, once PEP 747 is accepted ↩︎
I don’t believe it’s as expensive for simple non-overloaded, non-generic signatures ↩︎
if TypedDict gains even more keyword parameters in the future, the inlined typed dictionaries would gain those for free as well ↩︎
And by “the more expressive, future-proof syntax” you are referring to what? (a) TypedDict[{"foo": int}] or (b) TypedDict(foo=int)?
FWIW, I don’t buy the argument that you can’t use Python keywords or non-identifiers as keys when using (b) pretty weak – the same is true for the class-based TypedDict and the solution is the functional non-inline form Foo = TypedDict("Foo", {"foo": int}).
In general when designing Python I’ve not worried much about designing in the way of future syntax improvements; those are really hard to predict and often don’t happen even when anticipated. And we always find a solution. New syntax is held to a much stricted standard though – it must be backwards compatible (not break any existing code) and it must “fit in” the existing syntax. It must also of course be user-friendly and solve a practical (not theoretical) pain point.
Anyway, is there even a future syntax for typed dictionaries that is generally agreed upon as better than what we now have or is being proposed in PEP 764?
(a). Just the ability to specify total=False, closed=True, extra_items=Foo with this syntax on its own makes it more powerful.
I was inclined to agree with you initially, but the downside to this approach would be that, when we want to add more arguments like total/closed/extra_items in the future, that would now be a breaking change. Maybe we won’t ever want to add more, but given that PEP 728 hasn’t even landed yet, it might be premature to make that decision. We can always save this improvement for a later date, when we feel it’s worth the lock-in.
Definitely nothing generally agreed upon. But there is still a lot of open design space for mapped types, which are especially useful for structural types like TypedDict[1]. People generally agree that a comprehension style syntax for mapped types like there is in TypeScript would be easier to read, but if we have to piggyback on a runtime __class_getitem__ call, then comprehension instances are terrible for runtime introspection of that mapped type. So it would be nice to have a dedicated syntax for it[2].
They’re also interesting in the context of ParamSpec and TypeVarTuple↩︎
I’ve floated the idea of TypedDict{}/dict{} for that in the past ↩︎
This PEP seems very interesting to me, but I would like to understand if an additional scenario will be covered by it: “mixing” inlined typed dictionaries with class-based ones.
Most of the times I deal with TypedDicts it’s because I’m describing the request/response payload for external systems (and in the project it’s not necessary to introduce pydantic or other 3rd-party libraries).
Sometimes the data is pretty flat. Unfortunately most of the times is quite nested, and - worse - I really need some “leaf” data structures, while the “internals” of the tree is really not interesting. However I need to define a class for all the levels, even for stuff I don’t really care.
For example:
from typing import TypedDict
from json import loads
class Child(TypedDict):
baz: int
qux: str
class Element(TypedDict):
bar: int
child: Child
class Payload(TypedDict):
foo: int
elements: list[Element]
def get_qux_of_first_child(p: Payload) -> str:
return p['elements'][0]["child"]["qux"]
a: Payload = loads("""{
"foo": 1,
"elements": [
{
"bar": 2,
"child": {
"baz": 3,
"qux": "here"
}
}
]
}
""")
assert get_qux_of_first_child(a) == "here"
While I find useful to define Payload I would prefer to avoid defining also Element, and maybe Child.
Ideally I would like to be able to write something like
class Payload(TypedDict):
foo: int
elements: list[TypedDict[{'bar': int, 'child': Child}]]
or
class Payload(TypedDict):
foo: int
elements: list[TypedDict[{'bar': int, 'child': TypedDict[{'baz': int, 'qux': str}]}]]
So I can define in a “dedicated” class only the TypedDict I really care from a “documentation” point of view
At this point, I wonder if it’s worth reconsidering the simple dictionary for the base case, and the full functional form for cases where parameters or type unions are needed:
I would like to raise a different issue that we might want to address in PEP 764, as per @erictraut‘s comment from 2023, one which hasn’t been discussed so far(?): The ability to infer the TypedDict type for a “constant” dictionary. Consider the following snippet in TypeScript:
const someObj = {
a: 1,
b: 2,
} as const; // type inferred as `{ a: 1, b: 2 }`
We might want a similar mechanism in Python (provided some version of PEP 764 ends up being accepted), which automatically infers an anonymous/inline TypedDict type for the given variable. This would be very convenient for dictionaries you create only in a single place in your application, as you wouldn’t need to repeat the structure of the data twice (once as data, once as type). The latter is particularly annoying when writing complex nested dictionary structures, a use case that PEP 764 is trying to address. Should such a mechanism therefore be included in PEP 764?
Either way, I’m not sure how one would best implement this (short of introducing new syntax).
I think you’re overcomplicating this. Why not re-use Final for this? It already infers Literals, so the leap wouldn’t be that far.
Anyways, I feel like this is beyond the scope of this PEP and should probably be its own thread in the Ideas section if you wish to discuss this further.
I think you’re overcomplicating this. Why not re-use Final for this? It already infers Literals, so the leap wouldn’t be that far.
I think this would be a breaking change? PEP 591 says:
Note that declaring a name as final only guarantees that the name will not be re-bound to another value, but does not make the value immutable.
(EDIT: More generally, my_var: Final = … is very close in spirit to TypeScript’s const my_var = … declaration. My suggestion, however, rather resembles TypeScript’s … as const.)
On top of what’s already been said about mutability, inference behavior of Final isn’t specified and shouldn’t be relied upon. If you need it interpreted as a literal, you should specify it as such even with Final.