from typing import Any, TypedDict
class Foo(TypedDict):
x: int
foo: Foo = {"x": 42}
def bar(x: dict[Any, Any]) -> None:
pass
bar(foo) # line 11
Checking this with mypy 1.7.1 yields:
foo.py:11: error: Argument 1 to “bar” has incompatible type “Foo”; expected “dict[Any, Any]” [arg-type]
While it makes sense to me that TypedDicts aren’t usually compatible to arbitrary dict types, as this is unsafe, I would expect that TypedDict is compatible with a dict where I explicitly turn off type safety using Any. Currently, there is no way to annotate something as accepting “any dict, I know what I’m doing”. See for example Add tests for TestCase.assertDictEqual() by srittau · Pull Request #11154 · python/typeshed · GitHub where this would be useful and more type safe than the status quo.
It’s hard for me to imagine a common case where the type has to be exactly dict (and not any other kind of mapping), but neither the key type nor value type matters at all. It seems like the example you’re pointing at is specifically about testing the functionality of the dict type itself, in the context of type-checkers… ?
It does bug me a little bit that TypedDict’s type checking behavior does not match its runtime behavior, it does create dict instances after all.
Maybe making typing._TypedDict inherit from dict[str, object] instead of Mapping[str, object] would fix things, but I imagine there’s some special casing in the actual type checkers that would need to change as well.
On that note, it should be compatible with dict[str, Any] and dict[str, object] too, not just dict[Any, Any]. Nevermind, a union of string literals should not cast to str.
That being said, the fact that the value type is not inferred from the members and is just using object makes it generally a bad idea to mix TypedDict with other mappings, especially since the expectation, without deeper knowledge of how type checkers infer TypedDict, would be that TypedDict is more safe than a regular dict not less.
As a side note: I never really understood the performance arguments type checkers use to justify not giving a more precise value type, since you would only really need to create the union of all value types once when the TypedDict is defined and afterwards it’s just as fast as any other type. The value type union could be built at the same time as the structure containing all the key value type pairs is built.
If type checkers did not take this shortcut, I think the need for making TypedDict act as a subclass of dict would increase and lead to a much stronger case for making this change.
This was a tradeoff between type safety and flexibility. Doing potentially unsafe things with dict[str, Any] (including mutation) seemed common in the code we analyzed while building the feature in mypy, whereas legitimate use cases where Mapping[str, Any] or Mapping[src, object] couldn’t be used instead seemed rare. Before we had TypedDict, most TypedDict-like data was annotated as dict[str, Any], and there was a real danger of violating runtime type safety, in case only a subset of relevant annotations were changed from dict[str, Any] to a TypedDict type. By having type checkers enforce this made it a lot easier to migrate from dict[str, Any] to TypedDict types.
Allowing compatibility with dict[str, Any] seemed like a fairly big practical type safety hole for relatively little gain. To argue for changing this, it would help to have more evidence that our original analysis of the tradeoffs did not represent typical use cases well. For many design decisions it’s easy to come up with counter-examples where the opposite design would be more helpful, but the most important bit is figuring out the overall cost/benefit across many different use cases.
I agree that passing a TypedDict to dict[str, Any] would be unsafe, as that dict explicitly allows any string to be used as a key. That’s not the case for dict[Any, Any], though. (In theory, we could also allow a TypedDict called X to be passed to a dict[KeyOf(X), Any], although that’s even esoteric.)
I don’t follow. dict[Any, Any] also allows any string to be used as a key, so it is also unsafe.
Editing to add: Any kind of implicit conversion to Any seems like it would be unsafe to me, regardless of whether we’re talking about TypedDicts or not.
That is the point of Any though. It is a gradual type, meaning matching any type in either direction is fine. Otherwise you wouldn’t need Any, you could just use object instead [1].
At least for covariant type parameters, for contravariant ones you would need Any too ↩︎
It is a gradual type, meaning matching any type in either direction is fine.
TypedDict types can be used when Mapping[str, Any] or plain Any are expected, so this seems to be working as intended. Similarly, Mapping[str, Any] can’t be used when dict[str, Any] is expected (but the other direction works).
A TypedDict type isn’t treated as a subtype of dict[str, <something>], since it doesn’t provide various destructive methods such as clear(). TypedDict is not just a special kind of dict type but a completely separate type from a type system perspective, though values happen to be dict instances at runtime.
I was replying to @davidfstr. But thanks for refreshing my memory about what a TypedDict is supposed to be from a typing perspective. I can now see why it shouldn’t be treated as a subclass of dict, even though it technically is one at runtime.
I think the problem is that the function (that accepts a dict) could write into the value passed in at any key. But typed-dicts don’t support writing at any key.
Also destructive changes, such as deleting a key that must be there, or clearing the dict altogether, removing all the keys.
If we had an Immutable type modifier, then any TypedDict would safely pass as a Immutable[dict[str, object]], although that type modifier seems really difficult to implement correctly without deep introspection of the methods and thus really expensive, unless you also add a way to explicitly annotate the methods that are fine to call on an immutable instance, forbidding every method by default.
Any is an escape hatch, but it doesn’t help you on the interface side because you need the interface to be more restrictive—not more general. If you want Any to help you here, pass an object of type Any rather than an object of type TypedDict.
In this case, it would be a lot simpler to just accept Mapping if you’re really not going to mutate the dictionary.
Yes, although there is a subtle difference, you lose the information that you’re dealing with an instance of dict, which you might care about if you’re doing things like an isinstance check. But I agree that in the vast majority of cases Mapping is sufficient.
As a side note: Mapping is not actually sufficient to guarantee immutability, in mutability analysis you should e.g. reject defaultdict in a function that accepts Immutable[dict], since __getitem__ on defaultdict can cause new keys to be inserted.
