Generally, TypedDict types are not subtypes of any specialization of dict[...] type, since dictionary types allow destructive operations, including clear(). They also allow arbitrary keys to be set, which would compromise type safety.
However, this seems a bit vague w.r.t. checks of the form isinstance(obj, dict), when obj can be a TypedDict. Consider the following example, annotated with what I would personally expect from a type checker here.
from typing import TypedDict, reveal_type
class Config(TypedDict): pass
def expects_dict(arg: dict) -> None: ...
def expects_config(arg: Config) -> None: ...
def check_1(obj: Config | str) -> None:
if isinstance(obj, dict):
reveal_type(obj) # E: Config
return expects_dict(obj) # bad since it may call dict.clear()
reveal_type(obj) # E: str
def check_2(obj: Config | str) -> None:
if isinstance(obj, dict):
reveal_type(obj) # E: Config
return expects_config(obj) # type safe.
reveal_type(obj) # E: str
I checked this with GitHub - astral-sh/multiplay: multi-typechecker playground · GitHub, and only 2 type checkers give the expected result (pyright and zuban). mypy and pyrefly thinks the inside of the if-branches are unreachable, and ty and pycroscope have false negatives on the expects_dict call.
Thinking a bit about it, it seems that with TypedDicts as a special case, we kind of want them to be subtypes of (for the purpose of narrowing), but not assignable to dict. However, under the current definitions in the spec this is contradictory (Glossary — typing documentation).
I thought I’d get some opinions here before opening an issue in python/typing.
That’s right, although one can modify that example to a sitatuation where using isinstance(obj, Mapping) instead of isinstance(obj, dict) would be unsafe. For instance, say we have some object factory that should initialize objects from config dictionarys (think hydra.utils.instantiate), and pass already instantiated objects through.
from typing import TypedDict
class Config[T](TypedDict):
cls: type[T]
def initialize[T](spec: T | type[T] | Config[T], /) -> T:
if isinstance(spec, type):
raise NotImplementedError
if isinstance(spec, dict):
raise NotImplementedError
return spec
If T can overlap with Mapping, then replacing isinstance(spec, dict) with isinstance(spec, Mapping) is unsafe. For example, T could be upper bounded by torch.nn.Module, but a subclass like torch.nn.ModuleDict may mixin Mapping.
But we’d want to have isinstance(..., dict) keep working anyway, right? So, isn’t it fine in this example? If I understand it correctly, the point here is that we need to narrow on dict specifically here because we don’t want to capture objects that merely implement Mapping.
While I’m all for fixing the spec to be both more correct and sound, I personally consider TypedDicts to be a mistake so I’m not super invested in driving the change.
They are dicts at runtime, but are not valid replacements for dicts at type time. This creates the contradiction here, as effectively, TypedDicts violate substitution.
There’s probably a way to word this better in the spec, but there won’t ever be a good outcome that’s totally consistent without inventing some new “type-only allowed use constraint, allowed to break substition rules”, or changing TypedDict to not be a dict at runtime.
One way to deal with this situation is to model isinstance(..., dict) better. This doesn’t require any strange exceptions to the subtyping/assignability rules. Instead, isinstance(..., dict) should not narrow to dict (as most type checkers currently do), but to a type that’s essentially dict | any TypedDict. The “any TypedDict” type can be spelled as class EmptyTypedDict(TypedDict, extra_items=ReadOnly[object]): pass: any TypedDict is a subtype of that. Ty is likely to implement this approach in the future.
that approach looks worse for anyone narrowing something that isn’t already seen as a typed dict, as it would ban any use of the mutating methods of dict. (one side of that union disallows it) just because it might be used as a typed dict somewhere.
Anything not seen as either a dict or a TypedDict, which is a much less common case. If you narrow e.g. dict[int, str] | None with isinstance(..., dict), you’d still narrow to just dict[int, str], with no TypedDict component. (Edit to clarify: not because this needs to be special-cased in the narrowing logic, but this falls out naturally because EmptyTypedDict is disjoint from dict[int, str] – and from None as well, of course.)
The case where you’d actually see both dict[...] | EmptyTypedDict is basically when you are narrowing from a prior type of object. In this case you can give up any kind of soundness and narrow to dict[Any, Any] (as most existing type checkers do). If you want soundness then the type “all possible dicts” is already extremely restrictive, it doesn’t seem significantly more restrictive to also prevent .clear() to maintain soundness with possible TypedDicts.
and if you had dict[str, object] | SomethingElse narrowed by isinstance on dict, that’s not disjoint from the potential of a TypedDict, so narrowing to dict via isinstance suddenly errors if you want to use .clear?
This is a bad outcome that results from TypedDicts being themselves unsound and not a safe replacement for dict
That’s the crux of this issue. all typed dicts are dict[str, object], and have a .clear method at runtime, and we arrived here by runtime narrowing (in this case, isinstance) You can’t treat all possible dicts as “maybe this is a typed dict”, but then somehow exclude the more specific dicts that might still be typed dicts.
I don’t think the issues here are fundamentally any different from the issues that already exist with all generics. A list[str] is disjoint from a list[int] as static types despite both being inhabited by the same indistinguishable runtime empty list object. Both create awkward impedance mismatches, yes.
It’s easy to say “everything is terrible and there is no good solution,” but it’s not very useful to users of the type system. I think treating a runtime check of dict instance narrowing to the union of all dict and all typed dict types doesn’t actually create significant new problems, and is the best available option.
I don’t think this is the right way to handle this.
I’m not saying that there isn’t an improvement possible, but that this wouldn’t be an improvement. It’s prone to many issues we already have around isinstance not matching type semantics, but in an arguably worse way than the existing ones.
I think here, the pragmatic answer is that because TypedDicts are already a special case, where at runtime it’s a lie, typecheckers need to take on the burden of tracking TypedDicts and understanding that they are a subtype of dict at runtime, so they branch into the positive branch of the isinstance(..., dict) call, but without doing it in a lossy manner, and not suddenly allowing .clear, rather than treating everything that passes into such a narrowing construct as suspect.
No special casing other things that pass into it that aren’t typed dicts as “maybe typed dicts” here.
Food for thought, the same argument applies to treating isinstance(..., collections.abc.MutableMapping) as “maybe a typed dict”, as well as any user defined runtime checkable protocol that isn’t disjoint from a dict at runtime.
Where do you draw the line when trying to special-case the rest of the world to handle typed dicts instead of keeping typed dicts narrowly special-cased and just handling the places where a type checker knows it has one, rather than places it might have one?
Might be stupid for reasons I don’t understand, but why not making TypedDict completely compatible with dicts ? From what I understand, the problem with TypedDicts interpreted as dicts is that one could :
.clear()
del typeddict[key]
typeddict[key] = newval
.pop or .popitem
Why not making TypedDict an actual type that would be a subtype of dict[str, object] where those methods are implemented as well but in a way that they might crash at runtime rather than a virtually labeled dict where those are just virtually forbidden ? Like, calling .clear() on a TypedDict that has required items would throw an error, same for replacing values that are ReadOnly etc. However, those methods should run fine whenever they can.
To me it sounds it solves most problems and should be type checkable, except maybe calls to .popitem as those relate to a LIFO structure that type checkers might not be able to detect. To me it still sounds acceptable, it will just raise at runtime and that should be fine ?
Indeed, maybe it should be the other way around then ? dict[str, object] (and other dict[str, ValType]) can indeed be seen as a strict subclass of TypedDict that provides the extra methods for mutability. After all, you’re right, it’s the common pattern in Python, e.g. MutableMapping being under Mapping etc.
But then a new runtime class has to be created rather than just relabeling dicts.
