Introduce a TypedMapping (analog to TypedDict but frozen)

[Kound (Carli* Freudenberg)]

I prefer frozen datatypes as they prevent accidental modification of data that isn’t supposed to be modified.

Often I have to move around datas in dicts (i.e. read from a json).
To have this data properly typed I used TypedDict.

But I can’t set the TypedDict to “ReadOnly”.

Currently I do something like this

from typing import TypedDict
import json
import types

class Data(TypedDict):
    foo: str
    bar: int

def read_data() -> Data:
    with open("file.json", "r") as f:
        return cast(Data, types.MappingProxyType(json.load(f))) 

Which works fine to prevent runtime modification of data. But I would like the TypeChecker to understand as well, that setting data is invalid.

Would it make sense to introduce a TypedMapping to Python which is basically a TypedDict but without the __setitem__ method and del methods.

If yes would it also make sense to create a class methods for TypedDict like TypeDict.from_typed_mapping(DataMapping) and TypedMapping.from_typed_dict(Data)?
This would help to only define larger TypedDicts and use them in different contexts.

Also this could help to write the types for types.MappingProxyType. Because as I user I would expect that if I insert a TypedDict inside it, a TypedMapping is detected by the type checker.

I am aware that a Mapping[str, str] is different of dict[str, str] as Mapping[str, str] can contain SubclassOfStr("a") wheras dict[str, str] can’t. That might be worth to consider.

[tmk (Thomas Kehrenberg)]

Does PEP 705 not address this?

[Kound (Carli* Freudenberg)]

Thanks for the hint.
Yes it goes in the direction. But having a lot of attributes that could be cumbersome.

I am thinkinng more of

from typing import TypedDict, ReadOnly
import json
import types

class Data(TypedDict):
    foo: str
    bar: int
    # super many other arguments

def read_data() -> ReadOnly[Data]:
    with open("file.json", "r") as f:
        data = cast(Data, json.load(f))
        return types.MappingProxyType(data) 

Important here

• I have runtime safe read only behaviour (the PEP explicitly states it only changes the type behaviour)
• correct typing of types.MappingProxyType if handed in a TypedDict

[alicederyn (Alice)]

Unfortunately that opens up more issues, like, what about nested TypedDicts, should we recurse into them and mark them ReadOnly too? What about lists and other mutable collections?

It’s certainly possible to solve this problem but I think it will be through a more generic type transformation system rather than cherry-picking simple cases.

[dreamsorcerer (Sam Bull)]

I would expect such a type would conform with the runtime types, therefore mutable values would likely be allowed. For myself, I just want some way to return a MappingProxyType with typed information for each key.

i.e.

def foo() -> MyType:
    return MappingProxyType({"a": 5, "b": "foo"})

reveal_type(foo()["a"])  # -> int

Whether this is with a specific TypedMappingProxyType, a generic TypedMapping, or marking all the keys of a TypedDict ReadOnly, doesn’t matter too much to me. (Though I’m guessing the last option won’t work because of the inheritance behaviour in PEP 705).

[Kound (Carli* Freudenberg)]

I am with Sam here.

Using types.MappingProxyType on any dict does not care about nested mutables, so shouldn’t ReadOnly (or what ever).
The thing you suggesting sound more like a freeze which converts all mutables to immutables.
PEPs going in this direction have been rejected so far, so there is no runtime representation of this atm in the Standard Lib.
So why should I need a typing representation for this?

Basically I want be able to express types.MappingProxyType(typed_dict) properly for a TypeChecker.

I could also imaging just adding a .as_mapping_proxy_type property to TypedDict.

So this would be:

def read_data() -> Data.as_mapping_proxy_type:
    with open("file.json", "r") as f:
        data = cast(Data, json.load(f))
        return types.MappingProxyType(data) 

But reading PEP 705 again I already see this example:

blur["members"] = ["Damon Albarn"]  # Type check error: "members" is read-only
blur["members"].append("Damon Albarn")  # OK: list is mutable

So actually I would like to propose only ReadOnly[AnyTypedDict] that return a modified AnyTypedDict with all attributes being marked as ReadOnly.

Actually unsing ReadOnly on the complete TypedDict was discarded by PEP 705 saying:

This would naturally want to be defined for a wider set of types than just TypedDict subclasses, and also raises questions about whether and how it applies to nested types. We decided to keep the scope of this PEP narrower.

I guess the question for nested types is answered using above and I would argue that other will types already could want it to be defined for.

[hwelch (Hayden Welch)]

I’m curious if there’s been any further movement on this. I’m running into the exact same issue using MappingProxyType for freezing global defaults that are used to then generate/update mutable runtime dictionaries.

I have all this typed using TypedDict, but my peppering of # type: ignore and forcing the type feels dirty.

I know frozendict is in the pipeline, but I’m not sure if that will behave any diffently than MappingProxyType since it is also not a dict subclass (even though TypedDict is actually revealed as Mapping[K, V]).

I do think that your initial idea of just a TypedMapping that makes no assumption of mutability or immutability would be useful. Especially since there’s plenty of times that I’m handling the actual important stuff in some other way that isn’t easily followed by the type checker.

For now I’m just gonna keep lying and documenting my lies thoroughly.

[tmk (Thomas Kehrenberg)]

There was some discussion about how to type frozendict here:

PEP 814: Add frozendict built-in type

Yes, we’ll have to think about how this new builtin type (if it is accepted) interacts with TypedDict. (“We” here doesn’t necessarily mean the authors of PEP 814; this is something we can add separately later.) Currently, TypedDict is specified as being restricted to instances of exactly dict: no subclasses, no other kinds of Mappings. So if we don’t change anything, frozendicts would not be compatible with TypedDict at all. A few options: Add class TD(TypedDict, frozen=True): (as Thomas sug…

I’m somewhat optimistic that frozendict can be provided as a backport pypi package to older Python version, so this will hopefully also help you.

[hwelch (Hayden Welch)]

Thanks for the link! I had been following that thread and just missed that reply from Jelle. I’ll continue to watch this space, but it seems almost all of my concerns have been considered by the typing team already.

[Jelle (Jelle Zijlstra)]

If you want something to happen, you should actually propose something; things aren’t going to happen just by waiting.

[hwelch (Hayden Welch)]

I agree, I just wanted to check in on the old threads. I’ve still got to get deeper into the mechanics of TypedDict and I also want to mess around and use frozendict first.

I’m going to use this project as a starting point to try and figure out a good way of adapting TypedDict to work with frozen mappings.

Sorry if it seemed like I was begging for something, I was just trying to do due diligence before I tried to come up with a proposal.

Edit:

One thing I would like to ask, since you’re one of the core typeshed developers, would adapting TypedDict to accept ‘dict like’ builtins be something that would confound existing proposals like PEP 827. As well as the existing discussions about KeyOf and TypeOf ideas

Both of those are much larger in scope, and I’m not sure if something like allowing a MappingProxyType/frozendict to use TypedDict typing would get in the way of those.