PEP 837: Extensible JSON serialization

mikeshardmind · July 18, 2026, 4:17pm

I’m not a fan of it being a global state, but I somewhat think copyreg makes more sense for a location for this than most places, including on the class itself (different applications need to serialize types differently, we know this). (Constructing and passing around a serializer/deserializer with the appropriate options seems the best option, but that’s already possible and people don’t always do it when they should)

I wonder if the API in copyreg can be imrpoved so that consumers of the api can get a point in time snapshot that can’t change under them, without having to create than snapshot themselves.

brettcannon · July 20, 2026, 9:47pm

Sure, but the point from Serhiy is that while potential breakage will be taken into consideration, we also don’t have to feel bad if there is some due to people not following this rule.

It would be very hard to implement in a way that’s forwards-compatible. Python is a language of consenting adults, so people should realize what risks they are taking on when doing something.

storchaka · July 24, 2026, 5:03am

Thank you, I fixed this little inaccuracy in in the top post (it was not in the PEP).

that would otherwise head toward the isinstance checks, default= or TypeError.

jvdillon · October 20, 2026, 9:13pm

Hello! I maintain treekle, which converts arbitrary objects to plain trees and back for json.dumps.

We started with exactly that pure-Python pre-pass. On 43 real experiment configs (nested dataclasses), the Python walker was 6.3x slower to encode and 95x slower to decode than the
same logic in Rust (4.8 ms vs 0.77 ms; 16.2 ms vs 0.17 ms). GitHub - rekursiv-ai/treekle: Strongly-typed fast pickler producing plain python trees. · GitHub

A wrapper works, but each one walks the tree a second time in Python, and that walk dominates the cost. Hooks inside the C encoder avoid it. Msgspec follows suit but defined their own container to speed it up whereas we opted to support native python.

Question on “Neutral ground”: is the intent that third-party native encoders read copyreg.json_dispatch_table directly?

Benchmark:

github.com/rekursiv-ai/treekle

scripts/bench_configs.py

main

# house-ignore[cli-shape] -- Runs as `python` in an environment that has priml, msgspec and jsonpickle.
"""Time msgspec, treekle, pickle and jsonpickle on every priml ``exp000`` config.

priml, msgspec and jsonpickle are not treekle dependencies, so run this in an
environment that has all three:

  python scripts/bench_configs.py
  python scripts/bench_configs.py --priml my.vendored.priml --columns msgspec-struct pickle

Encode times each library on the config itself; msgspec's ``to_builtins`` takes
``enc_hook=repr``, its only way past the callables the configs hold. msgspec
cannot decode the configs at all: its unions may not mix a custom type with
anything but None. So decode times the closest analogue: each config rebuilt as
generated dataclasses holding only plain leaves (a callable, type or dtype
becomes its repr). ``msgspec-struct`` decodes it into a tagged ``msgspec.Struct``
twin, the reference every ratio is taken against; ``msgspec-dataclass`` into the
dataclasses themselves. A decode that raises or
does not equal its input prints as ❌, as does a total that includes one;
msgspec-dataclass raises wherever a union holds two dataclasses, which it cannot
tell apart.

This file has been truncated. show original

bwoodsend · October 21, 2026, 12:34am

To be clear, when I said wrapper function, I just meant something like:

def json_dumps(*args, **kwargs):
    return json.dumps(*args, **kwargs, dispatch_table={decimal.Decimal: str}, default=json_default)

Not walk the entire structure (although I suppose that’s also an option – and one that works with any JSON library).

jvdillon · October 23, 2026, 9:43pm

I see your point; probably my point was just noise then. I did want to raise however that the walking is the hard/slow part.

But regardless, I definitely agree we should have an agnostic __serialize__ and this was what I optimized for in treekle. In fact, this is why its “tree”+“pickle” not “json pickle”–I felt going to plain python tree is better because then last mile serialization can be handled elsewhere.

There is a caveat though. Converting object to tree and tree to JSON can be made 2x faster if you go directly to JSON so there is a tension between agnosticism and performance.