Question: How to json.dumps(d), skipping any field which value is of type A? I would like to avoid creating a temporary version of the dictionary d in which those fields are missing.
Suppose I have
class A:
__slots__ = ('value',)
def __init__(self, value):
self.value = value
def __repr__(self):
return f'A(value={self.value})'
def __str__(self):
return self.value
d = {
'id': 'aaaa',
'count': 222,
'child': {
'id': 'bbb',
'child': A('foreign object')
}
// Large dictionary with moderately deep nesting
}
One possible way that I see is:
Inherit from JSONEncoder, redefine its encode(...) method to not pass _one_shot=True to its iterencode(...) call. This, such that it doesn’t call its C implementation.
Redefine its _make_iterencode(...), specifically to modify its inner function _iterencode_dict(...), which is where I need to check the type of the value before the key and the separator ": " are yielded.
Yes, this is the solution using a temporary, rc in this case. Imagine the variable that you called input holds a really large dictionary that already occupies a sizable chunk of memory. If only a few fields are of type skipped_value_types, then rc will also be large. The data might not get duplicated, but still, it is a large dictionary. Also we would be doing two passes, one to exclude, and one to serialize. What I want is that the act of serializing skips the required fields. That way, no temporary, and a single pass over the data.
The would require implementing your own serializer as you described. You can bring this over to the ideas section as a proposal but you have the general idea for a solution now.
Can’t you build a small class that wraps your dict and implements items() such that it filters out the unwanted types? In the end, the core of the JSON encoder is a loop like this:
for key, value in obj.items():
# Produce serialised representation
Replying to myself here. It is actually not that simple. What items() returns is a dict_items object. This itself implements the iterator protocol so you can iterate over the keys and values in a loop. I experimented a little with the __iter__ and __next__ methods of the dict_items object to see if I can replace them with a custom function that drops the unwanted types, but no luck so far. I will keep trying and see if I can come up with something that works.
It is fine. The wrapper class’ items() can just yield the pairs.
class A:
pass
class D(dict):
def items(self):
for k in self.keys():
v = self[k]
if not isinstance(v, A): # The values that we want to exclude
yield k, v
d = D({1: 111, 2: A(), 3: 333})
import json
json.dumps(d) # returns '{"1": 111, "3": 333}'