Hi, I’m having an issue with a library I wrote. Any ideas to work around this? Or is this an issue with Python?
>>> import jsonyx as json
>>> obj = {}
>>> for _ in range(100_000):
... obj = {"": obj}
...
>>> json.dump(obj)
Traceback (most recent call last):
File "<frozen abc>", line 119, in __instancecheck__
RecursionError: maximum recursion depth exceeded in comparison
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\wanne\AppData\Local\Programs\Python\Python312\Lib\site-packages\jsonyx\__init__.py", line 614, in dump
).dump(obj, fp)
^^^^^^^^^^^^^
File "C:\Users\wanne\AppData\Local\Programs\Python\Python312\Lib\site-packages\jsonyx\__init__.py", line 274, in dump
fp.write(self._encoder(obj))
^^^^^^^^^^^^^^^^^^
SystemError: <built-in method __length_hint__ of dict_keyiterator object at 0x0000023CAC1EF8D0> returned a result with an exception set
It looks like a straight-forward recursion error: you created an extremely deeply-nested object, and because your serialization code is interrogating the types recursively, it eventually hits the limit.
I can imagine two workarounds, one easy and one hard. The easy option is to raise the recursion limit–there will still be some limit depending on your platform, but you might be able to raise it high enough for any given use-case.
The other, harder option would be to re-write your serialization code so that it isn’t recursive–IIRC any recursive algorithm can be re-written this way, but it might make for uglier code.
Eh, it’s supposed to raise a recursion error, not a system error, this is a unit test that’s failing:
It works correctly for lists, but not for mappings and I don’t understand why.
It’s calling len() somewhere before checking if an error occured, but I can’t get the call stack.
(Also, you can’t raise the recursion limit on the C side)
If you want to get the best help, it would help if you explained what you’re doing, what you expect to happen, and how that differs from what actually happens in the original post. Remember, we’re nowhere near as familiar with your code as you are. What’s obvious to you won’t necessarily be obvious it us .
As far as the system error goes, it looks like somewhere in your code you’re missing a check for whether an exception was raised and returning from a function like normal instead of signaling to the caller that an exception was raised.
OK, so I’m recursively calling a function, which in this case checks PyObject_IsInstance(obj, (PyTypeObject *)s->Mapping) before repeating. The problem is that when I hit the recursion limit this function raises a system error instead of a recursion error by the looks of it. (I printed a “<” before it and it’s not printing the “>” after it.) The thing is, PyObject_IsInstance is not implemented by me, so I expect that this is a problem in Python.
You can try querying and displaying the raised exception. Details on how to do that can be found in the “Querying the error indicator” and “Printing and clearing” sections of the manual:
.