PEP 805: Safe Parallel Python

[pitrou (Antoine Pitrou)]

@bwoodsend

From trying locally, it looks like the GIL is blocking the destructor from happening in another thread whilst ujson.dumps() is still going and thereby keeping itself from being released?

A pure Python default function would allow releasing the GIL.
This crashes reliably for me:

import threading
import time

import ujson

l = [object()] * 1_000_000

def default(obj):
    time.sleep(1e-3)
    return None

def mutate():
    time.sleep(1e-2)
    l[:] = []

threading.Thread(target=mutate).start()
ujson.dumps(l, default=default)

(note the sleep calls only increase the likelihood of a race condition happening; the code is inherently racy even without the sleep calls)

And, yes, this doesn’t involve a destructor, but any pure Python destructor running in the middle of C code would present the same issue. I just don’t want to find how to trigger a pure Python destructor call inside ujson.dumps.