PEP 788: Reimagining native threads

It’s not. This is a frequent source of deadlocks with threading threads. This PEP proposes introducing that same source of deadlocks for native threads.

The current status quo is that extension module developers who know what they’re doing are able to use atexit handlers to ensure their native threads are joined, but naive extension module developers who never stop their threads produce things that might occasionally crash or deadlock during finalization, depending on timing.

The non-daemon native threads proposed by this PEP make it so that extension module developers who know what they’re doing can no longer use atexit handlers to join their native threads, and naive extension modules developers who never stop their threads produce guaranteed deadlocks.

Assuming the thread doesn’t run infinitely, how could there be deadlocks? Python won’t wait once all PyThreadState_Ensure calls have been PyThreadState_Release’d. An atexit handler can still join the thread if it needs to do additional work, it just won’t be able to talk to Python at that point.

@ZeroIntensity and I caught up offline. I was missing something here; the analogy to threading threads threw me off: Py_Finalize will wait until non-daemon threading threads exit, but will only wait until non-daemon native threads release the thread state, not until they’ve actually exited.

Which means I’m wrong, and you can still use an atexit handler to gracefully stop native non-daemon threads.

1 Like

Would someone mind locking this? The new thread is up at PEP 788: Reimagining Native Threads (new updates)

1 Like