PEP 788: Reimagining native threads

PEPs
2

Hi,

Some questions:

For backwards compatibility, all thread states created by existing APIs will remain daemon by default.

  1. Which “existing APIs”? C ones? Python ones?
  2. Why are we concerned about backwards compatibility? Do we really view hanging the thread as a feature?

int PyThreadState_SetDaemon(int is_daemon)
Set the attached thread state as non-daemon or daemon.

Shouldn’t there be a PyThreadState_GetDaemon counterpart?

PyInterpreterState *PyInterpreterState_Hold(void)
The caller must have an attached thread state, and cannot return NULL.

What cannot return NULL here? The caller? PyInterpreterState_Hold?

PyInterpreterState *PyInterpreterState_Lookup(int64_t interp_id)
Similar to PyInterpreterState_Hold(), but looks up an interpreter based on an ID (see PyInterpreterState_GetID()). This has the benefit of allowing the interpreter to finalize in cases where the thread might not start, such as inside of a signal handler.

Is this API function really signal-safe? That sounds like a very constraining requirement for a function that will probably have to access a mutable global structure.

int PyThreadState_Ensure(PyInterpreterState *interp)
The interpreter’s interp reference count is decremented by one.

Are you sure that’s safe to do? I would expect PyThreadState_Release to decref the interpreter, not PyThreadState_Ensure. Otherwise, what happens if there is a Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS pair inside the
PyThreadState_Ensure / PyThreadState_Release pair? Could Py_END_ALLOW_THREADS fail reacquiring the interpreter?

void PyThreadState_Release()
Detach and destroy the attached thread state set by PyThreadState_Ensure().

It doesn’t always destroy the thread state, does it? It should only do so if the thread state was created by the matching PyThreadState_Ensure() call.

I would also expect the PEP to answer a couple more questions:

  1. Are nested pairs of PyThreadState_Ensure() and PyThreadState_Release() calls supported?
  2. What is the use case for PyInterpreterState_Lookup()? Is it when you don’t want to keep a strong reference to an interpreter? Is the interpreter id guaranteed to be unique for the entire process call (i.e. it cannot be recycled after the interpreter was destroyed)? The PyInterpreterState_GetID doc doesn’t say so.
  3. How does this change the shutdown sequence? Does Py_FinalizeEx wait for all subinterpreters to be released (this could certainly introduce new deadlocks)? Or does Py_FinalizeEx only finalize the main interpreter, letting subinterpreters die when their refcount drops to zero?
1 Like