PEP 788: Reimagining native threads

[ZeroIntensity (Peter Bierma)]

@mpage

• I’d like to understand a bit better why the current finalization behavior of “native threads” (hanging them after finalization) is problematic. I believe this is the approach taken by the JVM and they seem to have made it work.

The two main cases I quoted in the motivation were:

• There are cases (for example, with C++ destructors) where Python code should be ran in addition to some other work. If we hang the thread, nothing else can run, which isn’t very helpful. I’m not sure how users of JNI deal with this.
• If a daemon thread grabs a lock and then gets hung, the main thread can possibly deadlock during finalization.

@mpage

• I prefer it if we could make the “daemonicity” of threads immutable.

Could be worth it, but probably should be done in a future PEP. I think threading can change the “daemonicity” of a thread whenever it wants, and we’re probably going to end up unifying threading’s shutdown and native threads in the implementation.

@mpage

• This is probably an unpopular opinion, but I don’t understand why we have the distinction between daemon and non-daemon threads. Having only daemon threads would lead to a simpler runtime. I know that we probably can’t change this for pure Python code, but I’d prefer we don’t introduce the distinction for extension code too.

I’m not too sure how daemon threads make things simpler. Any lock acquired during finalization is cloudy with a chance of deadlocks right now.

That, and aren’t non-daemon threads typically more useful/common? There was recently a proposal to deprecate daemon threads. I can’t think of many cases where it’s desirable to let your thread disappear at the nearest Python call. Isn’t it more common to want your computation to finish?