Internally it does something similar to what is talked about here:
Edit: In theory, this recipe could be changed to allow critical sections to complete by having a status flag and waiting for it to be ‘killable’ to send the exception to raise, or something like that.
(Not that I personally have a use for this behavior. I don’t think it is needed in the standard library. Just pointing out that something on pypi exists that gets this part of the way there.)
On the contrary, who is responsible is crucial from a programming point of view, when talking about threads. In general, a thread is aware of its own state (and therefore: at what points it is safe to stop, and what cleanup is necessary before stopping at that point). Other threads cannot be aware of that, and the to-be-killed thread cannot plausibly “respond” to being killed in order to figure out where it is and what cleanup to do, nor can it “resist” and wait until the next point when it is safe to abort. That is exactly why cooperative methods are normally recommended instead: a thread that is told to stop can poll for that request whenever it is reasonable to acknowledge it, and do whatever is necessary before exiting.
If a thread raises its own exception, similarly, it can clean itself up immediately before. If it isn’t safe to bail out at that point but an uncaught exception is raised anyway, that’s clearly programmer error. Not so if the bailing-out is coerced from outside - there’s no way to anticipate that.
Aborting a process with ctrl-C etc. is similarly not comparable - because the entire point is that there will ordinarily be nothing running after that, so all concern about inconsistent state is generally moot and certainly not actionable (yes, a process could be killed in the middle of writing a file; the power could also go out).
Only because in general we design threaded code to be so, because this is currently the only clean way to interract and stop them since current threads are not safely interruptible.
and the to-be-killed thread cannot plausibly “respond” to being killed in order to figure out where it is and what cleanup to do, nor can it “resist” and wait until the next point when it is safe to abort
Again, they currently can’t, but that’s the point the proposed eit() and nit() functions are intended to solve: with such functions threads can control when they are accepting interruptions and when interruptions are kept for later on.
cooperative methods are normally recommended instead: a thread that is told to stop can poll for that request whenever it is reasonable to acknowledge it
Yes, that’s what I described in my initial post. The big downside is that, this cooperation needs the thread to be a loop or a coroutine or to be unreadable with tons of poll checks eveywhere, plus it prevents you for stopping a call to a third party function.
Aborting a process with ctrl-C etc. is similarly not comparable - because the entire point is that there will ordinarily be nothing running after that
If the purpose of ctrl-C is meant to only cause immediate program termination, and if exceptions are not meant to be caused by something else than the receiving thread itself, then why are KeyboardInterrupt and SystemExit exceptions even existing ?
If the only expected behavior is to end the program with no guaranty on the cleaning, what is the point in having ctrl-C triggering a KeyboardInterrupt rather than simply immediately ending the interpreter ?
I have the feeling that we are in the end confronting 3 paradigms here:
first paradigm states “external events must not alter the controlflow of a thread”
Used in C and Rust and many other languages. These languages do have signal handler callbacks, but do not trigger exceptions on signal reception. The default behavior on ctrl-C is to exit the program whatever its state is. The default behavior is to rely on the OS to cleanup the mess.
second paradigm states “external events can alter the controlflow of a thread through exceptions”
Used (AFAIK) only by python for reacting to ctrl-C or reacting to exit(), in asyncio with for task cancelation and in specific cases using PyThreadState_SetAsyncExc (and for all other cases, first paradigm is generally applied)
Since some portion of code (like cleaning) are sensitive, it is good in this paradigm to have means to temporary set the thread insensitive to external events.
third paradigm states “external events can stop controlflow of a thread, then cleaning is performed”
Used by python coroutines and rust futures: tasks can be stopped from outside (can be signal handlers or any other thread).
For sensitive portion of codes (like cleaning), we have the guarantee then cleaning code will be called. To write cleanup code needs to attach this code to the destructor of an object.
It is also often good to have means to temporarily set the task insensitive to external stopping (much like asyncio.shield)
What I am saying is that python threading model should choose one of these paradigms and become consistent with it, don’t you think ?
if first is choosen, then KeyboardInterrupt and SystemExit and PyThreadState_SetAsyncExc shouldn’t exist
if second is choosen, then it could be good to have a standard implementation of exception sending- and interrupt disabling- functions
if third is choosen, then likewise it needs a thread stopping- and a stop disabling- functions
Nice finding !
It seems this is using the same ctypes.pythonapi.PyThreadState_SetAsyncExc trick as I do in my project
Unfortunately it is currently only good for stopping threads because of the catching issues I mentionned above.
True, but external events can always trigger exceptions, so the key here is how you handle the exception. You already have to assume that an exception could be raised at any time (eg MemoryError), and design accordingly.