After spawning a subprocess, there are several options for becoming aware of its termination, none ideal. One that I see mentioned in a lot of places is “spawn a thread, wait for the process, then do what you need to do”, which is extra overhead for what ought to be directly available. Another is regular polling - also a lot of overhead. The best way that I’m aware of is a signal handler:
vlc = subprocess.Popen(["vlc", ..., ..., ...], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
def ended(*a, **kw):
if vlc.poll() is not None:
... do what you need to do ...
signal.signal(signal.SIGCHLD, ended)
To someone who’s deeply familiar with how Unix signals work, this makes good sense, but I wouldn’t call it obvious, and if you have multiple subprocesses (especially a potentially growing and shrinking collection), this requires extra management. It’s also not mentioned anywhere in the subprocess docs, and (possibly because) it’s not cross-platform.
Proposal: Add a termination callback to Popen. When the process exits, this function will be called. Ideally it should work on all supported platforms, but at least on Unix, it would broadly behave as above, with SIGCHLD.
Yes, I’m aware of that (actually, maybe that should be listed as a “prior art” of a sort?). Not every Python program needs an event loop, and I don’t think it should be necessary to run an event loop in a dedicated thread just to handle process termination callbacks. (Though that WOULD be better than spawning a thread for each one.)
Ideally, they should become less limited Other than a callback on termination, what else do you feel is limiting?
aka one needs to start threads to deal with multiple and waiting for them
the real detail here is that hte event loop correctly solves this and in addition is the exact domain for the callbacks
adding a magic hacked on fake async callback to a sync api is pretty much overstepping all reasonable bounds of domain and should be rejected with prejudice
in particular if thers already a solution that
id much rather see an async version of subprocess.run
that one can then perfectly easily chained toa direct task - no hack callback needed - just a async function awaiting the process to run and then act on
and much better api than a misguided callback on a sync api
the way to wait for a subprocess to exit and then do something has a clear flow - subprocess.run
when one wants a callback version there shouldnt be a code called at a distance in a sync api, there should be a actual async version one can put into a task
SIGCHLD is called in the main thread, which is a significant limitation.
IMHO, spawning a thread is a better compromise that works everywhere; the overhead is not that big assuming you have a limited (up to thousands) number of spawned processes.
Other than a callback on termination, what else do you feel is limiting?