Should `sys.is_finalizing()` report interpreter finalization instead of runtime finalization?

Core Development
1

I really wanted to avoid marking a background thread as a daemon thread, so I set it up as a regular thread that terminated if sys.is_finalizing() was true.

To my surprise, that didn’t work - interpreter shutdown still got stuck waiting for that thread to terminate, even though the interpreter is marked as finalizing before the shutdown process waits for the non-daemon threads to terminate.

Checking the code revealed why: sys.is_finalizing() only becomes true when the runtime is shutting down, which happens much later in the shutdown process than when the interpreter is marked as finalizing.

@eric.snow has a somewhat related open issue around the fact that the corresponding C API (Py_IsFinalizing) is also confusing: The Word "Finalizing" in C-API Function Names is Sometimes Misleading · Issue #110490 · python/cpython · GitHub

Do we actually have a compelling reason for making sys.is_finalizing() report the runtime flag instead of the per-interpreter flag? Reporting the per-interpreter flag seems way more useful, since that way polling background threads can use it as a signal that they should exit.

The documentation for Py_IsFinalizing also suggests that it should be reporting the state of the finalizing flag for the main interpreter rather than the runtime flag.

1 Like
2

But if you change it to return true before non-daemon threads are joined (rather than after), then you may break existing uses it of this API.

I think what is needed is more granular visibility into the various phases of interpreter shutdown. Including, perhaps, being able to add dedicated callbacks at each step (e.g. “I want this callable to be invoked before all non-daemon threads are joined”).

3 Likes
3

True, I think my suggestion last time this background thread termination problem came up was a new atexit.register_early interface (for lack of a better name).

However, if @eric.snow gets his public interpreters.Interpreter type, perhaps those could have is_finalizing and register_exit methods.