A question for @wjakob, @pitrou, @davidhewitt and others in similar situations: what will you do if PyGILState_EnsureOrSafelyFail()
returns an error code indicating failure? How are you going to handle that?
For PyTorch, I think the only safe thing we can do is to hang the thread, like PyGILState_Ensure()
now does in 3.14. That case is going to happen sometimes even with a PyGILState_EnsureOrSafelyFail()
: if you have non-trivial code between PyGILState_EnsureOrSafelyFail()
and PyGILState_Release()
, there’s a chance you may incidentally call code that internally releases and attempts to re-acquire the GIL, which will hang the thread. For example, any Python code that calls into the interpreter may trigger a GC, and collecting things like file
objects whose finalizers temporarily release the GIL.
Furthermore, we can’t reliably pass an error up because we’re sometimes in situations where C++ calls into Python, which calls back into C++, which calls into Python:
C++ -> Python -> C++ -> Python
[^^ PyGILState_EnsureOrSafelyFail() fails here]
I don’t think there’s any way to safely pass an error or throw a C++ exception through the earlier Python context while the Python interpreter is shutting down.