Hello there,
We have an application that has support for multiple subinterpreters that execute logic isolated from each other. They only have access to an API to call into the application. Each subinterpreter is created using _tstate = Py_NewInterpreter, we then import a module and call a func inside it.
When it’s time to destroy the subinterpreters, we do something like this:
...
PyEval_AcquireThread(_tstate);
if (EndInterpreterSafe(_tstate))
PyEval_ReleaseLock();
else
PyEval_ReleaseThread(tstate);
...
EndInterpreterSafe returns true when Py_EndInterpreter runs, which only happens if all of the criteria on its implementation are met (given thread is the current one, there’s no frame and it is the last thread).
Despite using a deprecated method, this works fine with 3.7. We were attempting a migration to 3.9, however, the code above crashes on PyEval_ReleaseLock after ending the subinterpreter because the current thread will be invalid by that time. We need to run something that releases the lock since it is still held after Py_EndInterpreter returns.
It’s fair to say that I’m new to using the python c-api. Been reading the documentation and issues about the topic and noticed that it is an area of active development. However, I’m still failing to connect the dots or grasp what are the relevant changes from 3.7 → 3.9 and what’s the correct way to destroy a subinterpreter in this scenario.
Any help is welcome, thanks in advance.