Interrupting long running PyObject_Call inside subinterpreter

Python Help
, ,
1

I’m working on Python foreign function interface for Elisp.
Python methods are called in separated pthread with PyObject_Call.
I’m creating sub-interpreters to isolate Elisp packages from each other.
I want user to be able to interrupt long running python methods in same manner as we can press control-C in interactive interpreter.

My problem is PyErr_SetInterrupt seem to not work and i have no idea how to do it.
Naive attempt with signals and PyErr_SetInterrupt is in commit naive attempt at C-g implementation · 813gan/emacspy@2eecf89 · GitHub

2

Subinterpreters don’t support CTRL+C yet, at least when they’re on the main thread: Ctrl-C Does Not Do Anything While A Subinterpreter Is Running in the Main Thread · Issue #113130 · python/cpython · GitHub
(If you try to run it in a seperate thread, you’ll get a crash: Multithreaded subinterpreters can be running during finalization · Issue #126016 · python/cpython · GitHub)

Speculatively, you could install a signal handler and then use PyThreadState_SetAsyncExc to send a KeyboardInterrupt to the interpreter, but I suspect you’ll run into some interesting edge cases.

3

But GIL is needed for both PyThreadState_SetAsyncExc and PyObject_Call, no?
I’m a bit confused because documentation for PyObject_Call doesnt say anything about GIL but i got crash when i released GIL before call to it.

4

Also. thanks for your response especially for links to existing issues.
Lack of reaction under Ctrl-C Does Not Do Anything While A Subinterpreter Is Running in the Main Thread · Issue #113130 · python/cpython · GitHub makes me feel like using sub-interpreters was not good idea in first place.

5

A long time ago there used to be a callback in the ceval code that you could hook into. I was using it for a while. No idea if that hook is still in the code.
Suggest you read the code of the evaluation loop and see if it is still there.

I used the hook to simulate a KeyboardInterrupt to prevent the python code from locking up the app.

6

Hi, thanks for response. I don’t want to go beyond stable ABI as is i need to support wide range of python versions and platforms.

7

The hook was part of the API at the time I was using it I think.