I discussed this topic a year ago and feel like it might be a good idea to revisit it now that I’ve properly built a reference implementation. I suggested adding some sort of API for calling asynchronous Python functions from C. It seemed to be a somewhat impossible task, but I did manage to come up with an API that works relatively nicely.
You can see my implementation here
It works like this:
- Instantiate a “transport” class (this is called
Awaitablein my implementation) viaPyAwaitable_New - Use
PyAwaitable_AddAwaitto submit coroutines to be handled byAwaitable, along with callback functions to be used for the results - Save values needed in callbacks via
PyAwaitable_SaveValues - In either the main function or one of the callbacks, call
PyAwaitable_SetResultto set a “return value” for the awaitable (i.e. what is returned after doing anawait)
To show it visually, here’s an example:
static int // -1 is an error, 0 is OK
awaitable_cb(PyObject *awaitable, PyObject *result)
{
PyObject *some_object;
if (PyAwaitable_UnpackValues(awaitable, &some_object) < 0)
return -1;
// result is a borrowed reference to the return value of the coroutine
return PyAwaitable_SetResult(awaitable, result);
// PyAwaitable_SetResult returns 0 or -1, so we can just return the value of that here
}
static PyObject *
test_awaitable(PyObject *self, PyObject *args)
{
PyObject *some_object;
PyObject *coro;
if (!PyArg_ParseTuple(args, "OO", &some_object, &coro))
return NULL;
// precondition: coro is a coroutine instance
PyObject *awaitable = PyAwaitable_New();
if (!awaitable)
return NULL;
// this saves a reference to some_object inside of the awaitable
if (PyAwaitable_SaveValues(awaitable, 1, some_object) < 0) {
Py_DECREF(awaitable);
return NULL;
}
// the fourth argument here is a callback for if an error occurs when awaiting the coroutine
if (PyAwaitable_AddAwait(awaitable, coro, awaitable_cb, NULL) < 0) {
Py_DECREF(awaitable);
return NULL;
}
return awaitable;
}
Now, when you call test_awaitable from Python, you can use it like any other async function:
async def foo():
await asyncio.sleep(1)
return 1
print(await test_awaitable(foo()) # prints 1
I do think this does need further development, but it is an async C API that works. This does introduce a problem of callback hell, so that is something to think about. Perhaps a PEP could be written for clearer specifications or more opinions?