PEP 818: Adding the Core of the Pyodide Foreign Function Interface to Python (2)

PEPs
1

This is the second discuss thread for PEP 818. The previous thread is here. I got the following feedback:

  1. Various name changes were requested, which I implemented.
  2. The PEP was too long, so I reduced the length by about half by removing implementation details from the spec section.

I also added support for proxying and conversion of buffers, and for asyncio including an event loop, conversion between Python and JavaScript awaitables, and deferred destruction for arguments to an asynchronous JavaScript function called from Python.

In terms of motivation, I got confirmation that the maintainers of Emscripten-forge (an alternate Emscripten Python runtime) strongly support the proposal. They don’t have an easy way to use Pyodide’s foreign function interface currently so they reimplement it. But they would much prefer to share code, and this proposal is the most direct way to do that.

3 Likes
PEP 818: Adding the Core of the Pyodide Foreign Function Interface to Python
2

Regarding the asyncio integration, I’m wondering if we could make it more generic to allow integration with other Python event loops (for instance, Trio). It seems the only asyncio-specific code is ensure_future, add_done_callback and Future.

Also IIUC promise cancellation doesn’t really exist in JavaScript (even with abort signals), so I’m wondering what implications this has on the Python side.

1 Like
3

if we could make it more generic to allow integration with other Python event loops

You mean something similar to set_asyncgen_hooks? I think it makes sense, I will add it.

Also IIUC promise cancellation doesn’t really exist in JavaScript (even with abort signals), so I’m wondering what implications this has on the Python side.

Well abort signals allow cancellation of certain promises, in the sense that when used the promise will be rejected and the IO work that the promise is waiting for also stops. But we don’t have any integration with Python cancellation and I don’t really know how to do such integration. For the most part the implication is that when a Python task is cancelled, the current IO that it’s waiting on keeps going. When it finishes, no further work will happen on the cancelled Python task. But the current IO job is not affected.

It would be possible to introduce some API like CancellablePromise(promise, on_cancel). Then you could do

controller = AbortController.new()
resp = fetch(some_url, signal=controller.signal)
wrapped = CancellablePromise(resp, controller.abort)

Annoyingly this essentially has to be done from user code. But I think it can be implemented downstream.

4

For integrating any Python event loop, I just meant that the Python code of the WebLoop should not depend directly on asyncio, but should use an API that could be implemented using asyncio or e.g. Trio. But given that AnyIO plays this role at the Python level, maybe this is not so important.

The way I see cancellation in JavaScript using abort signals is that the code has to be cooperative all the way down, and in general it is not. So a lot of IO work can be done even though it was cancelled on the Python side. There is an interesting blog post about that here.

1 Like