Make WASM a 1st class platform in the Python ecosystem

[brettcannon]

@antocuni

Another big difference is that emscripten provides support for dynamic linking (i.e., dlopen(), but not only) which is needed to actually import extension modules in Python. AFAIK WASI doesn’t support that yet.

Correct, WASI does not have a dlopen() equivalent.

@antocuni

As I said above, I think I would be happy enough with emscripten-only wheels for now.

And I’m the exact opposite. We’re using WASI for Experimental - Python for the Web - Visual Studio Marketplace , so that’s where my motivation/interests lie.

@antocuni

Also because AFAIK WASI doesn’t support dynamic linking, so it’s not even technically possible to publish wheels for it (is it?)

Depends on what you want in the wheels. If we allowed for .o files to be shipped instead of .so files then you could do a static link to a Python WASI binary. Toss in freezing Python code and you end up with a self-contained WASI app for Python code.

There’s also the other direction of using WASI to load code into Python itself, e.g. Support WASM wheels on PyPI - #4 by konstin .

@antocuni

More concretely, I think that a good practical way is to declare that a given CPython version supports a precise emscripten version, and when you are building a wheel you need to ensure that you are using the right one.

Are you saying you would want e.g. CPython 3.11 to always be built with emscripten 3.1.26 (and I don’t know how strict the versioning would have to be as I don’t know what API guarantees emscripten makes)? Or that simply a CPython build would need to be able to declare what version it’s built with as the platform and that there could be various CPython builds out there and thus various wheels for different emsripten versions? And if it’s the former, who decides which version gets used and does the builds? And how long does emscripten maintain older versions, so could CPython 3.11 rely on 3.1.26 or 3.1 working in five years time?

You’re also going to have to deal with emscripten’s various back-ends, like storage. That will change whether the build is for Node or the browser, so it will potentially have to extend beyond just emscripten version depending on what the target outcome is.

This is going to require some PEP to define all of this much like manylinux.

@antocuni

What is missing for that? Looking at the PEP it seems that the only extra requirement w.r.t tier 3 is to have two core developers taking care of it, and WASM already has both.

Christian hasn’t been available as of late, so it’s just me right now (and I don’t know if that’s a permanent thing). The SC also has to approve the switch to tier 2, so it also depends on what the SC wants to see happen.

It also doesn’t help that the emscripten builds have gotten a bit of a reputation of being hard to debug when the buildbot picks up a failure (typicaly lack of stack space). So there might be asks to improve that situation when it comes to other developers trying to debug why their change broke the buildbots to prevent their PR from being reverted in 24 hours. For instance, the emscripten builds were broken for a few days last week due to stack issues; luckily they fixed themselves but people had a hard time getting a build going to try and debug why things failed.