Hello from the Paris JuypterLite workshop.
In the past three days many people who are involved at various levels in the
Python/WASM ecosystem were in the same room: we had representatives from
Pyodide, emscripten-forge, JupyterLite, PyScript, Basthon and other projects
(sorry if I forgot any).
Among the many interesting conversations that we had, we discussed one point
which I think it’s good to share with the broader Python community:
what can we do to make WASM a 1st class platform in the Python ecosystem?
Before starting, it is probably useful to draw a parallel with existing
architectures:
-
WASM is basically the CPU, i.e. equivalent to x86 or ARM
-
on top of it, we have the “operating system” and/or platform: for WASM we
have emscripten and WASI, so they are equivalent to e.g. Linux, Windows or
MacOs.
By 1st class platform I mean that it should be easy to build, test and
distribute libraries and apps on Python/WASM.
Having WASM as an officially supported Tier-3 platform is a huge first step,
but in order to have a better developer experience we probably need better
tooling around it.
To better explain what I mean, let me write down some of the things which I
personally think they should happen:
-
it should be easy for library authors to test their code on WASM: ideally,
they should be able to do something likepython-wasm -m pip install ...
,
python-wasm -m pytest
, etc. The good news here is that both Pyodide and
emscripten-forge have written tools which go in this direction. Note that for many usages, it is not even required to run them in
the browser: it is totally possible to run and test the code inside node,
from the command line. -
It should be easy to build and deploy packages. Ideally, it should be
possible to upload WASM-specific binaries to PyPI and conda-forge. -
It should be easy to do the steps above in CI.
During the workshop we successuly manage to import packages built by
emscripten-forge into Pyodide, so it seems that we are at a point in which
this scenario is technically possible.
I think that these points are extremenly important for the long term
sustainability of the effort: currently, pyodide and emscripten-forge maintain
a big set of patches to be able to build a selected set of packages to WASM,
but if we make it possible to easily test it on CI, I can imagine that as a
community we can start to send these patches upstream.
Obviously it doesn’t mean that individual projects will be forced to accept
these patches, but at least it gives a possible path forward. Basically, we
would end up with a situation similar what we currently have with Windows:
many libraries are developed on Linux or Mac, but they are willing to accept
contributions to make them working on Windows, so WASM would just be another
instance of this pattern.
So, the first concrete question is: what does need to happen to be able to
upload packages in PyPI?
The second question is broader: I would love to hear general thoughts and
opinions on this, and collect suggestions for additional things which are
needed to make WASM a 1st class platoform.