PEP 818: Adding the Core of the Pyodide Foreign Function Interface to Python

PEPs
1

I wrote a PEP to upstream the core of the Pyodide FFI. This is necessary for using most JavaScript APIs on the Emscripten platform. Asyncio support, buffer protocol support, and a more complete JS API are all deferred.

9 Likes
PEP 783: Emscripten Packaging
PEP 783: Emscripten Packaging
PEP 783: Emscripten Packaging
2

Could I suggest we not use the word “Upstreaming” in the title as that is a relative term and means something different based on where the reader sees themselves. Use a more concrete title.

I use the term “upstream” all the time to mean integrating something into an “upstream” OSS project. But a PEP is about enhancing Python so I read that “Upstreaming” in the title to mean taking something in CPython and pushing it into another project.

4 Likes
3

Renaming it to “Adding the Pyodide FFI to Python” would be fine. I think anyone who is aware that Pyodide is a downstream project would not be confused though, and otherwise the first sentence of the abstract is enough to clarify. I do end the abstract with:

This PEP proposes adding the core of the Pyodide foreign function interface to Python.

so maybe the title should be

Adding the Core of the Pyodide Foreign Function Interface to Python

4

It sure confused me! Thanks for being flexible about the title.

5

So it’s not just Greg. I updated the title.

1 Like
6

number and bigint awkwardly correspond to float and int … We also make a new type JsBigInt to act as the conversion for bigint .

That seems like a contradiction.

There is some naming inconsistencies. For instance, why is jsnull lowercase but JsBigInt isn’t? As well, JsBigInt violates PEP 8 and should be JSBigInt. Same goes for other spellings of “Js”.

I would move up the mention of the js module and pyodide package. I kept seeing them in examples and I didn’t know where they came from.

I would prefer not to use the pyodide name and just move everything under js. The “pyodide” name doesn’t explain what the package is for unless you happen to know what Pyodide is. Otherwise it feels like branding when things other than Pyodide should be able to use what the proposed package provides.

3 Likes
7

Thanks for the feedback @brettcannon

For instance, why is jsnull lowercase but JsBigInt isn’t?

jsnull is a value of type JsNull. JsBigInt is a type. It would be nice to make jsnull like None so that it could be used as a type hint, but that requires changes to typing tools.

I would move up the mention of the js module and pyodide package.

Okay, will do that.

I would prefer not to use the pyodide name and just move everything under js

The js name is the JavaScript global scope, which has a set of values that depend on the JavaScript runtime. Node, Chrome, and Firefox all have different global scopes and there are also differences between running in a webworker vs the main thread. So I think we need two separate modules, one for utilities that are Python functions and one that just is the JS global scope. Of course you could provide the global scope as pyodide.js or something to avoid having a second top level module.

The main reason for keeping the names is compatibility. Existing packages on PyPI like urllib3 use the current names. It wouldn’t be the end of the world to fix them, but it means that existing versions of these packages will only work with Pyodide Pyodide not with CPython.

My preference would be that if we rename things, we adopt both names with a deprecation period for the original names.

As well, JsBigInt violates PEP 8 and should be JSBigInt. Same goes for other spellings of “Js”.

Same, this will be backwards incompatible so I would prefer to add both with a deprecation period.

8

I pushed a commit changing this, now it reads:

Of these, string and boolean directly correspond to str and
bool. We convert a number to an int if Number.isSafeInteger()
returns true and otherwise we convert it to a float. Conversely we
convert float to number and we convert int to number unless it
exceeds 2**53 in which case we convert it to a bigint. We make a new
subclass of int called JsBigInt to act as the conversion for
bigint.undefined is the default value for a missing argument so it
corresponds to None. We invent a new falsey singleton Python value
jsnull of type JsNull to act as the conversion of null. All other types are proxied.

Is that better?

1 Like
9

Yes it is. Thanks!

10

I also added a section at the end of Rationale describing the top level packages js and pyodide.

11

I agree with this. So the questions here are:

  1. If we came up with the best possible names without any backwards compatibility concerns what would they be?
  2. Are they enough better to justify the churn of changing them?
  3. Is it important enough to preserve backwards compatibility to provide deprecated aliases?

In my opinion the answers to 2 and 3 are both yes. I am personally terrible at naming things so I would appreciate help from anyone else in proposing new names.

12

If it’s Emscripten-specific then call it emscripten? Otherwise, could be jstypes or jsffi (by analogy with ctypes and cffi).

2 Likes
13

pyodide.ffi is already part of the pyodide namespace but I wonder if that namespace could be renamed as js and the js itself could be js.globalThis as that would reflect, without needing much explanation, what it refers to. I would also say that emscripten is “one tool” that is able to bring WASM + FFIs but not necessarily the only one (today or in the future) yet I like the jstypes and jsffi idea and agree pyodide itself as a name might be confusing for anyone that wouldn’t know the project already, although I am just suggesting here, no strong opinions around namig (I am terrible too).

edit → globalThis - JavaScript | MDN

14

Emscripten also has the disadvantage that people will not necessarily know or care what it is. jsffi seems fine with me, though I don’t love having from jsffi.ffi import JsProxy…

15

I guess from jstypes.ffi import JsProxy would be fine.

16

Or you provide a shim on PyPI that people can install instead of having the stdlib carry the deprecation of the old names if they aren’t using Pyodide.

17

Yeah that’ll work.

18

The big question I have after reading this is: why does all of this need to be in CPython?

Pyodide’s API design makes a lot of non-straightforward decisions, and “blessing” them will make it very hard to try different approaches.

It’s good to have “ergonomic” API when using it directly. But for building libraries on top, or generating code, it’s better to have API that avoids special cases as much as possible. I think the stdlib should prefer the latter, but Pyodide tends to the former (as it probably should!).

Would it be better to add smaller, more explicit API that Pyodide and other projects can build on?

1 Like
19

The reason is that this code has to be written in C and statically linked. So the question is whether we want CPython to distribute the Python to be used in JS runtimes or do we want Pyodide to continue distributing it? If we want Pyodide to continue distributing it, then I’m not sure there’s any good reason for this pep. On the other hand, if we want the CPython build to work, it needs to implement most of this logic. A small amount of it is implemented in Python and could be moved to pypi, but most of the surface area has to be in statically linked C unfortunately.

On the plus side, Pyodide has been shipping most of this stuff for years and I think we have a lot of evidence that it’s a good enough design. By comparison I’d say that ctypes is also pretty opinionated.

1 Like