Well I link to the Pyodide abi documentation for the definition of the platform. I could put that information in the pep if people think it’s appropriate but would we then update the pep with each new abi? I thought making an external “living document” and linking it from the pep was the appropriate approach.
So if I have a private Emscripten build of Python and want to build my own wheels for it, is there any guarantee that there will be a wheel tag I can use for that Emscripten version?
I tag them with platform tag like emscripten_3_1_58_wasm32. But these shouldn’t be uploaded to pypi because they are underspecified in terms of abi. I had this in the pep before but removed it because @pf_moore felt it was inappropriate.
people do build stuff on their own for their own reasons and I think we should try to allow for that.
I think it’s helpful for packaging to have basic tools for this but for pypi upload they’d presumably need to define their own abi and get their own platform tag for it?
That’s normal if the PEP is marked as “active” and thus is a living document.
I guess what you’re saying is Pyodide is the platform and will always own the meaning of the tag, and thus hosting the details externally on the platform’s site is fine. And so if we are all good with making Pyodide a platform on the level of macOS and manylinux then having Pyodide keep its own docs on this is reasonable. It also means that any other Emscripten-based platform will either need their own tag of let Pyodide control what they do if they want to reuse the tag (and I’m not saying that’s a bad thing, just a thing).
In the long run, perhaps we will want to upstream control of the Emscripten runtime into CPython. At that point recording the details of the ABI in a PEP would make the most sense. Then Pyodide could turn into a set of packaging tools for the platform. I think in the meantime, keeping the platform definition for humans closer to the code that defines the platform is a good way to go.
any other Emscripten-based platform will either need their own tag of let Pyodide control what they do if they want to reuse the tag
I spoke with Hood about this at PyCon. I’d love to see this move forward. The externally hosted definition of the platform is linked from the PEP, and there’s a PR waiting in the packaging repo. cibuildwheel’s been able to build these for a long time now.
One thing I think helps is considering the future of the platform. Having a tag for an implementation is common; PyPy and GraalPy are some examples. However, the interesting thing here is that if CPython support progresses to Tier 2 (it’s not even back up to tier 3 yet, this is forward thinking), then CPython will want to have a defined Emscripten platform tag (likely identical to the pyodide one). I think that’s completely fine; either the source of the pyodide tag definition moves to CPython, or a new tag can be introduced - if they become synonyms, that’s similar to what happened with manylinux_2*.
I think this comparison between manylinux and emscripten is a good one, actually, though they are different. A “linux” wheel works on just the system it was generated on, much like an emscripten wheel. Tagging it with manylinux means it is ensured to work on a larger set of Linuxes, while tagging an emscripten wheel with pyodide means it was generated with a specific emscripten version and set of ABI flags to work on Pyodide or any build of Python with matching flags. From a distribution standpoint, this is nearly identical; you have a way to describe specific wheels you build locally and only work on a specific system, and a way to describe wheels that are suitable for distribution and work with an ecosystem.