Dear friends,
I have two questions to consult with experts:
First, LoongArch is a new instruction set and we hope that PyPI will support LoongArch’s Wheel. What do we need to prepare for this?
Second, ‘And second, every time we move manylinux forward to a newer range of supported platforms, or add support for a new architecture, we have to go through a fairly elaborate process: writing a new PEP, updating the PyPI and pip codebases to recognize the new tag, waiting for the new pip to percolate to users, etc. None of this happens on Windows/macOS; it’s only a tax on Linux maintainers.’ in PEP 600.How much does it cost?
Someone would need to champion this change. You describe it as only a bureaucratic process of approving standards, but supporting a new architecture across the python ecosystem is much more than that. The community of c-extension maintainers (NumPy, SciPy, pytorch, arrow, cryptography, pybind11 are just the first few of thousands that come to mind) must now decide whether to release wheels for that platform. If so how can they test them, since there is no CI that supports LoongArch and emulation is sloow. Conda must also decide whether to roll out support for the new architecture. This added burden is not something to be taken lightly, so it is only fair that there are some hoops to go through to get the support recognized.
You are correct that the macOS support for arm64 did not go through a PEP process. But there was a long discussion here and subsequent PRs to pypa/packaging. The adoption process for c-extension library maintainers was, and is still, quite an effort.
I could not find a similar discussion for the win-arm64 tag here. I think since the pypa/packaging/tags.py code uses sysconfig.get_platform() for windows, which needed no adaptation for arm64. However it too suffers from a lack of adoption, even the cgholke repo for such wheels is still “experimental”.
@mattip thanks for your reply and suggestions, perhaps my description was not very good, causing a misunderstanding. I have also conducted some research and testing work on uploading wheel, including reviewing a lot of relevant PEPs. I will take the first step forward based on your suggestions .
conda[-forge] has a somewhat more structured way of supporting a new architecture, by slowly migrating through all its packages in the order of the dependency graph (fixing problems problems as they arise and ideally upstreaming them).
For example, a roll-out for win-arm64 will happen relatively soon in conda-forge. Similarly, preparation work for support for linux-s390x & linux-riscv has been undertaken (but not yet progressed to ecosystem-wide roll-out).
Assuming you can convince conda-forge/core & Anaconda that supporting loongarch is a worthwhile endeavour (and contribute to fixing problems as they arise!), you’d have a steady but manageable stream of work to ensure that each package builds correctly (first in conda-forge, then contribute the changes back to the respective packages).
To do so, it would be indispensable to have good cross-compilation support from linux-64 to linux-loongaarch, and ideally also QEMU support in order to run tests without having native hardware in CI (or also emulate the build step where necessary).
.
in conda-forge. Similarly, preparation work for support for linux-s390x & linux-riscv has been undertaken (but not yet progressed to ecosystem-wide roll-out).