PEP 720: Cross-compiling Python packages

Packaging
1

Hi, I pushed a draft PEP attempting to document the current state of cross-compilation and relevant information. The goal is to both serve as a reference document as we try to improve the status quo, as well as a supporting document for build backend (or related applications) authors to help possibly migrate away from direct compiler introspection (see New sysconfig API 🤞 (meta-issue) · Issue #103480 · python/cpython · GitHub for the future plans on that front).

I would like to gather feedback on the PEP. This is a very complex topic, so it’s probably there’s something I missed.

Additionally, if you are a contributor to a project that has to do with cross-compilation (eg. build backends) and would like to contribute an entry to the case-studies section, please let me know [1], or open a PR against the GitHub - python/peps: Python Enhancement Proposals repo.

  1. You can reach out to me via email at lains@riseup.net, or join the PyPA discord to discuss. ↩︎

6 Likes
2

A few things that I am still confused about:

  • What are all the variables supposed to expand to under this PEP? I assume that most of them would be pointing to the target’s values? How do we interrogate the system’s environment if needed?
  • This PEP is primarily aimed at introspecting the cross-compilation environment within a python environment, such as for PEP517 python native builders?
  • What about python3-config? I presume that that should point to the target’s environment, but should stuff like python3-config --prefix point to the staging locations or the installed paths?

As far as I’ve tried the crossenv approach, it is actually working quite nicely. I presume that with this PEP, it would be compatible with that approach?

3

This is an Informational PEP, it attempts to document the current status-quo. No changes are proposed.

4

Ok, but in that case I would expect some clarification of the current tools and what do the variables expand to for the system or target.

5

The PEP is in Draft status. Feel free to submit a PR with the information you’d like to add and I’ll review it as soon as I can.

6

One factor that’s worth extracting is whether the various tools/case studies try to assume that they’re running in the target environment, or if they expect/require additional information.

We’re in the midst of helping a lot of Python packages add support for Windows ARM64, and it looks like most are set up in ways that are completely incapable of parameterising the target platform. But some more efficient ones allow overriding it, which makes it easier to build for other targets (for example, my backend can build all Windows architectures and versions without having to launch any of them, and potentially for other platforms too but I haven’t tried it).

Any future work is going to have to decide between faking the environment around existing scripts vs. providing the controls/settings to multitarget without pretending that it’s “native”. A survey of the existing landscape that doesn’t cover past choices isn’t going to be helpful.

2 Likes