I have been fully out of the python packaging & python-for-android world (to which I had been a somewhat active contributor) for a few months and had time to reflect. Given how affected the things I personally worked on are by pips new direction, I couldn’t help but think a bit about the build isolation.
I have since reflected on the advantages I see in build isolation:
- More reproducible builds
- Better isolation than just virtual envs, which always were a little shaky for native code especially
- Cleaner design
However, I couldn’t help but also think back on the issues I’ve seen:
- Speed issues especially for native code builds due to wheels being rebuilt so much: https://github.com/pypa/pip/issues/7294
- Complicating corner cases of optional manual build environment packages: https://github.com/pypa/pip/issues/7509 & https://github.com/pypa/pip/issues/6144
- Making life of users with conda and packages with no wheels complicated: https://github.com/pypa/pip/issues/6193#issuecomment-457189313
- No way to guarantee that the same version of the package is used at build-time and run-time for binary packages, and general lack of distinction between build tools (like setuptools) and build dependencies (e.g. Cython or native lib dependencies): Support for build-and-run-time dependencies
- Lack of cross-compilation support in pip, and the build isolation making tools that attempt to fix this way more difficult because it rebuilds wheels at will: https://github.com/pypa/pip/issues/6718
And from the responses of pip developers I’ve mostly gathered, quick workarounds are not desired. I understand this, these things could add up to a lot of nasty code and even muddier concepts if all worked around thoughtlessly.
However, I also can’t help but think that these issues are quite numerous and fundamental: no cross-compilation and no real ABI awareness for native packages in particular, as well as intentionally no plans to allow easy disabling of the build isolation without knowledge of how to manually process all of pyproject.toml: that seems like nothing that will be addressed in the near future. And I understand this as well, these things need thought and exceed what pip has been meant for at its core, and could derail the pip codebase if done hastily. And maintaining a secondary, no build isolation world for everyone duplicates a lot of work.
This leads me to one concluding thought, though. And as a disclaimer, I am personally affected by this through the python-for-android complications, so I am not neutral. But with the other user tickets above I don’t think I am alone with this thought: that python packaging might be better off if build isolation was treated as experimental and put off for now until these fundamental related areas are better addressed.
I can’t obviously convince anyone, I am just an outsider to pip. I used it, I have written code to deal with it, but I’m in absolutely no position to ask pip developers what to do, or to know what’s best. This is just my very own view which I wanted to share it in the hope you also find this worth thinking about. Thanks for all your work on pip!