Thought piece: on build isolation

I have been fully out of the python packaging & python-for-android world (to which I had been a somewhat active contributor) for a few months and had time to reflect. Given how affected the things I personally worked on are by pips new direction, I couldn’t help but think a bit about the build isolation.

I have since reflected on the advantages I see in build isolation:

  • More reproducible builds
  • Better isolation than just virtual envs, which always were a little shaky for native code especially
  • Cleaner design

However, I couldn’t help but also think back on the issues I’ve seen:

And from the responses of pip developers I’ve mostly gathered, quick workarounds are not desired. I understand this, these things could add up to a lot of nasty code and even muddier concepts if all worked around thoughtlessly.

However, I also can’t help but think that these issues are quite numerous and fundamental: no cross-compilation and no real ABI awareness for native packages in particular, as well as intentionally no plans to allow easy disabling of the build isolation without knowledge of how to manually process all of pyproject.toml: that seems like nothing that will be addressed in the near future. And I understand this as well, these things need thought and exceed what pip has been meant for at its core, and could derail the pip codebase if done hastily. And maintaining a secondary, no build isolation world for everyone duplicates a lot of work.

This leads me to one concluding thought, though. And as a disclaimer, I am personally affected by this through the python-for-android complications, so I am not neutral. But with the other user tickets above I don’t think I am alone with this thought: that python packaging might be better off if build isolation was treated as experimental and put off for now until these fundamental related areas are better addressed.

I can’t obviously convince anyone, I am just an outsider to pip. I used it, I have written code to deal with it, but I’m in absolutely no position to ask pip developers what to do, or to know what’s best. This is just my very own view which I wanted to share it in the hope you also find this worth thinking about. Thanks for all your work on pip!