With Packaging progressively moving towards pyproject.toml
, one use-case I found fairly difficult to support is “how to ship a package that depends on the C/C++ (or even rust probably) ABI of a different package”.
Well most of the time, you don’t have the choice and need to ship an sdist (instead of a wheel) because it really depends on what version (or even commit) the user base library is on.
Now it’s not necessarily “easy” to tell the user “RTFM” (aka. read the documentation) especially when they might get this dependency “transitively”:
pip install <cool_package>
== which depends on ==> A (sdist)
== needs ABI of == > B
Given this situation, pip would by default download B during the build phase of A, even though B might already be in the machine.
And that creates a few issues:
- The wrong version of B (not the same as on the user machine) might be pulled at build time
- B might not even be a publicly released package or released at all (think about a dev version sitting on the developer’s computer).
- B might be a massive library (think about how large AI libraries are) and downloading a bunch of dependencies is making the build time explodes for just no good reason most of the time.
My ideal world:
We should be able to say in the pyproject.toml
of A:
- passthrough library B (if it exists) otherwise download B as usual.
This would vastly reduce the load on warehouse servers and the user’s network.
And allow more flexibility when it comes to ABI compatibility.
I understand that’s the reason why --no-isolation
exists, though I think there are incredible advantages to have build isolation. And would very much like to not have to deactivate this important feature all together
Just gauging the interest / feedbacks on this idea - may lead to opening a PEP if the community thinks it a valuable “option” to have.
I believe it’s a feature that uv
already has, and that pip
should also adopt.
https://docs.astral.sh/uv/reference/settings/#no-build-isolation-package
no-build-isolation-package
Disable isolation when building source distributions for a specific package.
Assumes that the packages’ build dependencies specified by PEP 518 are already installed.
[tool.uv]
no-build-isolation-package = ["package1", "package2"]