[Build Isolation Passthrough] Allowing a "specific package" to always passthrough the build isolation

With Packaging progressively moving towards pyproject.toml, one use-case I found fairly difficult to support is “how to ship a package that depends on the C/C++ (or even rust probably) ABI of a different package”.

Well most of the time, you don’t have the choice and need to ship an sdist (instead of a wheel) because it really depends on what version (or even commit) the user base library is on.

Now it’s not necessarily “easy” to tell the user “RTFM” (aka. read the documentation) especially when they might get this dependency “transitively”:

pip install <cool_package> == which depends on ==> A (sdist) == needs ABI of == > B

Given this situation, pip would by default download B during the build phase of A, even though B might already be in the machine.

And that creates a few issues:

  • The wrong version of B (not the same as on the user machine) might be pulled at build time
  • B might not even be a publicly released package or released at all (think about a dev version sitting on the developer’s computer).
  • B might be a massive library (think about how large AI libraries are) and downloading a bunch of dependencies is making the build time explodes for just no good reason most of the time.

My ideal world:

We should be able to say in the pyproject.toml of A:

  • passthrough library B (if it exists) otherwise download B as usual.

This would vastly reduce the load on warehouse servers and the user’s network.
And allow more flexibility when it comes to ABI compatibility.

I understand that’s the reason why --no-isolation exists, though I think there are incredible advantages to have build isolation. And would very much like to not have to deactivate this important feature all together :smiley:

Just gauging the interest / feedbacks on this idea - may lead to opening a PEP if the community thinks it a valuable “option” to have.

I believe it’s a feature that uv already has, and that pip should also adopt.

https://docs.astral.sh/uv/reference/settings/#no-build-isolation-package

no-build-isolation-package
Disable isolation when building source distributions for a specific package.
Assumes that the packages’ build dependencies specified by PEP 518 are already installed.

[tool.uv]
no-build-isolation-package = ["package1", "package2"]

@msarahan @aterrel @barry

2 Likes

I think this is a tool UI matter, and as such isn’t something we should be putting in a standard. As you say, if users want this, it’s available in uv. If someone were to create a PR for pip, it would be considered (although I imagine it could be pretty complex, and with the resource issues pip has, it might be difficult to get it reviewed).

1 Like

I’m not sure how many threads there are specifically about this but in general there are many threads in which this problem is discussed. A conceptually simple solution is making it possible for a wheel to depend on another wheel. Unfortunately it is a lot easier for me to describe that solution in a few words than it is for anyone to turn it into something that is useful in practice.

Do you have a pointer to the main thread on this topic maybe? I didn’t find it

Well let me know if there’s something I can do to help on this.

I’m trying to seriously ramp up on PIP and core packaging libraries in 2025 and hopefully be able to help with general maintenance / review after some time. This could be an interesting mentorship opportunity if you’re interested in getting some help with packaging tooling.

1 Like

I personally don’t think this is strictly just an UI matter. It certainly can be worked around on a case-by-case basis by every tool itself. However, in order for this to work reliably for all users (read: not require every user to special-case installing the package in question), it would need to somehow be integrated into a PEP 517-based workflow.

To speak in pseudocode, we’d need something like:

[build-system]
requires = ["B == {system_version}, >= 1.2.3"]

That would:

  1. If a frontend is building a sdist specifically to install it in an environment where B >= 1.2.3 is installed, it would replace {system_version} with that version.
  2. Otherwise, == {system_version} would be removed entirely.

That is, presuming it would be sufficient for the isolated build to install the exact same version, and not actually fall back to non-isolated build entirely.

Or we could declare that this particular sdist does not support isolated builds entirely.

Of course, the other part is that A also needs to dynamically adjust its dependencies to B == {version_used_to_build}, so that it is clear that wheels are only going to work with that version. Which opens the other problem of how should such wheels be handled — should frontends always force replacing B with required version and fail if the version of B can’t be installed, or should there be mechanism to indicate “fall back to building sdist, as it has a wider version range”?

It is a hard problem.

There is Depending on packages for which an ABI matters on the general topic, I think. It has a bunch of links. I do not know if this website is kept up-to-date.

2 Likes