Linux distro patches to `sysconfig` are changing `pip install --prefix` outside virtual environments

FFY00 · March 6, 2026, 12:20am

My recommendation with my maintainer hat on is to stop using Python installations provided by your system distributior. Instead, I’d recommend using something like GitHub - astral-sh/python-build-standalone: Produce redistributable builds of Python · GitHub (available via uv). If the systems are provisioned by you, even if you use another distribution as a base, you can also build Python from source without any patching.

It’s sad that my recommendation has to resort to this, but if you want a Python distribution that works as it was designed to, and that the Python upstream can support, this is it. My hands are tied.

Upstream support for patched versions of Python is provided in a best-effort capacity, without any guarantees.

stefanor · March 12, 2026, 2:12pm

I have been hoping for an upstream response to Fix: Patch scheme to posix_prefix if posix_local and --prefix is specified by cielavenir · Pull Request #13634 · pypa/pip · GitHub, either merging the PR, endorsing the idea of the change (for Debian to carry the patch) or rejecting it for a reason. Any of those outcomes would allow us to improve this situation.

pf_moore · March 12, 2026, 2:48pm

To expand on the question from the PR in a bit more detail:

I think that Debian are patching sysconfig.get_preferred_scheme("prefix") to return posix_local rather than posix_prefix. As far as I can see, that value is not a documented scheme name in the CPython docs (which is not in itself an issue, as my understanding is that distributions extending the available schemes is supported usage).

When tools (in this case pip) are passed a --prefix XXX option, they (not unreasonably, IMO) use the preferred “prefix” scheme, with the various scheme variables (base, platbase, prefix, …) set to the supplied prefix.

The PR is requesting pip to replace a preferred scheme of posix_local with posix_prefix in this case.

For me, there are some key questions:

Are tools expected to be prepared to handle non-standard schemes, and in particular to add distribution-specific workarounds for any such schemes? I’ll note that pip already has workarounds for the scheme osx_framework_library, so there is a precedent for this.
Assuming so, where should the various workarounds be documented? It seems like a huge risk if the only place we can find the “right” things to do is by looking at the source code of pip (or maybe uv, or somewhere else…?)
Should there be some way of reserving distribution-specific scheme names? The posix_local name doesn’t immediately say this is Debian-specific, and I wouldn’t be at all surprised if another distro wanted to do something similar, and used the same name. Who would arbitrate in that case?
Alternatively (or as well!) can we have some guidance on what distros and tools like pip should do in a situation like this? Should the distro be using a _venv prefix in a virtual environment (like the existing posix_venv and nt_venv ones)? Should tools be passing specific configuration variables, so that distros can write their schemes knowing exactly what will get substituted where?

We could simply accept the PR in pip, and take on the maintenance burden of another distro-specific patch. Or Debian could maintain their own patch to pip implementing the required logic. But that does nothing for uv, or for other installers. And it imposes a (small) ongoing maintenance cost on tools/distros. And it doesn’t do anything to help when this situation comes up again, in the future. So I’d personally rather take the time now, to get a proper answer on how sysconfig is intended to be used in this scenario, and get that answer written up in the sysconfig docs.

stefanor · March 12, 2026, 3:32pm

I don’t know the full history here, it’s a patch that’s been carried for a very long time. I always had the impression that it was intended to be pushed upstream (the naming fits in with the upstream conventions), but that the momentum for this had died a long time ago. And I seriously doubt there’s any upstream appetite for such a patch, these days.

I don’t think that’s at issue here. The prefix is being applied, with an additional (undesired) local element in it.

Both of these uses of the sysconfig scheme API are slightly awkward fits:

posix_local describes how Debian expects users (administrators) to install 3rd-party modules into the system path in /usr/local. It only works when base (etc.) is /usr, which it is on Debian Python install. All of the paths in it continue to point to correct system locations.
--prefix would build a scheme that Python is not necessarily configured to use, it’s up to the user to ensure that it points somewhere useful.

Are tools expected to be prepared to handle non-standard schemes, and in particular to add distribution-specific workarounds for any such schemes?

As a project you get to decide what to support and what not to support. I see both camps well represented in the PyPA community. As Python’s standard install tool, it’s obviously helpful for you to support as many standard environments as possible.

In this specific case, obviously we (Debian) can carry patches in our own packaged pip to support our own idiosyncrasies. This only works as long as users use our pip, which many would choose not to. But… it’s something in our power. If you prefer not to carry any changes, we can do this.

Assuming so, where should the various workarounds be documented?

I was wondering if PEP-739 was the way forward here?

We could maintain a PEP documenting known system idiosyncrasies.

I imagine this would quickly turn into an attempt to design the one-true install layout. And I’m not sure that it would be feasible to actually move towards a cross-distro standard like that. Things have ended up the way they have, due to a lot of local constraints.

I would be wary of trying to boil oceans as a prerequisite to fixing an integration bug.

But that does nothing for uv, or for other installers

The issue here is trying to install outside of the system, to a --prefix. Any installer acting with the bounds of the system or the virtualenv would behave correctly.

stefanor · March 12, 2026, 3:40pm

And I should say, I don’t think the posix_local scheme is adding much value any more. Especially once setup.py install is gone, directing package installation becomes a problem that can be dealt with in distro packaging tools and installers.

Obviously we can’t just pull it out overnight, we’d have to figure out the new strategy and get it in place. And people will be using new pip on old installs with this patch for a long time after that.

But… even though we’ve been carrying this patch for decades, it may be near the end of its useful life.

pf_moore · March 12, 2026, 4:29pm

This strikes me as the core of the issue here. Schemes are supposed to be supplied with config variables and handle them appropriately. So if posix_local can’t handle a base value other than /usr, it’s (IMO) the scheme that’s at fault. But trying to assign blame is counterproductive here, and I’m not trying to do that - just to understand better how we got into this situation (and work out how to avoid it in the future).

Yeah, I find the whole reasoning around --prefix (and --root, for that matter) to be pretty obscure. Maybe it’s because of my Windows background, but it makes very little sense to me. I’d personally^[1] rather we just had the system schemes (system, user and venv) along with a user-configurable “target” style scheme, that allowed the user to pick their own layout. But that’s a much bigger project, and one I simply haven’t got time for at the moment^[2].

Part of the problem is that I, personally, don’t feel qualified to make a call here. I understand that it’s a simple fix that would help Debian, but as I said, I don’t have a good view of the bigger picture. And the fact that @pradyunsg started this thread suggests to me that he also didn’t know the best answer (although in the intervening 3 years, enough might have changed to alter his view).

I really hope not. IMO, sysconfig gives us the flexibility to allow distros to set up their own layouts. All that is missing is a bit of documentation clarifying how schemes should map to typical installation scenarios. That’s something that should be in core^[3], but was traditionally hidden in the murk that was the distutils documentation - and when distutils was taken out of core, even the hints that were in there got lost. The only reason I think it’s a hard problem to address is because it crosses the boundary between “core functionality” and “packaging ecosystem” in a way that makes it feel like no-one owns the issue.

Yes, and uv pip install has a --prefix option as well.

On the other hand, if you’re saying that we should deprecate and ultimately remove --prefix, you won’t get any argument from me. Can we do the same for --root, please ? Realistically, that’s not an option, though, so deciding on correct behaviour for --prefix installs is necessary, like it or not.

github.com/pypa/pip

Rationalise and improve the `--target`, `--prefix` and `--root` options

opened 06:54PM - 08 Aug 22 UTC

pfmoore

type: feature request S: needs triage

### What's the problem this feature will solve? At present, pip allows the user… to select alternative install locations using `--target`, `--prefix` and `--root`. However, these are install-only options, and so do not work for other pip commands, like `list` or `uninstall`. ### Describe the solution you'd like Make the "target install scheme" options global, so that they apply to all commands. In particular, be able to manage directories populated using `pip install --target` the same as a site-packages directory (with support for upgrades, uninstalls, list and list --outdated, etc). ### Alternative Solutions Stick with what we have. Continue requiring users to delete and recreate directories populated via `--target` whenever they want to change things. ### Additional context In terms of implementation, experiments suggest that this can be achieved just by modifying `get_scheme` in `pip._internal.locations`, and `get_(default)_environment` in `pip._internal.metadata`. In practice, the bulk of the work will be removing the *existing* code in favour of the new approach. The proposed new approach will not work with the "legacy" `setup.py install` mechanism. We will need to decide whether to continue to support the existing options in that case. If we do, care will be needed to leave "just enough" of the existing code to support that route. The alternative would be to desupport using these options with the legacy install route, either by failing with an error suggesting use of `--use-pep517`, or by simply switching that option on automatically. Or we could delay implementing this feature until the legacy install route is finally removed 🙂 A feasible alternative/transition method would be to add one or more *new* global options that replicated the behaviour of the existing options using the new approach. The old options could then be removed at a later date. The big issues here are (1) thinking of good names for the "new" option(s), and (2) replicating all of the existing tests to check the new approach as well as the old one. One further point to consider is that `--target` currently appears to install into a temporary directory and move the files into the target location once the install is complete. It's not clear why this is done, and I could find no documentation that either explained or guaranteed this behaviour. Would it be acceptable to remove that as part of the reorganisation? (My best guess is that it's to avoid a failed install damaging the target location - if we make `--target` do a "first class" install, with upgrade and uninstall support, our existing protections against breaking environments should be a superior replacement for the "temporary location and copy" approach). ### Code of Conduct - [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).

↩︎
Plus, backward compatibility means it would just add to the confusion, not really address it ↩︎
Because core is in charge of defining the whole scheme/config var mechanism. ↩︎

FFY00 · March 18, 2026, 9:09pm

I have proposed this in the past, with just one restriction. Vendors would be able to add schemes, and change the default scheme(s), but they wouldn’t be able to change any of the existing ones. I think this restriction is crucial for the Python upstream to be able to maintain the code, ensure backwards compatibility, and provide a reliable UX.

This was deemed unsufficient.

As a maintainer, I genuinely feel trapped when trying to make any kind of improvement to sysconfig/site. I cannot design new APIs, or improve the UX, as these changes might be incompatible — even if just slightly — with downstream patching implementations. This makes any potential new APIs or other changes unreliable, hence effectively unusable in real-world code.

I have tried discussing with downstreams extensively, explaining which technical liberties I could give them, and which I could not, to be able to have sysconfig / site in a maintainable state. Where the liberties I provided where not enough, I tried proposing alternatives, and reaching a compromise. None of this was good enough.

Currently, sysconfig is not in a good state as far as maintainability goes, but it is my opinion that it’s in a better state that it would be if we were to accommodate downstreams with all the liberties they request. As such, my official stance is as stated in my reply above.

My social battery to engage in further discussions is severely low. Other folks are free to do so, and I am happy to discuss technical details, but I cannot put any more effort in trying to reach a compromise.

Sorry for long post, and venting here a little bit. I thought it was relevant to provide some context on what has been done to try solving this issue, as I see already discussed points being brought up again.