I’m not associated with the project, I just found out about it today, but I think it is interesting for the Python packaging world to reshare here: uv: Python packaging in Rust
Inparticular it has some resolution features in it’s command uv pip install which I know users have been strongly wanting:
--override <OVERRIDE>
Override versions using the given requirements files.
Overrides files are `requirements.txt`-like files that force a specific version of a requirement to be installed, regardless of the requirements declared by any constituent package, and regardless of whether this would be considered an invalid resolution.
While constraints are _additive_, in that they're combined with the requirements of the constituent packages, overrides are _absolute_, in that they completely replace the requirements of the constituent packages.
--link-mode <LINK_MODE>
The method to use when installing packages from the global cache
[default: hardlink]
Possible values:
- clone: Clone (i.e., copy-on-write) packages from the wheel into the site packages
- copy: Copy packages from the wheel into the site packages
- hardlink: Hard link packages from the wheel into the site packages
--resolution <RESOLUTION>
[default: highest]
Possible values:
- highest: Resolve the highest compatible version of each package
- lowest: Resolve the lowest compatible version of each package
- lowest-direct: Resolve the lowest compatible version of any direct dependencies, and the highest compatible version of any transitive dependencies
--prerelease <PRERELEASE>
[default: if-necessary-or-explicit]
Possible values:
- disallow: Disallow all pre-release versions
- allow: Allow all pre-release versions
- if-necessary: Allow pre-release versions if all versions of a package are pre-release
- explicit: Allow pre-release versions for first-party packages with explicit pre-release markers in their version requirements
- if-necessary-or-explicit: Allow pre-release versions if all versions of a package are pre-release, or if the package has an explicit pre-release marker in its version requirements
This joins the list with rip which is primarly a backend library for other tools like pixi (which is more focused on the conda world but uses rip to integrate both conda and pypi dependencies).
That came rather out of nowhere! For people who haven’t checked the link, uv is from Charlie Marsh and Astral, the people behind ruff. The website says that “uv is designed as a drop-in replacement for pip and pip-tools, and is ready for production use today in projects built around those workflows”.
If that’s the case, they kept that very quiet during the development phase!
I look forward to reports on how well uv works in practice - in particular, it would be great if we could direct people asking for those features in pip to uv as an alternative. From a quick look, it does seem very fast, but a little limited right now (there’s no equivalent of pip list, for example). That’s to be expected for a 0.1 release.
@mitsuhiko was posting about Rye 10 days ago but not a peep about this!
It’s great to see more cooperation and consolidation in this space, and I’m excited to see where these tools go in the future. And not just because I’m a Python+Rust fanboy.
FYI I plan to run many of the complex resolution requirements I am familiar with against uv to see how ready it is.
Initial testing seems to indicate that extras are not supported yet (at least apache-airflow[all] seems to fail to install any extra requirements). So I would quible over the term “ready for production”. (Update: author replied within a few minutes, extras are supported, could be bug, we’ll see, Update 2: The issue is recursive extras are not currently supported: `uv pip install "apache-airflow[all]"` fails to install the required extras · Issue #1342 · astral-sh/uv · GitHub).
I am filling issues as I find them, if I get the same kind of response like I do when I file ruff issues then things will be fixed very quickly.
Cool. One thing I’m personally a little sad about is the fact that they chose to name the subcommand uv pip. That’s going to invite confusion over the command sets of pip and uv pip as well as making for annoyingly long commands like uv pip install <stuff> I’m not planning on making a fuss about it, but I’m not looking forward to having to explain that pip and uv pip are two different things…
It’s early days, they might be responsive to a suggestion about avoiding uv pip (and IMO I don’t think pointing out to potential for confusion and the burden it might place on others rises to the level of making a fuss)
I’m going to video chat with Charlie next week (maybe other Astral folks will be there as well). It would be nice to have them not re-implement the world and disrupt users so I’m encouraging them to use Hatch as a base. If there is agreement, there will be a transitory period where Hatch will be a Rust wrapper that ships an in memory Python that will execute the current code while bits get rewritten. If you are running the current binaries that’s kinda how it works already but less efficient.
I won’t know more until then but I have two pragmatic concerns which I would view as blockers:
Funding. I don’t know what their runway is so I don’t know what kinds of guarantees can be provided.
Hatchling will obviously be untouched forever but care needs to be taken for the Hatch-level plugin types like environments and publishers. As far as I understand there has been a desire to do this in Ruff but it is an incredibly hard technical challenge which they have not figured out.
Another concern that I have which is far less pressing is Linux distributors and their reaction to the change. They technically already need Rust because Hatch depends on keyring which depends on cryptography which uses that but I’m expecting ire in at least a few issues that get opened. That is not a blocker but it’s quite taxing responding to their feedback usually.
Hopefully I know the path forward (or no path) by the middle of March so I can incorporate the information in my PyCon talk.
If there is no path, Hatch development remains unchanged and I still view it as the likely candidate for being the Cargo for Python. The only potential change would be that I might actually use UV for dependency resolution when it becomes more stable and supports targeting arbitrary platforms and architectures.
First, just want to say: thanks for the warm reception and interest in our work. I have a ton of respect for pip and all the work that goes into it. Honestly, the last thing I want to do is make things difficult for the pip maintainers. If it turns out that the uv pip interface causes you all a bunch of problems, that’d be disappointing for me.
Just to spell it out explicitly: we went with the uv pip interface since I thought it was the simplest way to make the API immediately-clear and self-explanatory for existing pip and pip-tools users. We also wanted to scope those commands under some non-top-level namespace, to leave our CLI open to adding a more opinionated workflow in the future, like uv build and such. (If we used uv install, we could never add a more opinionated installation command in the future.)
My inclination is to see how things go before changing the interface, but I just want to make clear that if it becomes a problem for you all, I’m happy to consider it.
Thanks for the response. There’s already a discussion going on at Provide `pip` -> `uv pip` alias in virtual environments · Issue #1331 · astral-sh/uv · GitHub which seems very related to this. I’m strongly against the idea of uv ever installing a pip command that redirects to uv pip rather than being the actual pip command, and I think the fact that the suggestion was made in the first place indicates that there are already people who see uv pip and pip as at least somewhat interchangeable
I’m speaking purely for myself, as one of the pip maintainers but not on behalf of the pip project as a whole, but I’d really appreciate it if you didn’t use the subcommand name pip.
I’ve advocated for a long time in favour of having more competition in the installer area, so that people have alternatives to pip. I stand by that view, but I think it’s crucial that such alternatives compete on their own merits, advertising themselves as distinct offerings and not as “alternative versions of pip”.
If you were replicating the exact pip command line interface, then I could see this as a valid argument. But as it stands, it’s not - the first thing I tried to do when trying out uv was to run uv pip list and it didn’t work. If uv had its own command structure, I’d never have tried that but would instead have checked what the uv equivalent was. And in general, while I’ll concede that I’m far too familiar with pip’s CLI to count as an “average” user, my impression of uv pip was that it was an uncomfortably unfamiliar experience, vaguely reminiscent of pip but annoyingly different. And while I fully expect you’ll add more pip features over time, IMO this will likely just make the differences more subtle, but still just as uncomfortable (the “uncanny valley” effect).
Understood. And I honestly struggled to think of a subcommand name that would work. Assuming you don’t want to simply omit the pip and make all of the uv pip subcommands into top-level subcommands in their own right, there really isn’t a good option - uv install install is silly, for example. But none of that makes uv pip any more reasonable than any other arbitrary name like uv ipk (Install PacKage…)
Hopefully the other pip maintainers will share their views as well, so you have a more balanced response from the pip project as a whole. /ping @pradyunsg@sbidoul@uranusjr
What about using a uv rc-install where the rc prefix indicates that is unstable and you’re offering not stability guarantees… And when you decide to commit to an interface, can move it to uv install…
+1 to the request not to use uv pip since there are already situations where “pip” is in a subcommand or argument, which actually run pip. For example, python -m pip, conda run pip and pipx runpip.
Other than that, I’m glad about this funding opportunity and excited to see where this will go.
… as if thousands of python tooling maintainers suddenly cried out in terror and were suddenly silenced [1].
No seriously, congrats for the release @charliermarsh and colleagues.
Regarding the naming of the command, I understand the idea of copying the pip CLI to facilitate adoption.
I tend to think it has the potential to create confusion. As Paul said, it will always be subtly different from pip. I would certainly find it problematic if uv would install a global pip command alias, if only because we would then receive tons of support issues directed to the wrong project.
I’m curious to understand what your stance will be regarding detailed compatibility with pip. There are many historical quirks in pip (such as the handling of relative paths in requirement files, or upgrade strategies to name only two). Do you plan to replicate them for the sake of compatibility or follow your own (better) path?
So, IIUC, a bunch of people working for the same company have been secretly working on a pip replacement that will create more fragmentation in the Python ecosystem? Please correct me if I’m wrong
From my user’s POV, regardless of the technical qualities of said tool, I’m not sure how this should be welcomed as a success. I would rather call it a social and political failure, especially if the two development communities don’t converge on a shared project.
I’m a bit confused by this stance, the the PyPA and core packaging community working on standards the past few years rather than marshalling resources for tooling. I thought the whole idea of defining standards is so that people with the resources can go out and build tools based on those standards.
And is not the recent explosion in new tooling attempting to handle Python packaging in one way or another is in part because those tools can largely follow a lot of standards now rather than having to reverse engine whatever setuptools and Pip does?
If so, then it seems inevetible to for there to be some fragmentation in the short and medium term, at least until best practises have time to be developed with all these new tools and which ones manage to keep being updated outlast ones that don’t.
The success would then be that these tools fit within the standard that the PyPA has been focusing on, rather than say going off and building their own standard and ecosystem as has already once happened with Python, i.e. another conda.
Is it? From what I see, the PyPA’s activity doesn’t stop at writing standards. The PyPA and its members maintain pip, wheel, setuptools, twine…
Well, conda hasn’t been obsoleted by any recent development, AFAIK. In particular, conda is language-agnostic and is able to handle non-Python software and dependencies. That functional gap is unlikely to be closed by pip or uv in the near future.
Being an a occasional contributor to Pip and probably the main person to triage dependency resolution issues I can tell you if the Core CPython team have the impression that Pip has successfully marshaled enough resources beyond “keep the lights on” bare minimum there is a significant communication problem going on.
Pip has barely enough resources to function, almost every single discussion ends with “unfortunately I don’t have time to review and/or make this”.
conda was developed exactly because of the lack of forward momentum on Python standards that the now “Anaconda” company was proposing. The fact they made their standard language agnostic is beside the point, their focus was on the Python data science ecosystem and remains that way, their maintenance of other language ecosystems like Perl fell by the wayside.
If the standards hadn’t existed then tools like this may see a similar opportunity to go and build another packaging ecosystem instead of relying on the one built around the core packaging standards.
I won’t have more details regarding collaboration until I meet with Charlie next week but I will say I am not a fan of how this project came about not just in secrecy (private repo) but also there was not from my point of view any effort of outreach on their part. This, again from my point of view, was a complete departure from what happened with Ruff which is how the original developer goodwill was fostered.
This also materially impacts not just users with increased confusion but also developers that have been putting in work for free. For example, the entire time this was being developed Brett was working on a resolver with his mousebender project which could have been collaborative were they actually interested in reaching out because all of the work we do in the PyPA (and others like PDM and Poetry) is in the open and everybody knows. Unless they actually didn’t know, in which case it’s unfortunate that building is occurring without knowledge of the landscape and target audience.
To be clear, I don’t mind competition among tools but specifically I find it in bad form to work on the exact thing that others are already spending their time on without at least an attempt at collaboration. Not just mousebender but also there is literally another company (prefix.dev) working full time on a resolution mechanism for Python packages.
So, do you think the existence of a competing tool claiming to replace pip will improve the situation for pip?
No, it’s a difference in vision. conda solves a set of problems that the Python packaging community is not interested in solving, and it does so by imposing specific constraints (such as custom-compiled binaries, including the Python interpreter and even foundational libraries such as OpenSSL). Specific constraints that would be counter-productive for the Python packaging community to impose.
So, no, it was not merely a problem with “momentum” or human resources, it’s a fundamental different set of choices and chosen constraints.
Yes, if one or more of these tools are completely successful, Pip can eventually be retired and require zero resources to maintain.
There has been many recent discussions to distribute CPython executables via a wheel, and tools following the core standards now appearing where you install them first and they will install the correct Python version for you.
Users using the the core packaging standard want to solve the same problems, I disagree it is a matter of constraints but rather a matter of timing that the core ecosystem and standards were not mature enough to even think about these problems.
