# A PyPI resolver library in Rust

**URL:** <https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973>\
**Category:** Announcements\
**Created:** [October 23, 2023, 6:31pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973 "2023-10-23T18:31:42Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![wolfv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wolfv/32/15462_2.png) [@wolfv](https://discuss.python.org/u/wolfv)\
**Post date:** [October 23, 2023, 6:31pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/1 "2023-10-23T18:31:42Z")

</div>

Hi friends,

we’re working on a new package manager that brings together the `conda` and `pip` packages more closely. Our package manager is called `pixi`.

We know that a lot of conda users also use Python depencies from PyPI. That’s why we recently made our resolver (called resolvo) generic and lazy, to work well with Python packages. We bundle that + the metadata handling in a new Rust crate called `rip`: [GitHub - prefix-dev/rip: Solve and install Python packages quickly with rip (pip in Rust)](https://github.com/prefix-dev/rip)

I also wrote a short blog post about it: [Introducing rip - the fast & barebones pip implementation | prefix.dev](https://prefix.dev/blog/introducing_rip)

I would love to see people try it out or integrate it into their projects. We are going to work on integrating `rip` into `pixi` very soon and hopefully have something to show at the end of November. Until then you can try `rip` standalone with `cargo run -- flask` (or any other package). Another big topic that we will have to work on for real-world scenarios is `sdist` support …

---

<div class="post-metadata">

**Author:** ![willingc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/willingc/32/20_2.png) [@willingc](https://discuss.python.org/u/willingc)\
**Post date:** [October 23, 2023, 6:52pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/2 "2023-10-23T18:52:16Z")

</div>

Thanks @wolfv. I’m looking forward to seeing `pixi` soon. Thanks for sharing.

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [October 23, 2023, 7:03pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/3 "2023-10-23T19:03:27Z")

</div>

We can’t use it until it’s able to support custom PyPI indexes (and work without conda packages). But otherwise seems nice.

---

<div class="post-metadata">

**Author:** ![wolfv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wolfv/32/15462_2.png) [@wolfv](https://discuss.python.org/u/wolfv)\
**Post date:** [October 23, 2023, 7:08pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/4 "2023-10-23T19:08:06Z")

</div>

@bernatgabor `rip` totally supports custom pypi indexes (we tested with the pytorch one). And also does not require any conda package at all 🙂 It looks at whatever `python` it finds on your `$PATH` and uses that to populate the environment markers.

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [October 23, 2023, 7:20pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/5 "2023-10-23T19:20:48Z")

</div>

This is very cool and seems relevant to some recent packaging discussions \[1\].

It’s cool that the solver works with both indexes. I’m curious, do you think there’s a way to cross the conda-forge / PyPI boundary while resolving? I’ve been thinking about this lately and it seems doable but there are some potential pitfalls. One obvious one is that the package names aren’t exactly 1-to-1, but I don’t know how often.

I’m thinking of the scenario where I want to install `A` from PyPI and it requires `B` which is on both PyPI and conda-forge. `conda install A` doesn’t work because `A` isn’t there. `pip install A` will install `B` from PyPI (let’s assume I don’t want to do that for some reason).

Right now the solution is `conda install B; pip install A` but it’d be great if a tool could figure that out on its own. There are a _lot_ of recipes on conda-forge that could be removed\[2\] if `conda` knew to install from PyPI when needed.

* * *

1. e.g. [this one](https://discuss.python.org/t/the-10-year-view-on-python-packaging-whats-yours/31834) and [this one](https://discuss.python.org/t/trusted-publishing-of-other-peoples-wheels/33173) 

2. lessening the small burden of updating versions and so on

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [October 23, 2023, 7:22pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/6 "2023-10-23T19:22:25Z")

</div>

> [@wolfv](#):
>
> @bernatgabor `rip` totally supports custom pypi indexes (we tested with the pytorch one). And also does not require any conda package at all 🙂 It looks at whatever `python` it finds on your `$PATH` and uses that to populate the environment markers.

How do I install rip? 😆

```auto
❯ cargo install rip
    Updating crates.io index
  Downloaded rip v0.0.1
  Downloaded 1 crate (295 B) in 2.34s
error: there is nothing to install in `rip v0.0.1`, because it has no binaries
`cargo install` is only for installing programs, and can't be used with libraries.
To use a library crate, add it as a dependency to a Cargo project with `cargo add`.

```

Or is there no frontend just yet? I seen no sdist support for now, is that correct? I tried using through pixi, but that does build on conda, based on outputs. (In a parallel world I’d like to replace pip with rip, not sure if that’s on the roadmap).

---

<div class="post-metadata">

**Author:** ![wolfv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wolfv/32/15462_2.png) [@wolfv](https://discuss.python.org/u/wolfv)\
**Post date:** [October 23, 2023, 8:03pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/7 "2023-10-23T20:03:48Z")

</div>

@jamestwebber yeah, there is quite some “churn” involved with maintaining a bunch of `noarch` packages on conda-forge, indeed. We’re not at the stage where we want to resolve with both indexes at the same time but we think it’s in the realm of possibilities. `conda-forge` maintains the following list: [https://github.com/regro/cf-graph-countyfair/blob/master/mappings/pypi/name\_mapping.yaml](https://github.com/regro/cf-graph-countyfair/blob/master/mappings/pypi/name_mapping.yaml)

The problem is that – to build a conda-forge package – even one with binary extension, you might need some lower-level noarch package. So even at package-build time the resolver might have to pull some packages from PyPI and not conda-forge. Not sure if we want to do that given the pretty different operating model of PyPI vs conda-forge, but let’s see how these things evolve 🙂

@bernatgabor  
I think the `rip` crate on [crates.io](http://crates.io) is something else, indeed! For the moment you need to pull the repository, and then run `cargo run`. E.g.:

```auto
git clone https://github.com/prefix-dev/rip
cd rip
cargo run --release -- flask

```

To use a different index, use `--index-url`.

---

<div class="post-metadata">

**Author:** ![ofek](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ofek/32/1033_2.png) [@ofek](https://discuss.python.org/u/ofek)\
**Post date:** [October 23, 2023, 8:29pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/8 "2023-10-23T20:29:01Z")

</div>

@wolfv Is there an option to pass the path to a particular Python executable?

---

<div class="post-metadata">

**Author:** ![wolfv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wolfv/32/15462_2.png) [@wolfv](https://discuss.python.org/u/wolfv)\
**Post date:** [October 23, 2023, 8:43pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/9 "2023-10-23T20:43:59Z")

</div>

I think there is currently no way of specifying the path to the python executable, but the `rip` binary will look into your `PATH` variable and find the first one available.  
From Rust, it would be very easy for you to give a concrete path to a Python executable (instead of searching on PATH).

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [October 23, 2023, 8:48pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/10 "2023-10-23T20:48:21Z")

</div>

> [@wolfv](#):
>
> The problem is that – to build a conda-forge package – even one with binary extension, you might need some lower-level noarch package. So even at package-build time the resolver might have to pull some packages from PyPI and not conda-forge. Not sure if we want to do that given the pretty different operating model of PyPI vs conda-forge, but let’s see how these things evolve 🙂

I do think this would require a [CFEP](https://github.com/conda-forge/cfep) or something similar to define how it should work, but I think it’d be a major improvement to the ecosystem if it’s possible.

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [October 23, 2023, 10:06pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/11 "2023-10-23T22:06:29Z")

</div>

Would it be possible to install it and not run it from source essentially?

```auto
cargo run --release -- pyspark
2023-10-23T23:02:59.573715Z WARN rattler_installs_packages::resolve: Not considering pyspark 3.5.0, 3.4.1, 3.4.0, 3.3.3, 3.3.2, 3.3.1, 3.3.0, 3.2.4, 3.2.3, 3.2.2, 3.2.1, 3.2.0, 3.1.3, 3.1.2, 3.1.1, 3.0.3, 3.0.2, 3.0.1, 3.0.0, 2.4.8, 2.4.7, 2.4.6, 2.4.5, 2.4.4, 2.4.3, 2.4.2, 2.4.1, 2.4.0, 2.3.4, 2.3.3, 2.3.2, 2.3.1, 2.3.0, 2.2.3, 2.2.2, 2.2.1, 2.2.0.post0, 2.1.3, 2.1.2 because there are no wheel artifacts available

```

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [October 23, 2023, 11:23pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/12 "2023-10-23T23:23:11Z")

</div>

> [@bernatgabor](#):
>
> Would it be possible to install it and not run it from source essentially?

After you do `cargo build --release` you should find a single executable in the `target/release` folder. You can copy that into a suitable `bin` folder.

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [October 23, 2023, 11:34pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/13 "2023-10-23T23:34:47Z")

</div>

Would be easier to just do, cargo install rip 🤔 not having to clone and copy things around. Then you can also use [GitHub - nabijaczleweli/cargo-update: A cargo subcommand for checking and applying updates to installed executables](https://github.com/nabijaczleweli/cargo-update#usage) to keep it up to date. From my side the project is unusable until it supports sdists 😊 because, well on any reasonable sized project there will be some dependencies that don’t have wheels. 😊

---

<div class="post-metadata">

**Author:** ![wolfv](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/wolfv/32/15462_2.png) [@wolfv](https://discuss.python.org/u/wolfv)\
**Post date:** [October 24, 2023, 6:20am UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/14 "2023-10-24T06:20:18Z")

</div>

For sure. We’re mostly shipping the rip binary as a proof-of-concept. The main use-case is to use rip as a library.

---

<div class="post-metadata">

**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\
**Post date:** [October 24, 2023, 3:17pm UTC](https://discuss.python.org/t/a-pypi-resolver-library-in-rust/36973/15 "2023-10-24T15:17:47Z")

</div>

Understandable, just makes it hard to give feedback on it 😊
