An update on pip and dependency resolution

I just posted an update on my blog, about the work being done toward pip’s dependency resolver (https://github.com/pypa/pip/issues/988) and what the plans are on that front.

https://pradyunsg.me/blog/2019/06/23/pip-update/


Thanks @sumanah for suggesting that I do this! =)

4 Likes

I’m going to use this thread to share some in-progress notes on the resolver work.

As of right now, people who install pip from GitHub master will have the ability to run pip install --unstable-feature=resolver and test the new resolver code. And less than half of the test suite fails! Expect errors and missing features, but it’s there!

A proper alpha or beta release will probably happen in May, and we’ll be sharing a “what to test” guide and asking for a lot of testing at that time. I’m working on a pre-announcement now that will go to the PSF blog and that we’ll be publicizing from there, and of course linking to from our wiki page.

Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation, for funding enabling this work! And thanks to @ei8fdb and Georgia Bullen from Simply Secure, and to @pradyunsg @pf_moore @uranusjr and Ilan Schnell for your work on the project.

2 Likes

The new PSF blog post is up. It discusses what is going to change, when, and how you can help (including some low-effort things you can do right now).

1 Like

Current work and near-term plans:

  • Our user experience researchers @ei8fdb and Georgia Bullen have been helping us think through specific decisions about the rollout. And they continue to interview you (please sign up if you haven’t already!) to learn more about people’s mental models, use cases, workflows, etc. They aim to put together a preliminary report this month to give pip’s developers some more systematic info, and I hope this will also be of use to some other packaging tools developers.
  • Ilan Schnell is working with us to improve our existing tests/testing and to add more tests to validate the new resolver.
  • We’ll be releasing pip 20.1b1 early this week, and it’ll include an alpha version of the new dependency resolver that people can test if they so feel inclined using --unstable-feature=resolver. Here’s context and some testing instructions. I’ll be publicizing that somewhat. This is thanks to the work of @uranusjr @pradyunsg & @pf_moore.
  • Then, in May, we’ll be releasing a pip version that will include a beta version of the new resolver, and I will publicize that (along with a more in-depth what-to-test guide) even more.

Thanks again to Mozilla & CZI for funding enabling this work!

4 Likes

I’ve posted our mid-year report at the PSF blog. One key thing to know: later this month we’ll be releasing pip 20.2 which will have a beta version of the new dependency resolver (pip 20.1 had an alpha version) available via an optional flag “--use-feature=2020-resolver”. We’ll be publicizing pip 20.2 a lot and asking a lot of users to put the new resolver through its paces.

We’ve released pip 20.2 and are discussing that at Announcement: pip 20.2 release! .

We made a small bugfix release, 20.2.1.

Also, members of the pip team gave interviews about the dependency resolver changes in two podcasts recently: in late May on Podcast.__init__ and more recently on Test & Code.

1 Like