Handling of Pre-Releases when backtracking?

[notatallshaw (Damian Shaw)]

Thanks for all the context! I would like to mention some real world use cases to be aware of (to be clear not saying any of them should stop adopting something simpler to reason about):

@dstufft

I would probably drop most of the information about prerelease handling, and just put in a recommendation that tools should exclude prereleases by default unless the end user provides some configuration or flag to tell it to use prereleases. I think this matches what most ecosystems that treat prereleases specially at all do now.

I see pre-release specifiers used in the wild in three different ways that would not work well with the -pre flag Pip offers:

1. A single dependency that doesn’t yet have a final release, or at least a final release for a given platform (came up a lot with the release of Apple Silicon)
2. A single dependency that has released a new feature that is only in pre-release
3. A hack to test a subset of dependencies for their pre-release versions (e.g. How do I exclude Pip selecting any pre-release? · Issue #12470 · pypa/pip · GitHub)

@dstufft

• Ditch the === specifier, which I suspect might never have been used?

I’ve seen these in the wild, but I think it was mostly to handle legacy versions, which Pip still supports (but emits a deprecation warning).

@dstufft

Maybe support grouping and and and or?

There certainly are users who would take advantage of this. But, at least in my experience, they make dependency resolution significantly more difficult. Simpler algorithms, like the mostly DFS one that resolvelib uses, might struggle in a world of these kind of dependencies.

@dstufft

As far as specifying dependency resolution itself goes, I don’t think that is something we should do. In theory any version that matches the specifier should be acceptable, and how the resolver works shouldn’t matter, except to users of that tool.

I would add that when version or specifier specifications that include conditional or optional behavior have an outsized impact on either the complexity, or ambiguity, of how dependency resolvers must handle them. For Python specifiers I’m are of two features that have this issue in the way they are defined:

• Pre-releases
• Extras