As the one of the original author of PEP 440, I can say pretty definitively that the reason that the PEP says this:
Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.
and not this
Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system or explicitly requested by the user.
Is because there were widely used packages at the time that had versions which didn’t technically comply with PEP 440 (because they predated PEP440) and got interpreted as prerelease versions, and we were trying to prevent breakages (a lot of effort in PEP 440 and a lot of weird edge cases came from trying to gain another 1%-2% of compatibility with existing versions on PyPI). One such package at the time was pytz which used versions like 2005a.
There were also a number of packages that were only available as pre-releases (for one reason or another) and we wanted to avoid additional breakages.
At the time of PEP 440, none of the Python installers had any sort of real dependency resolving or backtracking or anything of that nature, so the implications of that statement, wrt to how it impacted backtracking/resolving simply wasn’t a thing that existed to be considered. I suspect this is why (1) from above is built into the Finder instead of into the resolver as well.
I don’t have nearly enough time to do it (though I’d be willing to offer opinions where folks cared) but I suspect that a follow up to PEP 440 may be warranted to clean up some of the edge cases or “weird corners” that we put in explicitly for compatibility. It’s been 10 years since PEP 440 and I suspect a lot of those cases simply no longer apply to anything that has been released recently, and we’ve learned things as an ecosystem that weren’t obvious a decade ago.
Some possible things (would need to see how widely used each of these are, so these aren’t concrete suggestions):
- I would probably drop most of the information about prerelease handling, and just put in a recommendation that tools should exclude prereleases by default unless the end user provides some configuration or flag to tell it to use prereleases. I think this matches what most ecosystems that treat prereleases specially at all do now.
- Figure out if anyone ever actually used Epoch, and if not does it make sense to keep supporting it.
- Disallow combinations of pre-release, post release, and dev releases.
- Add specifiers that are easier to use (The biggest one being
^=, which is a semver compatible update is probably the major one of these). - Ditch the
===specifier, which I suspect might never have been used? - Maybe support grouping and
andandor? Currentlyandis supported through,, but more complex expressions may mean that some complicated version specifiers are easier to specify usingorand/or( )for grouping (though this also extends to PEP 508, and it might be better to do this there).
I dunno though, I do think there’s been enough pain points come up over the years, and if someone is going to tackle a Versioning v2 (or well… I guess v5?, distutils, setuptools, distuils2, pep 440, and hypothetical new thing), I think it would be very useful to try and go through past discussions and issue trackers to find out where they are.
As far as specifying dependency resolution itself goes, I don’t think that is something we should do. In theory any version that matches the specifier should be acceptable, and how the resolver works shouldn’t matter, except to users of that tool.