Version specifier sets and prereleases

Hi, I recently filed pip issue #7579, because pip surprised an user by installing a prerelease while there was a stable release satisfying the version specifier. Specifically, the dependency specifier was >=0.6.22rc1, and despite --pre not being passed and 0.6.22 being available, 0.7rc2 was installed.

The wording in PEP 440 here is not clear, but I still think it’s unambiguous:

Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release.

Note the “all”: All version specifiers (including the ones containing X.YrcZ) have rcs excluded unless the user explicitly requested them being enabled (using a global statement of intent like --pre). I think the behavior exhibited by pip is only technically correct (because the PEP says “should” and not “must”), but not what an user would expect. I’d expect pip to behave like this when >=0.6.22rc1 is requested:

  • If --pre is given, the newest available version is installed, no matter if prerelease or not.
  • If --pre is not given, and a stable version satisfying the specifier is available, it is installed.
  • If --pre is not given, and no stable version satisfying the specifier is available, the highest available 0.6.22 prerelease is installed.

This has implications for the PEP (which I think should be modified to be more clear for this case) and for the documentation/semantics of packaging.specifiers.SpecifierSet (I would link it but for some reason I can’t put 3 links), which doesn’t distinguish between a global preference like --pre and what’s encoded in the specification string. I think it makes no sense to derive this global preference from a specifier string, so the default should be SpecifierSet(specifiers, prereleases=False), and SpecifierSet('>=0.6.22rc1') should therefore mean one of the “--pre is not given” cases above.

/edit: Also this is thread 3000, which clearly means that it’s the future of Python :laughing:

1 Like

I think the only doc should be updated to document the status quo.
Something like: “using pre-release version specifier implies --pre for the installed package”.

Otherwise, even 0.6.22rc1 cannot be installed by >=0.6.22rc1 version spec without --pre; which is weird and breaks the backward compatibility.

Not true, PEP 440 states that tools should allow pre-releases when there is a pre-release that satisfies the constraint, but no full release that does so. So >=0.6.22rc1 will install rc1 until 0.6.22 is released, at which point the full release takes precedence.

The problem here is that currently >=0.6.22rc1 will install 0.7b1, even if 0.6.22 exists as well. And that’s very unlikely to be what the user wants, as far as I can see. PEP 440’s rules are quite careful to stop that happening without the user explicitly opting in.

2 Likes

Probably should cc @dstufft