PEP 440: clarification on prefix version matching with pre-release/post-release segments

In Possible bug with wildcard version matching · Issue #425 · pypa/packaging · GitHub, an issue was raised when using a specifier such as ==1.1.0.post0.*.

While this is most definitely an edge case, it’s still a valid specifier per PEP 440 – Version Identification and Dependency Specification | peps.python.org

pre-release and post-release segments are intentionally included in prefix matching in the PEP.
Some examples of this might be welcomed to clarify this usage.

My current understanding is:
==1.1.0.post0.* means 1.1.0.post0 or any development version of 1.1.0.post0
==1.1.0a0.* means 1.1.0a0 or any combination of post-release/development version of 1.1.0a0

The two specifiers shown above are semantically identical to the ones below where expectations might differ:
==1.1.0.post.*
==1.1.0.alpha.*

@ncoghlan, @dstufft, as authors of the PEP, can you confirm/infirm my understanding ?

cc: @brettcannon, @uranusjr, @pradyunsg who participated in discussions in Possible bug with wildcard version matching · Issue #425 · pypa/packaging · GitHub

There’s two things around this:

  • specifiers like ==1.1.0.post0.*.
  • specifiers like ==1.1.0.* not matching 1.1.a1.

For the first one… I’m wondering if the tooling should be rejecting those specifiers outright. :slight_smile:

My reading of the PEP is that it wasn’t intended for the prefix/wildcard/*-based match to be used for anything other than release-segment-only versions. That seems to match all the examples as well. I’d prefer if we only allow .* on versions without additional alpha/beta/rc/dev/post suffixes. The idea of “any development release of a post-release” seems very esoteric, and I’d prefer to not have additional complexity for such edge cases and outright reject them. So… ==1.1.post0.* would fall in this category too. We can relax this at a later date, if a user flags that this is useful to them somehow.

==1.1.0.* not matching 1.1.a0 could be considered a bug? I’m pretty sure that normalising these versions is a reasonable thing to do for this case.

I left out this one in my description. I included a variation of post->alpha rather than including the bug I reported in packaging for ==1.1.0.* not matching 1.1.a0 as I think it’s pretty well defined in the PEP and only a bug in packaging.

The idea of “any development release of a post-release” seems very esoteric, and I’d prefer to not have additional complexity for such edge cases and outright reject them.

Agreed, but is that the way the PEP was intended ?

I think the PEP probably had no specific intention in this case. While we try to make sure we’ve thought all of the implications through when writing a PEP (and it’s gratifying that people assume we’re that thorough :slightly_smiling_face:) the reality is that edge cases like this were probably never even thought about, and are at best interpretations after the fact.

To put it another way, I think that arguing over “what the PEP intended” is pointless at this stage, and we should decide based on practicalities - on which basis I agree with @pradyunsg that it’s probably best to simply declare these edge cases as invalid and reject them.

1 Like

I don’t think myself or Nick considered at all the idea of something like ==1.1.0.post0.* for the PEP to have a thought one way or the other.

I think that disallowing them makes the most sense though.

2 Likes

To put it another way, I think that arguing over “what the PEP intended” is pointless at this stage, and we should decide based on practicalities

I don’t think asking the question is pointless. I did assume there was an intention after reading the PEP but I’d rather assume there was one and ask to clarify what it was than assume it was not thought through.
Now that there’s a feedback from the authors, I won’t be arguing anymore.

Agreed (and sorry if I gave the impression that I thought your question was pointless). What I was trying to say was that the most likely explanation for the PEP being unclear was that no-one had thought of that case, and we’d be better just making a decision now rather than debating what the PEP text “must have been intended to mean”. But as it happened, that debate never occurred[1], so it looks like we have a successful outcome here.

Thanks for pointing out the gap in the spec!


  1. Unusually for packaging topics… :wink: ↩︎

Would a PR amending PEP 440 be accepted given its current status (I should have included the following quote from the start, sorry for the late introduction) ?
i.e changing:

It is invalid to have a prefix match containing a development or local release such as 1.0.dev1.* or 1.0+foo1.* . If present, the development release segment is always the final segment in the public version, and the local version is ignored for comparison purposes, so using either in a prefix match wouldn’t make any sense.

to something like:

It is invalid to have a prefix match containing a pre-release, post-release, development or local release such as ==1.0a1.*, ==1.0.post1.*, ==1.0.dev1.* or ==1.0+foo1.*.
Prefix matching is only valid on release segments such as ==1.*, ==0!1.*

In any case, I proposed pull-requests in packaging for both allowing (pre-dating the discussion, fixing an edge case) and disallowing pre-release, post-release specifiers with prefix matching as proposed here.

The existing devX special case is due to the order of clauses when combined:

[N!]N(.N)*[{a|b|rc}N][.postN][.devN]

It is necessarily last, so having a wildcard after it necessarily has no effect.

Whether amending the PEP text is appropriate depends on what tool and library authors have implemented. If we’re documenting an existing de facto consensus, then that would just be an editorial update via PR (no new PEP needed), but if we’d be imposing changes to published software (as in this case) then it’s best to cover it as a spec update that makes the PyPA specs page the official reference and tweaks the comparison rules (another thread pointed out an odd interaction between wildcards and inclusive ordered comparisons that could also stand to be amended).

Regarding x.y.z.* not matching pre-releases, it wouldn’t match them on its own, but that would be due to pre-releases being ignored by default. If pre-releases have been explicitly included, then a wildcard should match them the same way a non-wildcard version match would.