Reconciling `packaging` module & The Dependency Specifier Spec's non-version to version comparison rules

The packaging module has a bug resulting in it raising an error when comparing non-version values with valid version values in dependency specifiers (#774). Its behaviour is different to the Dependency Specifiers Spec, but discussion of the issue (#774)
on GitHub has revealed that implementing the spec as-is would result in counter-intuitive behaviour when comparing the order of non-standard version values. So some discussion is needed to decide how to bring packaging in line with the spec and whether the spec should be changed.

The bug

Say you want your package to require a dependency only on macOS 12 (Darwin 21). Your package metadata might contain:

Requires-Dist: requests; platform_system == 'Darwin' and platform_release < '22'

On macOS, import platform; print(platform.release()) is 22.1.0 (a valid packaging version). Whereas on Linux, platform release may be 6.10.14-linuxkit which isn’t a valid packaging version.

The spec requires that comparing two valid version values use the normalised version comparison rules. When one or both are not valid versions, regular Python operator rules are used (so lexicographic ordering of strings).

packaging currently raises an error when the right-hand side of an expression is a valid version and the left-hand side is not.

So in when evaluating this example on Linux, platform_release takes on the non-version value and it evaluates '6.10.14-linuxkit' < '22' which raises an error.

Reproduction instructions

For example, installing this pyproject.toml project with pip currently crashes:

# pyproject.toml
[project]
name = "version-comparison-example"
version = "1.0.0"
dependencies = [
    "requests; platform_system == 'Darwin' and platform_release < '22'",
]

[build-system]
requires = ["hatchling >= 1.26"]
build-backend = "hatchling.build"
$ python -c 'import platform; print(platform.release())'
6.10.14-linuxkit

$ touch version_comparison_example.py 

$ pip install .
[...]
ERROR: Exception:
Traceback (most recent call last):
[...]
pip._vendor.packaging.version.InvalidVersion: Invalid version: '6.10.14-linuxkit'

$ pip --version
pip 25.1.1 [...]

Similarly, using an extra name like v0 that is a valid version also crashes (example project config), as the package metadata ends up containing a specifier like this:

Requires-Dist: typing-extensions; extra == 'v0'

This v0 value is treated as a valid version and compared against non-version values, which crashes. (I opened a PR to fix this (#883), which is how I came to this general issue.)

The spec’s behaviour

The spec requires that mark expressions (e.g. platform_release < '22') are compared as normalised versions when both the mark’s constant value and the variable parse as a packaging version. When one or both do not parse as versions, the values are compared using the normal Python operator behaviour.

Spec's Wording

Comparisons in marker expressions are typed by the comparison operator and the type of the marker value. The <marker_op> operators that are not in <version_cmp> perform the same as they do for strings or sets in Python based on whether the marker value is a string or set itself. The <version_cmp> operators use the version comparison rules of the Version specifier specification when those are defined (that is when both sides have a valid version specifier). If there is no defined behaviour of this specification and the operator exists in Python, then the operator falls back to the Python behaviour for the types involved. Otherwise an error should be raised.

  • packaging (and projects using it, like pip and most Python packaging tools) raise an error instead of using regular Python operators
  • The uv project has its own implementation of dependency specifiers, and it follows the spec (uv can install the problematic example projects above).

The issue with the spec’s behaviour

Implementing the spec’s behaviour results in counter-intuitive comparison results when comparing the order of version-like (but not valid packaging version) values.


A distribution (Python package)'s version is required to satisfy the version spec’s rules, but the values of several environment markers are provided by the OS, system software, or are not expected to be versions (e.g. extras).

Indeed the spec has a table listing the environment marker names and their types (String, Version etc) and only python_version, python_full_version and implementation_version are defined to be Version, the rest are String (or sets of strings).

However the marker expression rules described above treat any value as a version if it parses as a version. This may happen to be the case (as in the macOS platform_release example and the extra example).

This results in most version comparisons using intuitive ordering based on numeric value of version components, but a hard cliff occurs where values are not valid versions, and the comparison instead uses regular string order by character-by-character value:

  • platform_release >= '20':
    • intuitively evaluates false when platform_release is 6.7.0 or 6.7.0+gentoo as these are valid versions,
    • counter-intuitively it evaluates true when platform_release is 6.7.0-gentoo, as this is not a valid version and so '6' >= '2' is evaluated, which is true.

There are some examples of version-to-version and non-version-to-version equality and ordering comparisons in the tests for MR #883.

Equality comparisons (==, !=, etc) are also affected by this, but it’s not really a problem in practice, because the difference in behaviour is limited to normalisation of version components being used in version-to-version comparisons but not otherwise. E.g. 'v8-dev' == 'v8-dev.0' is true because these are versions, but 'v8-foo' == 'v8-foo.0' is false because they’re not versions.

Previous packaging module behaviour

As noted in pypa/packaging#774, packaging<22 and (pip<24.1) had a lenient LegacyVersion parsing fallback when a value failed to parse as a version. However when I read more of the code and tested marker evaluation, I found that the lenient version comparison was not used when evaluating markers, comparisons involving LegacyVersion always evaluated false.

And even if it were used, LegacyVersion had an epoch of -1 (vs >=0 for regular Version), so comparisons of non-versions with versions would always evaluate the non-version as smaller.

Although it wasn’t used to intuitively evaluate non-version marker expressions, the presence of this lenient version parsing logic meant that in past versions packaging (& pip & Co.) would not raise an error when evaluating a non-version-to-version expression. The current behaviour seems to be a result of removing the legacy lenient parsing, which now results in the non-version-to-version cases raising rather than always evaluating false.

What to do?

In summary:

  • The spec requires non-version to version markers to be evaluated as plain Python strings
  • packaging currently raises an error evaluating non-version to version markers
  • uv implements the spec’s behaviour
  • Comparing the order of version-like strings with regular lexicographic string ordering results in unhelpful/counter-intuitive order, like 3.9 being greater than 3.10
    • It’s not obvious to an average user under what circumstances versions will be compared with version semantics vs string semantics, and it’s also not under their control, as the logic depends on the value found in the environment

How should this situation be resolved?

  • Update packaging to match the spec and accept the ordering issues with externally-sourced environment markers such as platform_release
  • Update the spec(s) to improve handling of non-standard versions
  • Something else?

Some pragmatic temporary workarounds/mitigations have also been proposed in #774 to avoid raising an error by evaluating mark expressions lazily. Maybe we could exclude environment names like extras from version parsing to avoid failing there.

6 Likes

So it looks likes my comment was misguided. Thanks a lot for digging into this. It means (as you noted) the intuitive comparison could not be relied on in practice.

If there were no complaints about the non-intuitive comparison in pip (I’m not aware of any) and there are none in uv either, then updating packaging to match the spec is probably the right thing to do.

1 Like

No problem! I’m tempted to do some more digging/testing to check I’m not missing anything about the old behaviour.

I think I’d agree that the current rules are probably good enough in practice, as although the non-version comparisons are unintuitive, they can be made in a known context by ANDing a version comparison with a known platform type to limit the scope they’ll be applied in.

If we did want to consider a spec change, I’ve got a suggestion for how we could resolve the comparison problem.


Modify the marker expression evaluation rules to work as follows:

Marker expressions are classified as involving Python Packaging Versions, Plain Strings, or Sets of Plain Strings based on their context, and different rules apply to each type.

Python Package Version rules

Python Packaging Version expressions are always and only used for:

  • The the version specifiers following a package name (the versionspec part in the name_req grammar rule):
    • e.g in Requires-Dist: importlib-metadata (>=4.6, <5) the >=4.6, <5 expression
  • Any marker_expr grammar rule where the marker_var is one of the markers defined to be a Version (python_version, python_full_version, implementation_version)
    • e.g: in Requires-Dist: importlib-metadata (>=4.6) ; python_version <= "3.9" the python_version <= "3.9" expression

Values in these contexts are required to be valid version values, so equality and comparison always uses the normalised version semantics. If a value is not a valid version, the expression evaluates false (which is what packaging used to do with LegacyVersion). However the === (triple equals) operator continues to act as an escape hatch to compare values as plain strings, without version semantics.

Plain String rules

Values are treated as Plain Strings in marker expressions not known to be Python Packaging Versions, i.e. any of the environment markers listed as type String in the markers tableplatform_release, sys_platform, extra, etc.

Equality (==, !=) is regular str equality, even if both values are syntactically valid package versions. However ordering comparisons (<, >=, etc) compare using a general version/natural sort algorithm, so that expressions like '3.9' < '3.10' evaluate true, with no syntactic constraints on the values (see below for more details).

Sets of Plain String rules

No change, evaluated as they are in Python.

Version / Natural Sorting

Debian dpkg has version sorting algorithm used to compare package versions, where the package versions have no specific syntax other than restricting the available set of characters. GNU Coreutils ls --sort=version and sort --version-sort uses an algorithm based on the Debian dpkg one, with some adjustments to make it more general purpose and suitable for sorting filenames (e.g. treating file extensions specially).

The rules are quite straightforward, and I think we could define a slight variation of the Debian dpkg scheme or a subset of the Coreutils rules (e.g. without the special handling of file extensions, single . etc.), and use it for ordering arbitrary strings that cannot be guaranteed to be Python Packaging Versions.

Summary

  • Comparison of valid version values is unchanged in Python Packaging Version contexts
  • Non-version to version comparisons in Python Packaging Version contexts evaluate false, unless === is used.
  • Arbitrary strings use a version/natural sorting scheme to compare order

The result should be that package versions compare as before, but environment marker comparisons for things like OS versions (platform_release) compare for order consistently and intuitively.

5 Likes

Just to confirm, I did some testing on packaging 21.3 that pip used to use, I’ve got a throwaway branch in my packaging repo fork with some unit tests to demonstrate/verify how invalid versions were being evaluated: Add throwaway tests to demo 21.3 invalid version evaluation by h4l · Pull Request #3 · h4l/pypa-packaging · GitHub. This confirmed that the LegacyVersion is not used for environment marker evaluation.

I also manually tested/verified pip 24.0 behaviour on a machine with platform_release 6.10.14-linuxkit.

Manual pip 24.0 dependency requirement evaluation test
  • The httpx requirement IS installed because '6.2.0-linuxkit' is not a valid version in the right-hand side, so regular str comparison is used, and 6.1 (prefix) is less than 6.2.
  • The requests requirement IS NOT installed because '6' is a valid version on the right-hand side, so version comparison semantics are used; however 6.10.14-linuxkit is not a valid version, so it’s parsed with LegacyVersion and environment marker evaluation always returns false with LegacyVersion.
# pyproject.toml
[project]
name = "v-extra-example"
version = "1.0.0"
requires-python = ">=3.9"
dependencies = [
  "httpx;    platform_release < '6.2.0-linuxkit'",
  "requests; platform_release > '6'"
]

[build-system]
requires = ["hatchling >= 1.26"]
build-backend = "hatchling.build"
$ python -c 'import platform; print(platform.release())'
6.10.14-linuxkit

$ pip --version
pip 24.0 from /home/vscode/.local/lib/python3.13/site-packages/pip (python 3.13)

$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing /workspaces/v-extra-package-issue
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting httpx (from v-extra-example==1.0.0)
  Using cached httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)
Requirement already satisfied: anyio in /home/vscode/.local/lib/python3.13/site-packages (from httpx->v-extra-example==1.0.0) (4.9.0)
Requirement already satisfied: certifi in /home/vscode/.local/lib/python3.13/site-packages (from httpx->v-extra-example==1.0.0) (2025.7.14)
Requirement already satisfied: httpcore==1.* in /home/vscode/.local/lib/python3.13/site-packages (from httpx->v-extra-example==1.0.0) (1.0.9)
Requirement already satisfied: idna in /home/vscode/.local/lib/python3.13/site-packages (from httpx->v-extra-example==1.0.0) (3.10)
Requirement already satisfied: h11>=0.16 in /home/vscode/.local/lib/python3.13/site-packages (from httpcore==1.*->httpx->v-extra-example==1.0.0) (0.16.0)
Requirement already satisfied: sniffio>=1.1 in /home/vscode/.local/lib/python3.13/site-packages (from anyio->httpx->v-extra-example==1.0.0) (1.3.1)
Using cached httpx-0.28.1-py3-none-any.whl (73 kB)
Building wheels for collected packages: v-extra-example
  Building wheel for v-extra-example (pyproject.toml) ... done
  Created wheel for v-extra-example: filename=v_extra_example-1.0.0-py3-none-any.whl size=1017 sha256=b21739ec62a46843023fc595797573b36572a2324b82bd212807ea3d9a653cde
  Stored in directory: /home/vscode/.cache/pip/wheels/0d/7f/ab/0927a9f4d4eb3859e98c10104f2cad4c8036afeec831e5ee80
Successfully built v-extra-example
Installing collected packages: httpx, v-extra-example
  Attempting uninstall: v-extra-example
    Found existing installation: v-extra-example 1.0.0
    Uninstalling v-extra-example-1.0.0:
      Successfully uninstalled v-extra-example-1.0.0
Successfully installed httpx-0.28.1 v-extra-example-1.0.0

[notice] A new release of pip is available: 24.0 -> 25.1.1
[notice] To update, run: python3.13 -m pip install --upgrade pip

One of my comments on the original issue was meant to be exactly this complaint. I didn’t emphasize it there because at the time changing the spec didn’t seem to be a realistic option - plus, the focus of the bug report was on packaging blocking installation of certain packages, not on whether the spec made sense - but now that changing the spec is on the table, yeah, I’ll go on record (for whatever little that’s worth) that I think the behavior described by the existing spec will lead to unintuitive results or is insufficiently flexible in enough realistic situations that it’s worth trying to change it. For example, if some package has a conditional dependency when running on version 6.9 or higher of the Linux kernel, the existing spec has no clean way to express that constraint, since kernels 6.10.X up through the current 6.16.X will fail the comparison platform_release >= "6.9".

It looks like someone made the same point on uv’s version of this discussion as well.

FWIW if a change to the spec does go ahead, I like @hal’s suggestion of using version sorting semantics (as in e.g. Coreutils) for plain string fields. My original half-baked idea was to extract the longest prefix of the string that constitutes a PEP 440-compliant version number and use that for the initial comparison, but it makes more sense to stick with well-tested algorithms.

3 Likes

I should have made it more clear what the conclusion was:

In packaging packaging<22 and pip<24.1:

  1. An invalid version in a dependency specifier environment marker would mean the condition was always evaluated with plain str semantics (even if the runtime value is a valid version).
    • e.g: This would always use plain str semantics to compare the runtime platform_release value with the constant '20.7.0-gentoo'.
      Requires-Dist: typing-extensions; platform_release >= '20.7.0-gentoo'
      
  2. A valid version in a dependency specifier environment marker would mean the condition was always evaluated to be false when the runtime value was not a valid version.
    • e.g: This would always evaluate false when the runtime platform_release value is not a valid version, e.g. false when platform.release() == '20.7.0-gentoo'
      Requires-Dist: typing-extensions; platform_release >= '20'
      

I mentioned this discussion to one of the uv developers (who had previously talked about this problem) and they’ve commented here: Different understanding of environment markers vs pip · Issue #3917 · astral-sh/uv · GitHub

I wanted to quickly thank @hal for the thorough write-up!

The summary is that platform_release and platform_version should be avoided (assuming I’m summarizing @konstin appropriately).

The options I’m thinking of are:

  1. Update packaging to follow the spec and put a warning in the spec that comparisons won’t work the way users probably expect
  2. Update the spec so only == and != are allowed for string markers
  3. The spec gains the concept of external versions and we adopt Debian’s comparison algorithm for comparing those external versions

I think what we decide to do depends somewhat on how often these markers are used and what comparison is usually used (although we really only seem to be talking about platform_release). I have never used either marker nor have I seen them used, so my opinion is option 1 since it’s the easiest one to implement. But I’m very aware I personally might have a blind spot on the usage without analyzing PyPI metadata.

5 Likes

I tried searching for uses of these markers with greater/less than operators, and it seems like they are very infrequently used, at least currently:

I would have thought that conditional dependencies by platform version should be largely unnecessary if the dependency used platform compatibility tags to control platform support.

Option 1. seems like a fairly safe option, as even if the comparisons can be confusing, very few people will be confused in practice if it’s a barely used feature. On the other hand, it would be cementing the sub-optimal behaviour, which until now has only been implemented in uv.

2 Likes

Fair. It seems like a bit of a shame to me to cement sub-optimal behavior when there is an opportunity to improve the behavior, but it’s still better than having no decision at all.

1 Like

It does. I would be happy to volunteer to implement external version support if we did decide to take that approach and needed someone to work on it.

1 Like

Whoops, I didn’t see your reply when you posted it, my bad!

I don’t think there will be any problem getting an implementation coded up, once a decision is made about how to proceed. (I also would be happy to do it, as I’m sure many others would.)

So… I guess I’m wondering, where do we go from here? In other words, what’s the next step that would need to happen to move toward formally proposing a change to the spec? (Not to say that you, Hal, should necessarily know, but someone should.)

No problem! I’m indeed not sure myself, but I think we first need a decision on which of the options Brett summarises we’re going to go with.

2 Likes