Spec change/bugfix: dependency specifiers simplification (PEP 508)

[henryiii (Henry Schreiner)]

(For context, this is a concrete proposal split out from Reconciling `packaging` module & The Dependency Specifier Spec's non-version to version comparison rules - #14 by henryiii)

PEP 508 has some very strange rules related to packaging markers, and packaging has never been compliant with these rules (and uv is not either). In short, the spec states any version comparison operator (like >, >=, and ==) must treat the specifier as a version, even if the table lists it as a string, trying to parse it as a version, and if it can’t parse, then using Python string comparison rules as fallback. This means implementation_name > "cpython" is valid and is supposed to evaluate False for cpython, but True for pypy. And the implementation is supposed to try to convert these to versions first, which in packaging is actually fairly expensive when done many times.

Here’s the table (minus a column for clarity here):

Marker	Type	Sample values
os_name	String	posix, java
sys_platform	String	linux, linux2, darwin, java1.8.0_51 (note that “linux” is from Python 3 and “linux2” from Python 2)
platform_machine	String	x86_64
platform_python_implementation	String	CPython, Jython
platform_release	String	3.14.1-x86_64-linode39, 14.5.0, 1.8.0_51
platform_system	String	Linux, Windows, Java
platform_version	String	#1 SMP Fri Apr 25 13:07:35 EDT 2014 Java HotSpot(TM) 64-Bit Server VM, 25.51-b03, Oracle Corporation Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64
python_version	Version	3.4, 2.7
python_full_version	Version	3.4.0, 3.5.0b1
implementation_name	String	cpython
implementation_version	Version	3.4.0, 3.5.0b1
extra	String	toml
extras	Set of strings	{"toml"}
dependency_groups	Set of strings	{"test"}

It should be noted, of these string fields, only one of them actually might look like a version: platform_release (platform.release()). On some systems, like macOS, this is a valid version. The only other one that anyone on GitHub has ever tried to use with a comparison that’s not equality is sys_platform >= "win32", which is obviously hoping that it would cover an imaginary "win64" (but "win128" would compare less!).

There is one valid use for platform_release; when you are on a system that you know has a valid PEP 440 style version, you can in theory gate it:

# scipy is not supported on Mac M1 with Mac OS < 12.0
scipy; platform_system != "Darwin" or platform_machine != "arm64" or platform_version >= "12"

In packaging<22, LegacyVersion was used, so these comparisons always returned False, and did not fall back to Python string comparison, since that’s how LegacyVersion worked. Starting in packaging 22, with the removal of LegacyVersion, these started throwing InvalidVersion errors instead (also not spec compliant). The spec does not specify if short circuit evaluation is required (since it basically has fallbacks for everything, there’s not really a point), so this means the above expression, in packaging 22-25.0 fails on any system that doesn’t have a valid PEP 440 version here, rendering it useless unless you only support the subset of systems where this does convert to a version. This was an issue adopting newer packaging versions in pip, and has basically resulted in every project that is actively maintained having to stop relying on this mechanism - less than 50 examples remain on GitHub of this. A significant number of pull requests and issues are open on packaging with various ways to fix these issues.

From some discussions with the uv team and looking at the code, uv follows the table, and does not try to convert everything to Version, though it does implement string comparisons; sys_platform == "win32" and platform_release > "9" would fail on Windows 11, for example, while it would pass on packaging - but packaging would currently crash with an error if this line was evaluated on most Linux systems.

Now we are faced with an issue: packaging 25.1 is about ready for release, and the way it handles invalid version comparisons has changed. As the code stands now, this will return False again, like it used to. Also, we’ve been really focused on performance; reading every Version on PyPI is 2x faster, and constructing/using SpecifierSet is ~3x faster (partially because we construct fewer Versions, which are costly due to needing a regex, even at 2x faster). This should improve the performance of the pip resolver, which constructs thousands of versions and specifier sets. One of the remaining areas where we are running Version on is on every dependency marker.

So there are three problems:

• The behavior is changing from an error to a non-standards complaint behavior (even though it’s the same behavior from years ago)
• We still have to try to construct versions on every single item in the above table. We are having to try to run the version regex on every marker, "cpython", "x86_64", etc.
• There are weird bugs open, like v0 can’t be used as the name of an extra, because it parses as a version. We have to take an expression like thing; extra == "v0" and parse the v0 as a version according to the spec, since == is a version comparison, the type of the field is meaningless according to the spec.

So I’d like to propose the following spec changes to align the spec with the way these have been handled since the beginning in packaging, and reduce our work required as well. It’s a minimal change; larger changes could be worked on later if someone wanted to work on a PEP for cleaning this up. But here’s my proposal:

• Change the spec to state only Version values must have the “convert to version if possible” behavior. This will allow implementations to fix errors like using "v0" as an extra, and provide a performance boost. uv is doing this anyway.
• Make platform_release a Version (could be indicated as string | Version in the table to help users realize it will often fail the conversion, but it keeps the legacy Version behavior above.)
• Define > and < as always False, and <=, >= as equivalent to ==, for strings and failed Version conversions. This is the legacy (<22) and current (in main) behavior of packaging. Python string ordering is never reliable; even if it happens to work going from 8 to 9, it will break on the next release because 9 is more than 10. And this requires that other languages, like Rust, follow Python’s rules for string ordering.

This should only affect <50 (legacy) packages on GitHub, and it will do the right thing for them as well (making the packaging <22 behavior official). (Most of these have other typos, like 21 instead of 12 for the macOS version, so pretty sure they are dead projects, but it won’t break them).

I’d like to do this now, since we are replacing an Error with our old behavior, which was not spec compliant, so packages may start appearing expecting this behavior (again).

(I’d also be fine if we kept the string comparison (drop bullet 3 above), and changed packaging to support that; there has been pushback since this isn’t reliable even for version-like values, and it’s viewed as easier to change a False to a True than the other way around, due to the asymmetry in markers not supporting not, and historically it has returned False here. I’ll quite Konsti from the uv discussion here, hopefully he doesn’t mind: “this version-to-string-comparion fallback behavior is a big footgun and i wholeheartedly endorse removing it”.)

[brettcannon]

@henryiii

Change the spec to state only Version values must have the “convert to version if possible” behavior. This will allow implementations to fix errors like using "v0" as an extra, and provide a performance boost. uv is doing this anyway.

+1

@henryiii

Make platform_release a Version (could be indicated as string | Version in the table to help users realize it will often fail the conversion, but it keeps the legacy Version behavior above.)

+0

@henryiii

Define > and < as always False, and <=, >= as equivalent to ==, for strings and failed Version conversions. This is the legacy (<22) and current (in main) behavior of packaging. Python string ordering is never reliable; even if it happens to work going from 8 to 9, it will break on the next release because 9 is more than 10. And this requires that other languages, like Rust, follow Python’s rules for string ordering.

+0

… and more characters to be > 10.

[pf_moore (Paul Moore)]

I’m OK with this. Basically my views are the same as Brett’s.

Technically, this change would affect interoperability, so it needs approval to be implemented as a PR to the spec, without a PEP. It’s not clear in the process whether “approval” means “consensus on Discourse” or if the PEP-delegate’s OK is sufficient. I’m willing to give my approval, but I’d like to see at least some level of consensus here that people are OK with the change, before it gets implemented.

[encukou (Petr Viktorin)]

@henryiii

Define > and < as always False, and <=, >= as equivalent to ==, for strings and failed Version conversions.

Please have the standard clearly say that this is legacy behaviour.

Tools that write/maintain/lint specifiers should be allowed to emit warnings, fail, or update to the modern equivalents – as appropriate for the tool and its backcompat policy.
Also, an install tool for a “controlled environment” (e.g. monorepo, distro) can simply fail and tell the user to patch their package.

[henryiii (Henry Schreiner)]

I’ve collected every marker on PyPI, and found every one that does not end in _version and uses one of the angle brackets. Here’s every single one that doesn’t also end in _release:

RPi.GPIO (>=0.6.3) ; platform_machine >= "armv0l" and platform_machine <= "armv9l"
RPi.GPIO (>=0.6.3); platform_machine >= "armv0l" and platform_machine <= "armv9l"
RPi.GPIO (>=0.7.0) ; platform_machine >= "armv0l" and platform_machine <= "armv9l"
brotli (>=1.0.9) ; platform_python_implementation >= "CPython"
brotli >=1.0.9 ; platform_python_implementation >= "CPython"
picamera (>=1.13) ; platform_machine >= "armv0l" and platform_machine <= "armv9l"
picamera (>=1.13); platform_machine >= "armv0l" and platform_machine <= "armv9l"
win32-setctime>=1.2.0; sys_platform >= "win32"

This proposal could affect those packages (if those are the current releases of maintained packages!); platform_machine >= "armv0l" and platform_machine <= "armv9l" would be the only one that was doing something reasonable; the other two don’t make sense and should have been ==. The armv*l ones are all from one package, pibooth, last released 2023 and not touched in a year.

The list with platform_release is much longer, and dominated by lines that can be summarized as this:

pyobjc-framework-MetalKit==10.3.2; platform_release >= "15.0"

With lots of different frameworks; I think most of these come from a fairly small set of packages that are macOS only. Some of the platform numbers don’t make sense, I see a range a values 10-21, but AFAICT this is the macOS version number, which skipped from 15 to 26, not sure why someone would set 16-21 as values here. There are about 20 of these (times dozens of frameworks), I believe they are mostly from three packages, akande 0.0.2-0.0.4, py3-tts-wrapper, and pyobjc. I’ve removed most of them, leaving only a couple of examples; otherwise this is every single other platform_release comparison:

PyChakra (>=2.2.0) ; (os_name == "nt" and platform_release < "8") and extra == 'all'
PyChakra (>=2.2.0) ; (os_name == "nt" and platform_release < "8") and extra == 'js'
TensorFlow-MacOS (>=2.12.0) ; sys_platform == "darwin" and platform_release > "20.6.0"
TensorFlow-Metal (>=1.0.0) ; sys_platform == "darwin" and platform_release > "20.6.0"
cgmetadata>=0.1.6; sys_platform == "darwin" and platform_release >= "20.0.0"
makelive>=0.5.0; sys_platform == "darwin" and platform_release >= "20.0.0"
psycopg[binary]==3.1.19; sys_platform == "darwin" and platform_machine == "arm64" and platform_release < "23.0"
psycopg[binary]==3.2.1; (sys_platform == "darwin" and platform_machine == "arm64" and platform_release >= "23.0") or (sys_platform == "darwin" and platform_machine != "arm64") or sys_platform != "darwin"
psycopg[binary]==3.2.3; (sys_platform == "darwin" and platform_machine == "arm64" and platform_release >= "23.0") or (sys_platform == "darwin" and platform_machine != "arm64") or sys_platform != "darwin"
pyarrow-stubs>=17.10; (platform_release >= "20" and extra == "typing") or (sys_platform != "darwin" and extra == "typing")
pyarrow-stubs>=17.10; (sys_platform != "darwin" or platform_release >= "20") and extra == "typing"
pyarrow<=14.0.2; platform_release < "19" and sys_platform == "darwin"
pyarrow<=14.0.2; sys_platform == "darwin" and platform_release < "19"
pyarrow<=16.0.0; platform_release < "20" and sys_platform == "darwin"
pyarrow<=16.0.0; sys_platform == "darwin" and platform_release < "20"
pyarrow<=17.0.0; platform_release < "21" and sys_platform == "darwin"
pyarrow<=17.0.0; sys_platform == "darwin" and platform_release < "21"
pyobjc-core <10.0,>=9.0 ; sys_platform == "darwin" and platform_release < "22.0"
pyobjc-core <11.0,>=9.0 ; sys_platform == "darwin" and platform_release >= "22.0"
pyobjc-core<10.0,>=9.0; sys_platform == "darwin" and platform_release < "22.0"
pyobjc-core<11.0,>=9.0; sys_platform == "darwin" and platform_release >= "22.0"
pyobjc-core>=9.0; sys_platform == "darwin" and platform_release >= "22.0"
pyobjc-framework-ARKit==12.0; platform_release >= "25.0"
pyobjc-framework-ARKit==12.1; platform_release >= "25.0"
pyobjc-framework-AVFoundation <11.0,>=9.0 ; sys_platform == "darwin" and platform_release >= "22.0"
pyobjc-framework-pubsub ==10.1 ; platform_release >= "9.0" and platform_release < "18.0" and python_version >= "3.9" and python_version < "4.0" and platform_system == "Darwin"

The proposal here would not break any of those packages, since we are keeping the old behavior for platform_release. (Technically, it will fix them in packaging, since these currently are broken on systems without version like numbers, and if uv picks up this behavior, it would help them there as well.)

scripts

import ast
import sqlite3

# connect to your database
conn = sqlite3.connect("pypi-data.sqlite")
cur = conn.cursor()

# run the query
cur.execute("""
    SELECT requires_dist
    FROM projects
    WHERE requires_dist LIKE '%;%'
""")

values = {val for (value,) in cur for val in ast.literal_eval(value)}

# write each value to a file, exactly as-is
with open("requires_dist.txt", "w", encoding="utf-8") as f:
    for value in values:
        f.write(value + "\n")

import re

# Regex to capture markers with comparison operators
pattern = re.compile(r"\b([a-zA-Z0-9_]+)\s*(>=|<=|>|<)\s*['\"]?[^'\"\s]+['\"]?")

def find_non_version_markers(line):
    marker = line.split(";", 1)[1].strip()
    if match := pattern.search(marker):
        # if not match.group(1).endswith(("_version", "_release")):
        if match.group(1).endswith("_release"):
            return True
    return False

found = []
with open("requires_dist.txt", encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if ";" in line and find_non_version_markers(line):
            found.append(line)

print(*found, sep="\n")

[henryiii (Henry Schreiner)]

Opened a draft PR with (hopefully!) the suggested changes above, so people can start reviewing it. fix(markers): Fix/change dependency specifiers by henryiii · Pull Request #1971 · pypa/packaging.python.org · GitHub

Will take it out of draft when the discussion is done here.

[notatallshaw (Damian Shaw)]

I am supportive of this and all proposed changes.

Pip users who have encountered this problem, after upgrading to 25.0+, have had no clear path forward for a use case that has historically worked, and the discussion has been stuck on packaging side because implementing the spec as written produces bad behavior.

Being able to move forward and tell users both how it should and does work will be a big win.

[konstin (konsti)]

From my perspective from uv implementation of PEP 508

1. Remove the behavior that we first need to try comparing strings as a versions.
   Strong support (uv does this already, and there are no user complaints I’m aware of).
2. Make platform_release a Version:
   In general, I think this is an improvement.
   More broadly, this API is still dangerous as platform.releases() may or may not have a PEP 440 compatible version on a platform, which ones they are is not documented, and this may over releases of the platform. It’s also not clear what granularity or kind of version the API returns. E.g. on my Windows box it’s 11 and doesn’t mention 25H2, while on my linux machine it’s 6.8.0-88-generic and not 24.04. These are reasonable values, but there’s a lack of documentation on both the value itself and it’s future compatibility. For context, the std docs say: “Returns the system’s release, e.g. ‘2.2.0’ or ‘NT’. An empty string is returned if the value cannot be determined.”
   I know they are cases where users need to check the macOS, and there clearly needs to be a way to check the macOS version, but the field as it currently works is something I’m discouraging users from using at all. If we do the work to change the field, I’d at least like to include a big warning about it on PUG.
3. Define > and < as always False, and <=, >= as equivalent to ==, for strings and failed Version conversions.
   This also an improvement. Should we go even further and treat all of >. <, <= and >= as false? Limiting the set of supported operators is simpler to explain. FWIW I’m not worried about string ordering differences between Python and Rust, but string order comparison is something that’s more confusing than helpful.

Thank you very much for doing the PyPI investigation and sharing the script! This increases the confidence in this proposal a lot

I’m still not sure about the adoption story in uv. This is clearly a breaking change, so likely we’ll have to wait until the next breaking release (0.10.0), but I’ll talk to the rest of the team about that.

[henryiii (Henry Schreiner)]

@konstin

Should we go even further and treat all of >. <, <= and >= as false?

I think you are asking if we should make >= and <= compare False even if they are equal? I chose this to minimize the effect and to do something reasonable; I think sys_platform >= "win32" should either be True or raise an error (and with Petr’s suggestion above, it’s now allowed for an implementation warn or even throw an error here). Making these always False would not affect much, though (and unlike throwing an error, marking something with False can be worked around by just adding whatever the package was trying to add manually). I think uv could error here, and packaging might be able to in the future, too.

[henryiii (Henry Schreiner)]

In the above scripts, I missed that platform_version has a _version specifier, even though it doesn’t look like a Version at all. I tend to forget about it as it’s basically useless, it’s just a human readable string most of the time, and can’t really be used for anything in a marker. But out of millions of uploads, I’m sure it gets used. I checked, and there are 9 packages on PyPI that have ever used platform_version (only 4 current ones).

Unaffected:

• nanosurf: platform_version != "Nanosurf_Linux" - this is fine. Might be the only valid use on PyPI.
• inteltk: platform_version == "Windows" - Weird, but not affected. Pretty sure this is never True.
• flet: <long expression> "embedded" not in platform_version Not sure if this works, but unaffected by this change - and it was removed in flet 0.26 (current version is 0.28)
• toolio: platform_version == "arm64" Was in 0.5.1, was changed to platform_machine in 0.5.2, current version is 0.6.0

I think these probably fail on pip 25+, and would become false. Not sure they work as expected, anyway, the version might not start with a number, and string comparison doesn’t work on version when they add a digit:

• ouster-mapping and ouster-sdk: python_version <= "3.11" and platform_machine != "aarch64" and (platform_system != "Darwin" or platform_machine != "arm64" or platform_version >= "21.0.0")
• console: "colorama; os_name == "nt" and platform_version < "10.0.10586"
• global-logger: same as above, but removed in current release (was in 0.4.5, removed in current 0.4.7)
• out: same as above, but removed in current release (was in 0.80, removed in current 0.81)

So the only affected package is ouster-mapping / ouster-sdk. That’s not working like the authors intended, though; the string does not start with a number (mine is "Darwin Kernel Version 25.1.0: Mon Oct 20 19:32:47 PDT 2025; root:xnu-12377.41.6~2/RELEASE_ARM64_T8103"). Likely also broken with pip 25+, the other few packages using this removed it shortly after the release of Pip 25. I also have no idea what “21” is supposed to mean here, macOS jumped from 15 to 26. Ahh, Darwin kernel version.

Also, by “affected”, I mean the standard behavior is changing; in packaging<22/pip<25, I believe these were treated as False, just like we are recommending now. And packaging 22+/pip 25+ throw an error, just like we are allowing now.

Script

# /// script
# dependencies = ["packaging"]
# ///

import sqlite3
from packaging.version import Version

con = sqlite3.connect("pypi-data.sqlite")
cur = con.cursor()

rows = cur.execute("""
    SELECT name, version, requires_dist
    FROM projects
    WHERE requires_dist LIKE '%platform_version%'
""").fetchall()

best = {}  # name -> (name, version, requires_dist)

for name, version, requires_dist in rows:
    if name not in best or Version(version) > Version(best[name][1]):
        best[name] = (name, version, requires_dist)

# Convert dict → list for use
results = list(best.values())

for r in results:
    print(*r)
    print()

[ncoghlan (Alyssa Coghlan)]

I missed the thread when initially posted, but belatedly saw the PR at fix(markers): Fix/change dependency specifiers by henryiii · Pull Request #1971 · pypa/packaging.python.org · GitHub

I agree with the intent of the change, but the technical details need to be tightened up, as == version matching is also only valid for actual version specifiers (due to things like zero padding, prefix matching, etc). This means that when nominal versions aren’t actually valid versions, the fallback from all of ==, != ~=, >= and <= should be to === (negated in the != case), with only < and > unaffected (and necessarily returning false).

We may also need to tighten up the definition of === further - it was cleaned up last year to be explicitly case insensitive, but remains silent on the question of how leading and trailing whitespace should be handled.

There are also two paths to handling the clarification of how non-version fields should be compared:

• comparison is defined in terms of === arbitrary equivalence
• comparison is defined in terms of a regular Python string comparison operation

The former approach can be consistently applied to all of string-only fields, string-or-version fields, and nominally version fields that fail parsing as versions.

The latter approach is potentially viable (and more closely matches the status quo), but creates ambiguity around case sensitivity for string-or-version fields.

Edit: After posting, I also considered whether we could just say that == in environment markers never means version matching and is always a regular Python string comparison. However, that doesn’t work, as we genuinely want python_version and python_full_version (and, to a lesser degree, implementation_version and even platform_version), to follow the same rules for things like zero padding as dependency version specifiers do)

[henryiii (Henry Schreiner)]

For a string field, == and != are string comparisons, they are not version comparisons. There are used by hundreds of thousands of releases on PyPI, we can’t change that. === is a special legacy comparison defined for LegacyVersion support. In the discussions about it last year, I think the main takeaway is that it was so rarely used/needed that people didn’t really care about how it was defined, so something was selected. You can’t upload a pair of versions to PyPI that can only be differentiated by === and haven’t been able to for years. 0.017% of releases on PyPI use ===, and AFAICT most of them could use ==, they look like typos. Maybe they are intentionally avoiding a post release?

If we did define everything in terms of ===, then one could assume that the “correct” (best, preferred, whatever) way to denote string metadata would be to use that instead of ==, so all 2.3 million releases with == should be updated to use === instead.

I think taking === and defining everything else in terms of it is not a good idea, and especially think taking the semantics of === and trying to apply them to == for strings is even worse. You don’t want to take something that is case sensitive and make it case insensitive, especially when no one is asking for it - case insensitive comparisons are more expensive and more complex (how to handle unicode, etc).

In this case, I would consider three tiers:

• == and =! are the core comparisons, and they are defined as Version, or case sensitive string comparisons.
• >=, <=, ~= are ordered Version comparisons, but for backward compatibility they are allowed to be defined as == for strings. > and < are also ordered Version comparisons, but for backward compatibility they are allowed to be defined as false for strings. Implementations do not have to include the back-compat feature.
• === is a legacy Version arbitrary equivalence, and is not defined on string fields; it only applies to Version fields (including the string fallback).

If you really wanted to, === could be included in the backward compatibility fallbacks, but from quickly looking over the results of ===, I don’t see it used for anything other than version comparisons. Allowing it on string fields would mean that users could do === to get a case insensitive comparison, but I don’t think we need/want that and there’s no negative version !==, and we don’t allow arbitrary “not” so you can’t build one.

By the way, platform_node is a example were you very much would not want == to be case insensitive on strings, as you could in theory have two Linux nodes with names differing only by their capitalization.

[ronaldoussoren]

@henryiii

The list with platform_release is much longer, and dominated by lines that can be summarized as this:

pyobjc-framework-MetalKit==10.3.2; platform_release >= "15.0"

This looks like one of the dependency specifiers in the pyobjc package, which is macOS only. The version numbers at the end are platform.release() values and not macOS releases. Luckily there’s a mapping between the two.

I use this to ensure that only relevant framework bindings are installed when you install the pyobjc package.

[henryiii (Henry Schreiner)]

Yes, those are mostly from pyobjc, and it works only because this cannot be used outside of macOS, where it would throw an error currently. With the change here, it would be possible to use it outside of macOS if you protected it with a platform check.

(Edit: the marker, not the package. )

[henryiii (Henry Schreiner)]

packaging 26.0rc1 is now out (Announcement: packaging 26.0rc1 released!) including this change. Final release planned in a week if nothing comes up (Jan 16th, 2026).

[ncoghlan (Alyssa Coghlan)]

@henryiii

In this case, I would consider three tiers:

• == and =! are the core comparisons, and they are defined as Version, or case sensitive string comparisons.
• >=, <=, ~= are ordered Version comparisons, but for backward compatibility they are allowed to be defined as == for strings. > and < are also ordered Version comparisons, but for backward compatibility they are allowed to be defined as false for strings. Implementations do not have to include the back-compat feature.
• === is a legacy Version arbitrary equivalence, and is not defined on string fields; it only applies to Version fields (including the string fallback).

Aye, I think that makes sense. I think the environment markers description could benefit from a larger restructuring to make this clearer, so I’m going to make an alternate PR that rearranges things a bit more than the currently open one does (but still specifies what you have described here). The goal now is to accurately and clearly describe how packaging 25.1 works, so I think the release itself is unblocked even if we take a bit more time to get this wording right.