Ban negative extras (!= for extras marker)

Packaging
4

It may worth expanding to “only extra == "x" is defined”.

Currently packaging.markers.Marker.evaluate returns the following for each operator on extra:

marker extra="gpu" extra="" extra="cpu"
extra == "gpu" True False False
extra != "gpu" False True True
extra <= "gpu" True False False
extra >= "gpu" True False False
extra < "gpu" False False False
extra > "gpu" False False False
extra in "gpu" True True False
extra not in "gpu" False False True
extra === "gpu" raises UndefinedComparison
extra ~= "gpu" raises UndefinedComparison

<= and >= behave as ==, < and > never match, in/not in do substring matching on the extra string (extra in "gpu" is true for the empty extra), and ~=/=== raise. Only == has a usable meaning.

!= evaluates cleanly as set non-membership, the problem is in the resolution model: whether a != dependency is present depends on the full set of extras requested for the package, which is not settled until every requirer is resolved, and activating an extra can remove a dependency.

Two consequences:

Some graphs have no valid install:

project:   Requires-Dist: cpu-only ; extra != "gpu"
           Provides-Extra: gpu
           Requires-Dist: gpu-only ; extra == "gpu"
cpu-only:  Requires-Dist: project[gpu]

cpu-only is required unless gpu is active, and cpu-only activates gpu. If cpu-only is installed, gpu is active, so project does not require it; if cpu-only is absent, gpu is inactive, so project requires it. No install satisfies both, and a resolver either errors or silently installs a wrong set (for example gpu-only with cpu-only dropped, though nothing requested the gpu extra).

Other graphs have a valid install that incremental resolution misses:

project:   Requires-Dist: plugin ; extra != "gpu"
           Requires-Dist: ancient>=2 ; extra != "docs"
           Provides-Extra: gpu, docs
           Requires-Dist: gpu-only ; extra == "gpu"
           Requires-Dist: docs-theme ; extra == "docs"
plugin:    Requires-Dist: project[docs]
ancient:   only 1.0 is published, so ancient>=2 is unsatisfiable

{project, plugin, docs-theme} is valid: plugin is required (gpu was not requested), plugin activates docs, and docs being active drops ancient>=2 ; extra != "docs". A resolver that decides the negative-extra deps before docs is activated commits to ancient>=2 and fails.

4 Likes