Why is the * needed at all?
Why isn’t >= 1.0 usable now?
I’d prefer we don’t relax this restriction. No really strong reason, just that it’s straightforward to fix the issue by removing the .*, and changing this would be a PEP change, which is a pain (and I don’t particularly want to manage the necessary discussion and approval of such a change).
Let’s reframe the question, though. Is this important enough that we want to require every other implementation of PEP 440 to make this change? IMO, it’s not.
2 Likes
Well, yes but what about existing releases which have >= 1.0.*? It’s non-trivial to change that.
I never said it isn’t. I’m saying that people have been using the .* form already, and for… historical reasons… we’ve had this be the logic for parsing requirements and versions:
>>> from packaging.version import parse as parse_version
>>> from packaging.requirements import Requirement
>>> Requirement("name >= 48752389.ty89324ut589438vm29ct3c9j4m")
<Requirement('name>=48752389.ty89324ut589438vm29ct3c9j4m')>
>>> parse_version("89573498t298gh23gc54892mx34th9849 u2523r24hgm198t t8914hr9t1 4cf8019h 0th rf9084h02 n0r58r03rh`n98g")
<LegacyVersion('89573498t298gh23gc54892mx34th9849 u2523r24hgm198t t8914hr9t1 4cf8019h 0th rf9084h02 n0r58r03rh`n98g')>
Understood, but arguably that’s no different to any other bug in a dependency specification. It only works accidentally at the moment, and only for consumers using packaging (which agreed, is likely the vast majority of them).
But as I said, I’m not that bothered. If you want to raise a request for the PEP to be changed, go for it. You’ll need to follow the process here and in particular I’d consider it as something that “has the potential to affect software interoperability” - anything that implements its own version comparison rather than using packaging will be impacted[1]. Having said that, I don’t expect it to be controversial - just tedious 
-
And the whole point of standardising is so that we don’t assume that everyone uses the canonical library. ↩︎
I’m not too concerned about the process or its tediousness TBH – I first want to see if others think this is meaninfully useful to allow. 
Personally, I’m ambivalent but, if enough people say that “yes, I think this is worth allowing”, I’d be happy to drive this.
Do we have a way to check how many recent releases on PyPI use the now invalid syntax? Is there a way to check this other than downloading all releases from PyPI and inspecting them?
I did a quick query of the JSON data on PyPI (I have an offline copy). It’s not going to be entirely accurate, and my check was extremely superficial, but I found 464 projects on PyPI with metadata that might be impacted. That’s not a huge number - and 120 of those appear to be part of an extra, so are not used by default.
List of problem values
Project name Requirement
aiocogeo stac-pydantic>=1.3.*
aioflowdock aiohttp>=3.*
aioflowdock pyee>=6.*
aiohubot pyee<7,>=6.*
aiohubot aiohttp<4,>=3.*
airflow-ditto apache-airflow<=2.*,>=1.10.10
airflow-hdinsight apache-airflow<=2.*,>=1.10.10
airium requests<3,>=2.12.*; extra == "parse"
airium beautifulsoup4<5.0,>=4.10.*; extra == "parse"
amnes flake8-bugbear<21,>=20.*; extra == "dev"
amnes mkdocs-material<5.1,>=5.0.*; extra == "dev"
animask opencv-python>=4.1.0.*
animask pygifsicle>=1.0.*
apdaemon psutil>=5.4.*
apispec-starlette apispec<=4.*,>=3.3.*
astrool astropy<4.*,>=3.0
astrool plotly<4.*,>=3.0
autogenes matplotlib>=3.0.*
autots pandas>=0.25.*
autots statsmodels>=0.10.*
autots scikit-learn>=0.20.*
autots fbprophet>=0.4.*; extra == "additional"
bdacore sphinx>=1.5.*; extra == "test"
bdacore sphinx-bootstrap-theme>=0.4.*; extra == "test"
bdacore nbsphinx>=0.2.*; extra == "test"
bdacore sphinx-gallery>=0.1.*; extra == "test"
binderhub kubernetes>=4.*
blog PyYAML>=5.1.*
blog Jinja2>=2.10.*
blog Markdown>=3.1.*
blog Pygments>=2.4.*
blog pymdown-extensions>=6.*
blog pyembed-markdown>=1.1.*
blog docopt>=0.6.*
boml tensorflow<=1.15.*,>=1.13.*
chooser wxPython>=4.0.*
codecomb numpy>=1.16.*
coreir hwtypes>=1.0.*
dagster-gcp google-cloud-bigquery<3,>=1.19.*
dantro sphinx-book-theme>=0.3.*; extra == "dev"
dantro sphinx-book-theme>=0.3.*; extra == "doc"
deriver pandas>=1.0.*
dgenies matplotlib>=2.1.*
dgenies Markdown>=2.6.*
dgenies python-crontab==2.*,>=2.2.*
directory-healthcheck django<3.3.*,>=1.11.22
django-calaccess-processed-data django>=4.0.*
django-calaccess-raw-data django>=4.0.*
django-calaccess-raw-data django-postgres-copy>=2.6.*
django-calaccess-scraped-data django>=3.2.*
django-gcloud-connectors django<=3.3.*,>=2.0
django-on-chain django<=4.0.*,>=2.2.*
django-saml-idp Django>=1.9.*
dropbox stone>=2.*
eapt pygeoip>=0.*
eapt pycurl>=7.*.*.*
eapt haversine>=2.*.*
evolutionary-keras tensorflow>2.1*
excel2xx mako>=1.1.*
fan-tools rest-framework-dyn-serializer>=1.3.*; extra == "doc_utils"
fastapi-plugins fastapi>=0.74.*
fastapi-plugins tenacity>=8.0.*
fastapi-plugins python-json-logger>=2.0.*
fastapi-plugins redis>=4.3.*
fastapi-plugins aiojobs>=1.0.*
fastapi-plugins fastapi>=0.74.*; extra == "all"
fastapi-plugins tenacity>=8.0.*; extra == "all"
fastapi-plugins python-json-logger>=2.0.*; extra == "all"
fastapi-plugins redis>=4.3.*; extra == "all"
fastapi-plugins aiojobs>=1.0.*; extra == "all"
fastapi-plugins aiomcache>=0.7.*; extra == "all"
fastapi-plugins fastapi>=0.74.*; extra == "fakeredis"
fastapi-plugins tenacity>=8.0.*; extra == "fakeredis"
fastapi-plugins python-json-logger>=2.0.*; extra == "fakeredis"
fastapi-plugins redis>=4.3.*; extra == "fakeredis"
fastapi-plugins aiojobs>=1.0.*; extra == "fakeredis"
fastapi-plugins fakeredis>=1.8.*; extra == "fakeredis"
fastapi-plugins fastapi>=0.74.*; extra == "memcached"
fastapi-plugins tenacity>=8.0.*; extra == "memcached"
fastapi-plugins python-json-logger>=2.0.*; extra == "memcached"
fastapi-plugins redis>=4.3.*; extra == "memcached"
fastapi-plugins aiojobs>=1.0.*; extra == "memcached"
fastapi-plugins aiomcache>=0.7.*; extra == "memcached"
fastapi-plugins fastapi>=0.74.*; extra == "test"
fastapi-plugins tenacity>=8.0.*; extra == "test"
fastapi-plugins python-json-logger>=2.0.*; extra == "test"
fastapi-plugins redis>=4.3.*; extra == "test"
fastapi-plugins aiojobs>=1.0.*; extra == "test"
fastapi-plugins aiomcache>=0.7.*; extra == "test"
fastapi-plugins fakeredis>=1.8.*; extra == "test"
feast googleapis-common-protos<2,>=1.52.*
feast PyYAML<7,>=5.4.*
feast dask<2022.02.0,>=2021.*
feast google-cloud-datastore<3,>=2.1.*; extra == "ci"
feast google-cloud-storage<3,>=1.34.*; extra == "ci"
feast google-cloud-datastore<3,>=2.1.*; extra == "dev"
feast google-cloud-storage<3,>=1.34.*; extra == "dev"
feast google-cloud-datastore<3,>=2.1.*; extra == "docs"
feast google-cloud-storage<3,>=1.34.*; extra == "docs"
feast google-cloud-datastore<3,>=2.1.*; extra == "gcp"
feast google-cloud-storage<3,>=1.34.*; extra == "gcp"
firelab tb-nightly>=1.14.*
flange jsonschema>=3.*
fsleyes jinja2>=2.*
fsleyes-props fsleyes-widgets>=0.6.*
fslgui nibabel>=2.3.*
gitir-download pytest<=4.*; extra == "dev"
gluonts rpy2<3.*,>=2.9.*; extra == "r"
gluonts pyarrow>=6.*; python_version == "3.6.*" and extra == "arrow"
gluonts pyarrow>=6.*; python_version == "3.6.*" and extra == "dev"
gluonts pyarrow>=6.*; python_version == "3.6.*" and extra == "pro"
good-first-issues pytest<=4.*; extra == "dev"
hepdata-lib PyYAML>4.*
hicexplorer numpy>=1.15.*
hicexplorer scipy>=1.1.*
hicexplorer matplotlib>2.2.*
hicexplorer tables>=3.4.*
hicexplorer pandas>=0.23.*
hicexplorer pyBigWig>=0.3.*
hicexplorer six>=1.11.*
hicexplorer future>=0.17.*
hicexplorer jinja2>=2.10.*
hicexplorer unidecode>=1.0.*
hicmatrix numpy>=1.16.*
hicmatrix scipy>=1.2.*
hicmatrix tables>=3.5.*
hicmatrix pandas>=0.25.*
hicmatrix intervaltree>=3.0.*
httpx-ntlm httpx>=0.21.*
i2t2 fastscript>=0.1.*
i2t2 ipykernel>=5.2.*
i2t2 ipython>=7.13.*
i2t2 jupyter>=1.0.*
i2t2 jupyterlab>=2.0.*
i2t2 matplotlib>=3.2.*
i2t2 nbdev>=0.2.*
i2t2 numpy>=1.18.*
i2t2 pandas>=1.0.*
i2t2 pillow>=7.1.*
i2t2 pydicom>=1.4.*
i2t2 twine>=3.1.*
i2t2 scipy>=1.4.*
i2t2 h5py>=2.10.*
jhub-authenticators jupyterhub>=3.*
jhub-swarmspawner docker>=5.0.*
jhub-swarmspawner jupyterhub>=3.*
jhub-swarmspawner flatten-dict>=0.4.*
jhub-swarmspawner jhub-authenticators>=0.3.*
jhub-swarmspawner traitlets>=5.3.*
json2python-models python-dateutil>=2.7.*
json2python-models inflection>=0.3.*
json2python-models unidecode>=1.0.*
json2python-models Jinja2>=2.10.*
json2python-models typing-extensions>=3.1.*
ldap-hooks ldap3>=2.5.*
ldap-hooks traitlets>=5.3.*
ldap-hooks gen>=0.*
ldap-hooks tornado>=6.0.*
lemay-ai-sidecar pandas>=0.24.*
lemay-ai-sidecar tensorflow<1.14.*,>=1.11.*
lemay-ai-sidecar numpy>=1.16.*
lusidtools lusid-sdk-preview>=0.11.*
maccabistats setuptools>=28.*
matmov pandas<2.*,>=1.1.2
matmov numpy<2.*,>=1.19.2
matmov matplotlib<4.*,>=3.3.2
mieaa requests>=2.19.*
mip gurobipy>=8.*; extra == "gurobi"
mlx-jira-juggler python-dateutil<3.*,>=2.8.0
mlx-jira-juggler natsort<8.*,>=7.1.0
mlx-traceability Sphinx<6.*,>=2.1
mlx-traceability matplotlib<4.*
mlx-warnings junitparser<2.*,>=1.0.0
nkocr opencv-contrib-python>=4.*
node-conformity numpy>=1.15.*
node-conformity tqdm>=4.20.*
nomeroff-net numpy>=1.16.*
nomeroff-net imgaug>=0.4.*
ocelot-materials numpy>=1.*
ocelot-materials pandas>=0.24.*
ocelot-materials scipy>=1.*
ocelot-molecular numpy>=1.*
ocelot-molecular pandas>=0.24.*
ocelot-molecular scipy>=1.*
ocelot-quantum numpy>=1.*
ocelot-quantum pandas>=0.24.*
ocelot-quantum scipy>=1.*
openergy python-slugify<2.*,>=1.2.1
orange3-imageanalytics numpy>=1.16.*
orange3-imageanalytics-dda numpy>=1.16.*
pjlsa cmmnbuild-dep-manager<3.*,>=2.5.0
pjlsa jpype1<2.*,>=1.0.2
pjlsa cmmnbuild-dep-manager<3.*,>=2.5.0; extra == "all"
pjlsa jpype1<2.*,>=1.0.2; extra == "all"
pjlsa cmmnbuild-dep-manager<3.*,>=2.5.0; extra == "core"
pjlsa jpype1<2.*,>=1.0.2; extra == "core"
pretty-format-json data-process>=0.3.*
prospecting uritemplate>=3.*
pulsar-client protobuf<=3.20.*,>=3.6.1; extra == "all"
pulsar-client protobuf<=3.20.*,>=3.6.1; extra == "functions"
pyawsutils pytrustplatform>=0.13.*
pyddapi six>=1.10.*
pyddapi psycopg2-binary>=2.6.*
pyddapi nbconvert<=5.4.*; extra == "test"
pyddapi sphinx>=1.5.*; extra == "test"
pyddapi sphinx-bootstrap-theme>=0.4.*; extra == "test"
pyddapi nbsphinx>=0.2.*; extra == "test"
pyddapi sphinx-gallery>=0.1.*; extra == "test"
pygeostat matplotlib>=2.2.*
pylabrad pyOpenSSL>=16.2.*
pylabrad pyparsing>=2.1.*
pylabrad twisted>=16.0.*
pylabrad futures>=3.0.*; python_version < "3"
pylabrad numpy>=1.12.*; python_version < "3.7"
pylabrad numpy>=1.15.*; python_version == "3.7"
pylabrad numpy>=1.17.*; python_version >= "3.8"
pyquant-ms scipy>=0.18.*
pytest-httpx pytest<8.*,>=6.*
pythainlp nltk>=3.3.*; extra == "full"
pythainlp nltk>=3.3.*; extra == "wordnet"
pytorch-lightning torch>=1.9.*
pytorch-lightning torchtext>=0.10.*; extra == "all"
pytorch-lightning torchvision>=0.10.*; extra == "all"
pytorch-lightning torchtext>=0.10.*; extra == "dev"
pytorch-lightning torchvision>=0.10.*; extra == "examples"
pytorch-lightning torchtext>=0.10.*; extra == "extra"
pytrustplatform pykitcommander>=2.3.2*
receptivefield pillow>=6.2.*
receptivefield matplotlib>=3.1.*
receptivefield numpy>=1.17.*
redexpect redssh>=2*
redexpect redssh>=2*; extra == "docs"
redexpect redssh>=2*; extra == "tests"
robotframework-jenkins python-jenkins>=1.*
robotframework-jenkins robotframework>=3.*
robotframework-jenkins requests>=2.*
s3like marshmallow>=3.*
stonky teletype>=1.*
stwfsapy scikit-learn>=0.24.*
superai protobuf<4.*,>=3.20.1; extra == "ai"
superai protobuf<4.*,>=3.20.1; extra == "complete"
superai protobuf<4.*,>=3.20.1; extra == "test"
telluric fiona<2.*,>=1.8.4
telluric shapely<2.*,>=1.6.3
teroshdl yowasp-yosys>=0.8.*
textract extract-msg<=0.29.*
thonny-microbit thonny<3.2.*,>=3.0.0
thonny-pi thonny<3.2.*,>=3.0.0
uplink-httpx httpx>=0.12.*
uplink-httpx uplink>=0.9.*
wfuzz pyparsing>=2.4*; python_version >= "3.5"
xkcd-wrapper requests<2.25.*,>=2.23.*; extra == "sync"
mistex mistune>=2.0.*
taktile-profiling numpy<2,>=1.21.*
fedot ete3>=3.1.*
fedot numpy>=1.19.*
fedot pytest>=6.2.*
fedot anytree>=2.8.*
fedot typing>=3.7.*
fedot Pillow>=8.2.*
fedot seaborn>=0.9.*
fedot joblib>=0.17.*
fedot SALib>=1.3.*
fedot deap>=1.3.*
fedot lightgbm>=2.3.*
fedot catboost>=0.25.*
fedot testfixtures>=6.18.*
fedot requests>=2.*
flask-requestpreprocessor Flask>=1.1.*
blizz nbsphinx>=0.7.*; extra == "docs"
blizz pytest>=6.2.*; extra == "testing"
blizz pytest-cov>=2.10.*; extra == "testing"
blizz flake8>=3.8.*; extra == "testing"
blizz flake8-comprehensions>=3.3.*; extra == "testing"
blizz isort>=5.7.*; extra == "testing"
lazuli protobuf<4.0.*,>=3.2.*
geotiff zarr>=2.10.*
insolver h2o>=3.36.0.*
insolver django>=3.2.*; python_version >= "3.8"
libpyka beautifulsoup4>=4.9.*
libpyka chardet>=4.*
libpyka cssselect>=1.*
libpyka requests>=2.*
libpyka types-chardet>=0.*
libpyka types-requests>=0.1.*
lmdirect authlib<1.*,>=0.15.5
reqwrapper requests>=2.*
endeless numpy>=1.17.*
fastface pytorch-lightning>=1.1.*
fastface scipy>=1.*.*
mlflow-openshift mlflow>=1.13.*
mlflow-openshift numpy>=1.19.*
mlflow-openshift openshift-client>=1.0.*
pychu pydantic<2.*,>=1.7.3
xgbse pandas>=1.0.*
xgbse pandas>=1.0.*; extra == "all"
kelvin-app pyarrow>=3.0.*; extra == "tests"
bluewhale3-imageanalytics numpy>=1.16.*
iotprovision pyedbglib>=2.17.*
iotprovision pymcuprog>=3.7.*
iotprovision pydebuggerupgrade>=3.4.*
iotprovision pykitcommander>=2.6.4.*
iotprovision pytrustplatform>=0.15.*
iotprovision pyawsutils>=0.11.10.*
iotprovision pyazureutils>=0.8.*
iotprovision pydebuggerconfig>=3.8.3.*
matomo requests>=2.*
aiotodoist aiohttp<4.*,>=3.*
aiotodoist todoist-python>=8.*
demcompare xarray>=0.13.*
metdig meteva>1.3.*
polars pyarrow>=4.0.*; extra == "pyarrow"
polars pyarrow>=4.0.*; extra == "pandas"
pulsar-client-sn protobuf<=3.20.*,>=3.6.1; extra == "all"
pulsar-client-sn protobuf<=3.20.*,>=3.6.1; extra == "functions"
amazon-textract-helper Pillow>=9.2.*
amazon-textract-helper PyPDF2>=2.5.*
amazon-textract-overlayer Pillow>=9.2.*
amazon-textract-overlayer PyPDF2>=2.5.*
flask-openapi3 pydantic<2.*,>=1.2
pyaml-env PyYAML<=6.*,>=5.*
flask-gatekeeper Flask>=1.1.*
bodo pandas<1.5,>=1.3.*
amazon-textract-pipeline-pagedimensions Pillow>=9.2.*
amazon-textract-pipeline-pagedimensions PyPDF2>=2.5.*
ccdc-opencivicdata Django>=4.0.*
img2text colorama>=0.4.*
nbmanips cloudpickle>=1.6.*
nbmanips click>=7.1.*
nbmanips Pygments>=2.10.*
nbmanips colorama>=0.4.*
nbmanips beautifulsoup4>=4.10.*
ptlflow einops<=0.4.*,>=0.3.0
ptlflow numpy<=1.22.*,>=1.17.0
ptlflow opencv-python<=4.6.*,>=4.0.0.21
ptlflow packaging<=21.*,>=20.0
ptlflow pandas<=1.4.*,>=1.1.0
ptlflow pillow<=9.2.*,>=5.0
ptlflow plotly<=5.9.*,>=5.0.0
ptlflow pytorch-lightning!=1.3.*,!=1.4.*,<=1.6.*,>=1.1.0
ptlflow requests<=2.28.*,>=2.0.0
ptlflow scipy<=1.9.*,>=1.0.0
ptlflow torch<=1.12.*,>=1.8.1
ptlflow torchmetrics<=0.9.*,>=0.2
ptlflow torchvision<=0.13.*,>=0.9.2
ptlflow tqdm<=4.64.*,>=4.41.0
sqlfileexecutor psycopg2>=2.*
sqlfileexecutor libpyka>=0.*
sqlfileexecutor requests>=2.*
sqlfileexecutor click>=8.*
fsqlexec psycopg2>=2.*
fsqlexec libpyka>=0.*
fsqlexec requests>=2.*
fsqlexec click>=8.*
django-sca-helper django>=2.1.*
medical-shape torch>=1.7.*
torchsde numpy>=1.19.*; python_version >= "3.7"
as-ws-wrapper zeep>=4.0.*
fortes-webservice-wrapper zeep>=4.0.*
jntajis-python flake8>=3.9.*; extra == "dev"
jntajis-python jinja2>=3.*; extra == "dev"
jntajis-python pytest>=6.2.*; extra == "dev"
jntajis-python sphinx>=4.*; extra == "dev"
jntajis-python sphinx-rtd-theme>=0.5.*; extra == "dev"
knarrow importlib-metadata>=3.9.*; python_version != "3.10"
knarrow importlib-metadata<3.9.*; python_version == "3.10"
tables-io numpy<1.22.*; python_version < "3.8"
tables-io astropy<5.*; python_version < "3.8"
tables-io pandas<1.4.*; python_version < "3.8"
amaranth amaranth-yosys>=0.10.*; extra == "builtin-yosys"
cuquantum-python cuquantum>=22.7.*
dj-cloud-task google-cloud-tasks>=2.5.*
django3-livesync watchdog>=2.*
django3-livesync tornado>=6.*
dpm360-lightsaber scikit-learn>=1.*
dpm360-lightsaber pytorch-lightning>=1.*
dpm360-lightsaber ray>=1.13.*
fancy-sa-filemodel SQLAlchemy>=1.4.*
lightning-nets torch>=1.7.*
mlvisualizationtools dash-bootstrap-components>=1.0.0*; extra == "dash"
stubgenj JPype1<2.*,>=1.2.1
stubgenj JPype1<2.*,>=1.2.1; extra == "all"
stubgenj JPype1<2.*,>=1.2.1; extra == "core"
actorch ray<2.*,>=1.13.0
cegalprizm-investigator cegalprizm-hub<2,>=1.*
cegalprizm-pythontoolpro cegalprizm-hub<2,>=1.*
marshmallow-sa-core SQLAlchemy>=1.4.0.*
micropipelines numpy<=1.22.*
pipkin filelock>=3.0.*
timetracker-cli2 click>=8.0.*
timetracker-cli2 dateparser>=1.1.*
timetracker-cli2 requests>=2.27.*
timetracker-cli2 rich>=11.1.*
mk-feature-store Click>=7.*
mk-feature-store PyYAML>=5.4.*
mk-feature-store tenacity>=7.*
mk-feature-store dask<2022.02.0,>=2021.*
mk-feature-store google-cloud-datastore>=2.1.*; extra == "ci"
mk-feature-store google-cloud-storage<1.41,>=1.34.*; extra == "ci"
mk-feature-store google-cloud-datastore>=2.1.*; extra == "dev"
mk-feature-store google-cloud-storage<1.41,>=1.34.*; extra == "dev"
mk-feature-store google-cloud-datastore>=2.1.*; extra == "gcp"
mk-feature-store google-cloud-storage<1.41,>=1.34.*; extra == "gcp"
cegalprizm-scripting-server cegalprizm-hub<2,>=1.*
ednaml torch>=1.10.*
ednaml torchvision>=0.11.*
ednaml Pillow>=9.0.*
ednaml tqdm>=4.63.*
elemeno-ai-feast PyYAML>=5.4.*
elemeno-ai-feast tenacity>=7.*
elemeno-ai-feast dask<2022.02.0,>=2021.*
elemeno-ai-feast google-cloud-datastore>=2.1.*; extra == "ci"
elemeno-ai-feast google-cloud-storage<1.41,>=1.34.*; extra == "ci"
elemeno-ai-feast google-cloud-datastore>=2.1.*; extra == "dev"
elemeno-ai-feast google-cloud-storage<1.41,>=1.34.*; extra == "dev"
elemeno-ai-feast google-cloud-datastore>=2.1.*; extra == "gcp"
elemeno-ai-feast google-cloud-storage<1.41,>=1.34.*; extra == "gcp"
finetuning-scheduler torch>=1.9.*
gocd-tools requests>=2.27.*
gocd-tools PyYAML>=5.*
hikari-toolkit uncertainties>=3.*
moai-hydra-searchpath-plugin moai-mdk>=0.1.*
polars-u64-idx pyarrow>=4.0.*; extra == "pandas"
polars-u64-idx pyarrow>=4.0.*; extra == "pyarrow"
pysequansutils pyedbglib>=2.19.3.*
pysequansutils pykitcommander>=2.6.3.*
timeeval scikit-learn>=0.24.*
dataloop-upipe numpy<=1.22.*
galahad sortedcontainers>==2.4.*
galahad scikit-learn>=0.24.*; extra == "all"
galahad scikit-learn>=0.24.*; extra == "contrib"
galahad scikit-learn>=0.24.*; extra == "sklearn"
mdade torch>=1.7.*
pymoode numpy>=1.19.*
pymoode scipy>=1.7.*
rp-tagger sqlalchemy>=1.4.*
rp-tagger flask>=1.1.*
rstar pyarrow>=4.0.*; extra == "pandas"
rstar pyarrow>=4.0.*; extra == "pyarrow"
termfactory numpy>=1.18.*
transtab pandas>=1.3.*
betty-ml torch>=1.6.*
betty-ml numpy>=1.9.*
geopolars pyarrow>=4.0.*; extra == "pandas"
geopolars pyarrow>=4.0.*; extra == "pyarrow"
pruby uncertainties>=3.*
scresonators-fit matplotlib>=3.0.*
scrfit matplotlib>=3.0.*
torchde torch>=1.7.*
polars-lts-cpu pyarrow>=4.0.*; extra == "pyarrow"
polars-lts-cpu pyarrow>=4.0.*; extra == "pandas"
togglreports requests>=2.25.*
togglreports datetime>=2.7.*
togglreports argparse>=1.3.*
spd-eda pandas>=1.4.*
spd-eda xlsxwriter>=3.0.*
spd-eda pyodbc>=4.0.*
spd-eda seaborn>=0.12.*
spd-eda scipy>=1.9.*
spd-eda awswrangler>=2.16.*
spyglass-neuro jupyterlab>=3.*
spyglass-neuro pydotplus>=2.0.*
spyglass-neuro pymysql>=1.0.*
torchfl torch<1.12.0,>=1.11.*
Actually, though, it doesn’t look like the current behaviour of packaging is correct anyway:
>>> Version("1.0") in SpecifierSet("<=1.0")
True
>>> Version("1.0") in SpecifierSet("<=1.0.*")
False
>>> Version("1.0") in SpecifierSet("==1.0.*")
True
I don’t think this behaviour is “obvious”, so surely even if we decide do permit the “obvious” equivalent (as @pradyunsg suggested), those projects would be broken anyway (and it would be a silent breakage, which is worse than an easily actionable error message).
2 Likes
What version of packaging are you using? That looks like 21.3 from within pip, if I were to hazard a guess (we’re on 23.0 now).
But regardless, yea, that’s certainly a bit weird.
Pretty sure you’re not on the latest version.
>>> import packaging
>>> packaging.__version__
'23.0'
>>> from packaging.specifiers import SpecifierSet
>>> SpecifierSet("<=1.0.*")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/pradyunsg/Developer/github/pip/.venv/lib/python3.10/site-packages/packaging/specifiers.py", line 711, in __init__
parsed.add(Specifier(specifier))
File "/Users/pradyunsg/Developer/github/pip/.venv/lib/python3.10/site-packages/packaging/specifiers.py", line 245, in __init__
raise InvalidSpecifier(f"Invalid specifier: '{spec}'")
packaging.specifiers.InvalidSpecifier: Invalid specifier: '<=1.0.*'
Regardless, thanks for checking that @pf_moore – that basically confirms for me that it’s fine for us to just… not bother with this.
That’s right. I was trying to demonstrate that projects would be getting the wrong answers with the previous version of packaging anyway, so there’s no compatibility benefit from allowing this.
1 Like
In a word, I disagree to permit wildcards in comparison specifiers.
According to the legacy implementation, >=X.* and >X.* are equivalent to >=X.0, and <X.* and <=X.* are equivalent to <X.0, in the transition period, a simple normalization function can fix this.
What about a silent normalization and a warning message to notify the users to migrate.
@frostming I’m confused by your reply.
I take this to mean that you wish to disallow comparison specifiers (it’s ambiguous what you’re disagreeing with).
This would imply allowing it, which is contradictory.
Given the use of disagree, I’m gonna assume that you meant that you disagreed with me… and want to permit such specifiers.
We can do that in packaging and we’ve already been printing warnings on such specifiers via the only mechanism that it has for over 2 years (or, erroring out).
packaging can’t be printing user-facing warnings and, given how late we’re catching this, I assume that we’re gonna have to do that in pip as part of migrating to this.
Here’s what I’m going to say… I don’t think there’s a gradient here: it’s either we permit this, or we don’t permit this. The question of migration models is a separate thing that I’d prefer we figure out elsewhere.
I don’t feel strongly about accepting this, and if someone else does, I’m happy to coauthor a tiny PEP to make the case for this. The process is straightforward (PEP 12 is long, but clear about what you need to do).
FTR we are assessing the impact of upgrading packaging from 21 to 23 in Fedora.
Our RPM dependency generators use packaging to parse requirements from installed distribution metadata. We asserted that packages that require things like foo>=1.2.* fail the build.
We have ~3.6k packages with Python dist-info/egg-info. Together they have around the same number of unique requirements.
Will report back when we are done.
1 Like
I don’t think these should be allowed, they cause users confusion over >= 1.0.* vs >= 1.0 and what > 1.0.* even means and PEP 440 is very clear that they are forbidden. I’d rather ban more things from PEP 440 and get a simpler, easier to understand standard.
To add some numbers, I’ve downloaded the pypi requires_dist for all files on pipy (8,175,706 entries) in the following format (which fwiw is partially not the real requires_dist in METADATA):
{"name": "pandas", "version": "1.4.4", "requires_dist": ["python-dateutil (>=2.8.1)", "pytz (>=2020.1)", "numpy (>=1.18.5) ; platform_machine != \"aarch64\" and platform_machine != \"arm64\" and python_version < \"3.10\"", "numpy (>=1.19.2) ; platform_machine == \"aarch64\" and python_version < \"3.10\"", "numpy (>=1.20.0) ; platform_machine == \"arm64\" and python_version < \"3.10\"", "numpy (>=1.21.0) ; python_version >= \"3.10\"", "hypothesis (>=5.5.3) ; extra == 'test'", "pytest (>=6.0) ; extra == 'test'", "pytest-xdist (>=1.31) ; extra == 'test'"], "filename": "pandas-1.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"}
regex suggestions welcome, this is just the first thing i came up with:
# legal
$ rg -c '==\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
27452
# all
$ rg -c '(>|<)=?\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
6721
# individual categories
$ rg -c '<=\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
962
$ rg -c '>=\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
5465
$ rg -c '<\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
393
$ rg -c '>\s*\d+(\.\d+)*\.\*' pipy_requires_dist.ndjson
134
That is it affects 6,721 of 8,175,706 files. I unfortunately don’t have the data to check how old these files are, i.e. how many of them are 10+ year old pre-PEP 440 releases.
I’d also be interested in the fedora data, i’d expect that none of them are pre-PEP440 ancient files and are better maintained and should have less matches.
1 Like
Problems from Fedora:
pcs
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/pcs/
python3dist() provides generator:
packaging.version.InvalidVersion: Invalid version: '0.11.4.15-f7301'
pyodbc
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/pyodbc/
python3dist() provides generator:
packaging.version.InvalidVersion: Invalid version: '4.0.0-unsupported'
See Use PEP 440 version name required for wheels · mkleehammer/pyodbc@996fce1 · GitHub
This is a verified bug in the RPM package of pyodbc (the above commit just makes it not manifest in this way).
python-btchip
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-btchip/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
python-pyscard>=1.6.12-4build1; extra == "smartcard"
~~~~~~~~~~^
python-dipy
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-dipy/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected closing RIGHT_PARENTHESIS
fury (>=0.8.0scikit-learn) ; extra == 'all'
~~~~~~~~^
python-dropbox
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-dropbox/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
stone>=2.*
~~~^
python-haversion
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-haversion/
python3dist() provides generator:
packaging.version.InvalidVersion: Invalid version: 'main'
This is a verified bug in Fedora’s RPM packaging of pyhaversion.
python-lacrosse
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-lacrosse/
python3dist() provides generator:
packaging.version.InvalidVersion: Invalid version: 'unknown'
This is a verified bug in Fedora’s RPM packaging of pylacrosse.
python-pdir2
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-pdir2/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
typing-extensions>=4.2.*
~~~~~^
This is caused by downstream patching.
python-pvc
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-pvc/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
pyvmomi>=5.5.0-2014.1.1
~~~~~~~~~~~~^
python-pytest-httpx
https://copr.fedorainfracloud.org/coprs/thrnciar/python-packaging/package/python-pytest-httpx/
python3dist() requires generator:
packaging.requirements.InvalidRequirement: Expected end or semicolon (after version specifier)
pytest<8.*,>=6.*
~~^
2 Likes
A lot of those look like local versions that aren’t using the local version syntax?
1 Like
Only 3 failures are directly related to this topic. And at least one of them is caused by sedding typing-extensions==4.2.* to typing-extensions>=4.2.* by the Fedora packager.
I disagree that “ignore the .*” is obvious semantics for the > and < case.
>= 1.0.* could reasonably be interpreted as “greater than or equal to any version with major.minor == 1.0”, and all versions that are “greater than or equal to any version with major.minor == 1.0” are also “greater than or equal to 1.0”, so that is equivalent to >= 1.0, indeed.
But > 1.0.* could reasonably be interpreted as “greater than any version with major.minor == 1.0”, and that’s not the same thing as “greater than 1.0” , because 1.0.42 is greater than 1.0, but not “greater than any version with major.minor == 1.0”, because it is a version with major.minor == 1.0.
Or more succinctly: if > 1.0.* means “greater than any version with major.minor == 1.0”, that’s equivalent to >= 1.1
2 Likes