A number of modules lived on PyPI prior to being added to the standard library, or they have backports that are no longer relevant. Unfortunately these tend to garner a fair amount of downloads for various reasons and can often cause people confusion when they’re used. This is particularly a problem in situations where the import name in the PyPI package is the same name as is used in the standard library, since the standard library will typically shadow the installed package .
In the backport cases, with modern packaging you can set
requires-python at the start of the backport in order to prevent that project from being installed on versions where the package is already a part of the standard library, at least when the import names match.
However that leaves the cases of packages that were developed on PyPI first, or in older backports where the
requires-python metadata wasn’t added, where people can still install these packages, but not actually use them because the stdlib takes precedence.
This outcome of people doing this tends to range from “harmless but silly” to "actively causing issues, but confusing everyone involved because there’s two versions of the same thing installed (PyPI and standard library) and a bunch of expectations are being violated all over the place.
So the question then becomes, what should the recommended approach be for projects in these situations? An obvious suggestion going forward is properly setting
requires-python for all backports, but beyond that?
I see a few options:
- Do nothing, let the projects just sit there unmaintained and just accept the occasional issue that crops up from the confusion.
- Delete the files that have been uploaded, forcing everyone using it to stop using it even on those older (presumably no longer supported) versions of Python.
- Utilize yanking to yank the files, allowing people to still pin to those versions (with a warning message) but otherwise act like those versions have been deleted.
- This also comes with a yanked marker in the PyPI Web Interface.
- Decide none of these work, and come up with something better and try and write a PEP and get it approved.
The behavior of the first two options are pretty easily understood, however the behavior of the yanked option can be a tad subtle, and it actually depends on what version of pip is being used to do the install, and how the thing is being referenced to be installed.
For pip 22+, the behavior is basically:
- If the version specifier is NOT
===, then ignore yanked versions.
- If the version specifier is
===and a yanked file is the only available file, then use it but print a warning (that can include a message to the user from the package) otherwise use the unyanked version.
For pip prior to 22.0, the behavior is basically:
- Prefer unyanked versions over yanked versions, assuming they both match the version specifier.
- If the only versions matching the specifier (regardless of what it is, even no specifier) is a yanked version, then use it but print a warning (that can include a message from the package).
To see the output, see:
pip 22+, not using
❯ pip download 'pip>=21.2,<21.2.1' ERROR: Could not find a version that satisfies the requirement pip<21.2.1,>=21.2 (from versions: 0.2, 0.2.1, 0.3, 0.3.1, 0.4, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.6.3, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 1.3, 1.3.1, 1.4, 1.4.1, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.1.0, 6.1.1, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.1.0, 7.1.1, 7.1.2, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.1.0, 8.1.1, 8.1.2, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 10.0.0b1, 10.0.0b2, 10.0.0, 10.0.1, 18.0, 18.1, 19.0, 19.0.1, 19.0.2, 19.0.3, 19.1, 19.1.1, 19.2, 19.2.1, 19.2.2, 19.2.3, 19.3, 19.3.1, 20.0, 20.0.1, 20.0.2, 20.1b1, 20.1, 20.1.1, 20.2b1, 20.2, 20.2.1, 20.2.2, 20.2.3, 20.2.4, 20.3b1, 20.3, 20.3.1, 20.3.2, 20.3.3, 20.3.4, 21.0, 21.0.1, 21.1, 21.1.1, 21.1.2, 21.1.3, 21.2, 21.2.1, 21.2.2, 21.2.3, 21.2.4, 21.3, 21.3.1, 22.0, 22.0.1, 22.0.2, 22.0.3, 22.0.4, 22.1b1, 22.1, 22.1.1, 22.1.2, 22.2, 22.2.1, 22.2.2, 22.3, 22.3.1, 23.0, 23.0.1, 23.1, 23.1.1, 23.1.2) ERROR: No matching distribution found for pip<21.2.1,>=21.2
pip 22+, using
❯ pip download 'pip==21.2' Collecting pip==21.2 Downloading pip-21.2-py3-none-any.whl (1.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 34.3 MB/s eta 0:00:00 WARNING: The candidate selected for download or install is a yanked version: 'pip' candidate (version 21.2 at https://files.pythonhosted.org/packages/03/0f/b125bfdd145c1d018d75ce87603e7e9ff2416e742c71b5ac7deba13ca699/pip-21.2-py3-none-any.whl (from https://pypi.org/simple/pip/) (requires-python:>=3.6)) Reason for being yanked: See https://github.com/pypa/pip/issues/8711 Saved ./pip-21.2-py3-none-any.whl Successfully downloaded pip
pip < 22, using anything
$ pip download 'pip>=21.2,<21.2.1' Collecting pip<21.2.1,>=21.2 Downloading pip-21.2-py3-none-any.whl (1.6 MB) |████████████████████████████████| 1.6 MB 6.4 MB/s WARNING: The candidate selected for download or install is a yanked version: 'pip' candidate (version 21.2 at https://files.pythonhosted.org/packages/03/0f/b125bfdd145c1d018d75ce87603e7e9ff2416e742c71b5ac7deba13ca699/pip-21.2-py3-none-any.whl#sha256=71f447dff669d8e2f72b880e3d7ddea2c85cfeba0d14f3307f66fc40ff755176 (from https://pypi.org/simple/pip/) (requires-python:>=3.6)) Reason for being yanked: See https://github.com/pypa/pip/issues/8711 Saved ./pip-21.2-py3-none-any.whl Successfully downloaded pip
I don’t personally have a preference, but others might! If a strong consensus doesn’t form, most likely it makes sense to just accept the status quo and let “do nothing” stand.
But not always! Users are ever inventive in finding ways to do unexpected things, and can get the PyPI version installed so it shadows the standard library. ↩︎