# Requires-Python upper limits

**URL:** <https://discuss.python.org/t/requires-python-upper-limits/12663>\
**Category:** Coordination\
**Created:** [December 20, 2021, 6:22pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663 "2021-12-20T18:22:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 6:22pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/1 "2021-12-20T18:22:47Z")

</div>

# Requires-Python upper limits

Requires-Python was added to allow older versions of Python to be dropped by packages without breaking installation on older versions of Python. Currently (and for the last 4+ years), pip handles this quite simply; Requires-Python is a free-form SpecifierSet, and it checks to see if the current version of Python is included in the set. If not, it starts going back through a package’s history to find the most recent passing version. This was prompted by IPython dropping Python 2 - 3.3, IIRC, and has accelerated dropping older Python versions like 2.7, 3.5, and now 3.6 and even 3.7 for the data science community (NEP 29). This was designed for and supports _lower_ limits very well.

However, a growing number of packages are starting to use this for _upper_ limits as well. The official specification ([PEP 345](https://www.python.org/dev/peps/pep-0345/#requires-python) and [PyPA Core Metadata specification](https://packaging.python.org/en/latest/specifications/core-metadata/#requires-python)) both state:

> This field specifies the Python version(s) that the distribution is guaranteed to be compatible with.

This is extremely problematic; since you cannot “guarantee” compatibility with any future version of Python, every package following the specification seems to be required to add an upper cap. Besides my [opinion that this is a horrible practice](https://iscinumpy.dev/post/bound-version-constraints/), the fact that Python itself expects _most_ packages that support the current version without warnings to continue to work for at least the next two versions, and the fact this would seriously impact the already slow support for new Python versions (see the pycln disaster that brought down pybind11, cibuildwheel, and others [simply because it had an artificial cap](https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special) on Python \<3.10), this is absolutely not supported by Pip correctly.

Take Numba as an example. Out of any package that I know, this has the hardest cap on Python version possible. It uses bytecode details, which are declared internal (and likely to be changing quite a bit in the optimization focused changes coming in the next few Python versions!). They always have to adapt to new Python versions. So, in version 0.52, they started adding an upper bound to the Requires-Python metadata slot. They also still do the “correct” way to get a nice error message; they add a check and error to setup.py. So, if you `pip install numba` on Python 3.10 (a 3.10 compatible 0.55 is about to be released, so you might need `pip install numba<0.55` to get this example to work soon), then without explanation **pip downloads 0.51** , then crashes trying to compile a pinned dependency with no 3.10 wheels (llvmlite). No nice message, and it downloads some seemingly random older version. The extra metadata made this worse; dropping this would have caused the more recent, Python 3.10 compatible llvmlite to be used, would have grabbed numba 0.54, and would have produced the more useful error message.

> Edit: I just realized via testing that PDM (and probably Poetry?) use backsolves here too. So using `requires-python = ">=3.8"` on a new PDM project that has a single dependency will get numba 0.51, since it thinks that version supports “all” future Python versions, while 0.54 doesn’t support 3.10+!

Other packages are adopting this too; SciPy is the most recent (and the reason I’m discussing this here). In [the full discussion](https://github.com/scipy/scipy/issues/14738), they are not interested in working with the current implementation in Pip, but want the metadata to be “better” and are following the PEP/description, and adding an old “last” release without the cap that will break when used from any newer Python. This workaround has issues (why download some old version? If there are any dependencies in the future that break, this might not be the first point of failure, etc.) Another workaround based on a manylinux1 idea would be to add Python `3.n+1` wheels as soon as they are allowed (ideally after checking, just in case it _is_ compatible with Python 3.n+1!); these wheels would be empty and would just have a single dependency on some `python-not-supported==3.n+1` SDist package that breaks with a message about that version of Python not being supported.

Another major factor (possibly the driving factor already) is that Poetry pushes upper limits on everything, including Python, very hard - it defaults to `python = ^3.x`, and the authors even suggest capping at the last tested version. While I’d love for this to change, I’m not hopeful.

Hopefully I’ve shown the current situation is a mess. I’d like to propose three possible solutions that would bring the pip implementation and recommendations in sync, and provide a better path for packages.

I should point out that _implementation_ for all three of these suggestions requires a new feature in packaging’s SpecifierSet: `specset.overlap(spec)`. This would compute the analytic overlap of a set with a single specifier - it would return a falsy value if there is no overlap. The True value is unimportant for this use, it could just be a boolean, or it could be the intersection SpecifierSet. I’ve already wanted this for cibuildwheel; asking `requires_python.overlap("3.9.*")` would let cibuildwheel know if it needs to build Python 3.9 wheels. For the below ideas, `requires_python.overlap(">{current_python_version}")` would allow tools like Pip or setuptools the necessary information to detect upper caps.

I also should point out that `Requires-Python` was added to _fix broken solves_. Using it as an upper cap is _trying to break working solves_. This is very different, and something that is not supported for other things - there is no `Requires-Architecture` or similar. If you are on Python 3.10 and trying to use a package that says `<3.10`, then there is no “different” working version. If you allow the solve anyway, then it will either break anyway if the author was correct (though possibly later at runtime), or work correctly if they were just assuming it might not work. Unlike other limits, solvers can’t downgrade the Python version - that’s in the user’s hands.

Finally, locking package managers currently all (Poetry, PDM at least) **force** the Requires-Python metadata to match the _lock file_. This means that if you depend on scipy, for example, you will be forced to add scipy’s version cap to your metadata, _even if you don’t cap scipy_. This is due to the fact the lock file will be “invalid” on the newer Python version. Even though your _library_ might be absolutely fine upgrading. This is really their problem for tying these two settings into one, but still something to keep in mind.

* * *

## Idea 1: Avoid upper capping

This is the smallest code change, as it really is just standardizing the current way that Pip works. This would change the wording of PEP 345 and the spec to mention that `Requires-Python` is _not_ for limiting upgrades to newer Python versions. Something like:

> This field specifies the oldest Python version(s) that the distribution is known to be compatible with.

A warning could be added to setuptools, flit, and twine (and suggested for other tools) if an upper cap was detected. This is not what `Requires-Python` is for. We already have a system for indicating if a package was tested with a specific Python version - Trove classifiers. Enforcing limits will impede Python upgrades. This is also the nicest solution for lock files with the current implementation of locking package managers, since they currently don’t handle upper bounds correctly for libraries.

## Idea 2: Implement upper capping properly

This is a larger code change, but is likely the closest to what users are currently expecting. Pip (and suggested for other solvers) would immediately error out if an upper bound is detected; that is, if all  
future versions of Python are also excluded by Requires-Python. I believe Poetry and PDM may already do this.

This still requires changing the PEP 345 and such wording, as adding an upper limit is still a very bad idea for most packages, and would still impair the now yearly Python upgrade process. It also makes it much harder to test alpha/beta/RC’s, which I assume some of the packages doing this may not really be doing. The wording probably should be something like this:

> This field specifies the Python version(s) that the distribution is known to be compatible with.

This would indicate that adding an upper cap should only be done if you _know_ the upcoming Python release will break your code. Users _will_ misuse this (as Poetry suggests) and will cap even if they are not sure the next version of Python will break. But they are doing that today too, and this would at least provide the right error message.

## Idea 3: Ignore upper capping

This is the only solution that doesn’t require updating the text of PEP 345. The code change is nearly identical to the above solution; Pip (and suggested for other solvers) would ignore the back-search if an upper bound is detected; that is, if all  
future versions of Python are also excluded by Requires-Python.

The nice thing (depending on your view) is that this makes the Trove classifiers and Requires-Python identical, so it makes it easier to automatically generate them from Requires-Python. Adding an upper bound would have no downside (as mentioned above, upper caps _never_ fix a solve), so you could use this to actually specify what you _are_ guaranteed to be compatible with without breaking what you might be compatible with.

The few packages that really are incompatible with a future version of Python could do exactly what they are trying to do today, and add an error to setup.py if a newer version is detected. Different build backend could provide ways to error out - in fact, tools like Poetry could use this custom setting to drive the locking instead of the Requires-Python field. Unfortunately, if Poetry, PDM, and others do not follow this ignoring of the field, this could make them harder to use if more packages start adding upper caps.

* * *

Option one formalizes the current Pip behavior. Option two gives people what they think they want, even if it is often destructive. Option three minimizes the chances of things failing when they don’t need to, and has some redundancy with Trove classifiers. Personally, I don’t have a strong preference. What do you think? Is there another option I missed?

Closest discussion I could find was [https://discuss.python.org/t/use-of-less-than-next-major-version-e-g-4-in-python-requires-setup-py](https://discuss.python.org/t/use-of-less-than-next-major-version-e-g-4-in-python-requires-setup-py).

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [December 20, 2021, 7:45pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/2 "2021-12-20T19:45:29Z")

</div>

+1 from me for idea 1. I believe that having an upper bound on the Python version is the wrong thing to do and we shouldn’t support it or make it easier for people to do it.

If there’s a genuine situation where adding a cap to the Python version is believed to be the correct solution, someone should present the underlying requirement and we can look at working out a means of supporting that, which doesn’t use Requires-Python with an upper bound.

---

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 8:02pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/3 "2021-12-20T20:02:39Z")

</div>

One extra data point: adding an upper bound breaks solving in both Poetry and PDM in a really bad way. If you add a basic file like this:

```auto
[project]
name = ""
version = ""
dependencies = [
  "numba",
]
requires-python = ">=3.8"

[build-system]
requires = ["flit_core"]
build-backend = "flit_core.buildapi"

```

(Or the matching, non-PEP 621 version for Poetry) (using numba since SciPy currently supports 3.10) then PDM/Poetry will back solve to the last version that “supports” all future versions of Python (the last one without an upper limit, 0.51), and will lock that. Which of course breaks, _even if you are currently running on Python 3.9_! I guess it’s nice that it doesn’t force the user to cap in that case, at least? 😉

Every single version ever released without a cap has to be yanked to fix this. Or the caps can be removed and it then resolves the latest versions as expected.

PS: I love that PEP 621 allows you to use other PEP 621 backends with PDM! I so wish Poetry would support this!

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [December 20, 2021, 8:05pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/4 "2021-12-20T20:05:20Z")

</div>

> [@henryiii](#):
>
> Every single version ever released without a cap has to be yanked to fix this.

And this is the key reason why I consider upper bounds on Python versions to be broken. There’s no good transition method to add a cap. A reasonable solution to the underlying issue people are trying to solve with caps (whatever that issue is) would have a proper transition mechanism.

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [December 20, 2021, 9:23pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/5 "2021-12-20T21:23:14Z")

</div>

@rgommers ^ FYI, in case you missed it on scipy/scipy#14738

---

<div class="post-metadata">

**Author:** ![EpicWink](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/epicwink/32/17968_2.png) [@EpicWink](https://discuss.python.org/u/EpicWink)\
**Post date:** [December 20, 2021, 9:39pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/6 "2021-12-20T21:39:49Z")

</div>

> [@pf\_moore](#):
>
> If there’s a genuine situation where adding a cap to the Python version is believed to be the correct solution, someone should present the underlying requirement and we can look at working out a means of supporting that, which doesn’t use Requires-Python with an upper bound.

In the original post, Numba already has an existing requirement (although it sounds more like a requirement to an upper bound of the version of CPython)

Edit: Numba’s (resolved) Python 3.9 issue: [Python 3.9 Support · Issue #6345 · numba/numba · GitHub](https://github.com/numba/numba/issues/6345#issuecomment-738696458)

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [December 20, 2021, 9:47pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/7 "2021-12-20T21:47:01Z")

</div>

I’m just a common Python developer, but this is an issue I’ve struggled with too and gone back and forth on for both Python and dependency versions, for my packages and others, so I appreciate the through and detailed post and consideration of options to resolve this. To me, the central issue is that without a way to retroactively declare that past versions of a package are or are not compatible with a certain Python or dependency version, we’re stuck having to guess and either apply an upper bound, or not.

Even assuming we do from the very first released distribution (for new projects), this still means that we block otherwise perfectly viable resolutions if it turns out newer versions of Python (or deps) are compatible, and cannot later go back and update that while frequently releasing new versions just to bump version specifiers once tested. On the other hand, if we don’t add an upper bound and a version does break, we must also release a new version, and there’s equally no way to mark old versions as incompatible. Unfortunately, as you explain in the linked issue, with the current architecture this is an _extremely_ involved process to fix, so it seems its more worthwhile to mitigate it instead, as best we can, via the suggestions here.

> [@henryiii](#):
>
> This field specifies the oldest Python version(s) that the distribution is known to be compatible with.

We would need to make sure the wording was clear that specifying lower bounds is okay, and upper bounds aren’t, to avoid any misinterpretation as to prohibit specifying the minimum for each major version, minor version, etc. Perhaps something involving “known version” or inverting the statement? We can bikeshed on this later if we choose to go this route, just wanted to point out that we’d really need clarity here with the final statement.

> [@henryiii](#):
>
> This is a larger code change, but is likely the closest to what users are currently expecting. Pip (and suggested for other solvers) would immediately error out if an upper bound is detected; that is, if all  
> future versions of Python are also excluded by Requires-Python. I believe Poetry and PDM may already do this.

To clarify, would the error be if _any_ upper bound was used in this field, even if the current Python version is included in the `SpecifierSet`, only if an upper bound was detected _and_ the current version of Python is _not_ within the `SpecifierSet`, or most specifically, if and only if the _upper bound_ was not satisfied by the current Python version? Of these three sub-options, the latter, I think, would be the most useful and least disruptive to packages that do use it, but possibly require more complex parsing of this field.

> [@henryiii](#):
>
> This field specifies the Python version(s) that the distribution is known to be compatible with.

Or, really, shouldn’t this say the inverse:

> This field specifies the Python version(s) that the distribution is not known to be incompatible with.

Otherwise, it essentially says the same thing as the current version, in different words, as it generally cannot be known that the distribution will be compatible with any and all future Python versions, so we’re stuck with the same issue?

---

<div class="post-metadata">

**Author:** ![mattip](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mattip/32/330_2.png) [@mattip](https://discuss.python.org/u/mattip)\
**Post date:** [December 20, 2021, 10:06pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/8 "2021-12-20T22:06:35Z")

</div>

While the trove specifiers do help mitigate this problem, I don’t see a fundamental difference between newly added caps on the requires-python field and newly added caps on the dependencies listing. Python is just one more dependency to be solved for. In both cases, the resolver will backtrack to the version just before the cap was added. So I am in favor of solution 3 to not back-search at all if version capping is detected in either the dependencies field or the requires-python field.

> The few packages that really are incompatible with a future version of Python

I disagree with the sentiment here. Adjusting to a new version of python requires work. Look how long it is taking for conda-forge to complete the Python310 migration and how many packages were/are still broken.

---

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 10:34pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/9 "2021-12-20T22:34:14Z")

</div>

> In the original post, Numba already has an existing requirement (although it sounds more like a requirement to an upper bound of the version of CPython)

I almost elaborated on that - I believe @pf_moore is _not_ saying that no package has a cap on Python version, I think he is saying adding a cap on Python version doesn’t _solve_ anything - it’s not really a solution to a problem. You can’t “fix” anything by capping Python. You can just break solves. (This is different in conda, where the Python version can be part of the solve, and can be rolled back if it fails). The benefit is a slightly more focused, nicer error message, but you can also do this with an error in setup.py, with the binary upload trick, etc.

> Even assuming we do from the very first released distribution

Just to add, if you get it wrong, say if you go from `^4.0` (Poetry’s _default_ cap) to `<3.10`, it’s too late, the old packages are broken and will break solves for anyone who uses anything that includes 3.10 or newer. If you start by capping every single version, that’s “better”, but then it causes all sorts of issues upgrading (every dependency has to manually update this field, and rerelease, linearly up the chain, even though Python carefully _avoids_ breaking changes!).

> We can bikeshed on this later if we choose to go this route

My examples were by no means final, just showing the basic idea.

> and only if the _upper bound_ was not satisfied by the current Python version

What do you mean by “the current version of Python”? I don’t think it makes any sense if the “current” Python is the one running, so I guess you mean the latest Python available? It would actually be cute if this was time-dependent; that is, upper limits were allowed around the time of the next Python (that’s now date dependent, after all) - but due to the issues above, this would not be sufficient. Even though Numba’s \<3.10 is absolutely “correct” as far as conceptual metadata, it is also wrong - it can break a solve on 3.9 for 3.9 on either locking package manager because it will grab an old version of numba in order to make a lock file that allows Python 3.10+ (at least it thinks it does).

The idea would be _all_ upper limits would be disallowed. You’d have to have at least one `>=` or `>`. (For option 1)

> Or, really, shouldn’t this say the inverse

Yes, wording was not great, it wasn’t meant to be final. I don’t like the double negative there, but it does specify the thing I was trying to say better. It could be explicit:

> This field specifies the earliest Python version(s) that the distribution is known to be compatible with, and must not have an upper bound.

(Again, option 1)

> I don’t see a fundamental difference between newly added caps on the requires-python field and newly added caps on the dependencies listing.

There is a key, fundamental difference (ironically, only for PyPI and not for Conda) - you can’t “solve” for the Python version. For a package, if you have a limit `<3`, a valid choice is to pick version 2. You can’t do this for Python. In conda, you actually can, though Python version is so important that much of the time, you select an exact (minor) release of Python directly. So the only choice is to scroll back through time and try to get an older version of the package with the cap.

In other words, `click<8` will very likely give me click==7; only if a package pins click\>=8 will modern solvers start looking through history. At least some of the time, I still get a valid solve. Capping the Python version will never scroll back the Python version, and so will never give me a valid solve.

This is also why lock files treat the Python version **fundamentally differently**. Packages are locked (hence the name, lock files), while Python is _never_ locked (again, not including conda). You can’t specify Python 3.9.9 or Python 3.9 or Python anything, instead you get a range (in a lock file!) of Python’s where this lock file is valid - you don’t know what Python version will try to install from this lock file. This is why PDM and Poetry force you to cap Python if python is capped by a dependency (to match the lock file limits), but they do not require you to cap any other dependency, because those are locked to a single value only in the lock file.

---

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 10:39pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/10 "2021-12-20T22:39:04Z")

</div>

> Adjusting to a new version of python requires work. Look how long it is taking for conda-forge

Over half of conda-forge is already converted, and conda started late. A good portion of it is just waiting for CI to run and a maintainer to push a button. Half of the remaining packages are waiting for parents to be ready. Brew was much quicker (almost same day as the release), and it got a massive number (almost 1/3 - 1/2 IIRC) within a week or so. Lots of packages do not require changes. Some do, but many do not. Also, both systems are _highly_ biased toward binary packages; pure Python packages are much less likely to require changes. Some of the changes are just getting the test suites to pass without deprecation warnings! If you add caps, 100% of them do. But I was also referring to the design intention, which is that “good” packages should work on the next two Python versions, approximately.

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [December 20, 2021, 10:40pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/11 "2021-12-20T22:40:08Z")

</div>

> [@EpicWink](#):
>
> In the original post, Numba already has an existing requirement

But version capping doesn’t _satisfy_ that requirement, because it results in older versions (which pre-dated the addition of the cap) being downloaded. So let’s work out what they _actually_ want so we can design a solution that does work. (I assume what they want is something along the lines of “ensure that tools never install any version of numba older than 0.55 on 3.10 or later”, and that can’t be done using only metadata on versions 0.52 and later).

I don’t have a solution here - it’s a hard problem. But applying the wrong solution and trying to patch it up to work, seems to me like the worst of both worlds.

---

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 10:46pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/12 "2021-12-20T22:46:59Z")

</div>

> because it results in older versions (which pre-dated the addition of the cap) being downloaded

This is currently the case, but Solution 2 would actually give Numba the solution they would like. I had a mix of good and bad points for that, but that is a “solution” for them (which currently does not work). The other solutions would actually be closer to solutions to their problem too - removing the capping would at least get the most recent version, so they could work on a way to break with the correct error message (the old versions don’t break correctly).

The biggest problem with Solution 2 was it makes it really easy to misuse capping to put limits on libraries. And it doesn’t fix the issue with locking package managers without their involvement; they would also have to avoid back solving for Python upper caps too.

For any solution, I am kind of volunteering to work on the needed `.overlap` method for SpecifierSet. 🙂

---

<div class="post-metadata">

**Author:** ![ilayn](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ilayn/32/5769_2.png) [@ilayn](https://discuss.python.org/u/ilayn)\
**Post date:** [December 20, 2021, 10:52pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/13 "2021-12-20T22:52:18Z")

</div>

I’ll just summarize what I have written in SciPy discussion here just to kickstart the argumentation. Not really knowlegable in this tooling however I do know professionally since I am doing similar things for a living it is not users fault if majority of them are getting it wrong.

First one is the easy one. As mentioned previously, `Requires-Python` is bad naming. Anybody who is not versed in packaging will not get this right. I didn’t even know there will be a difference. It says exactly which Python versions it will accept to be installed. There is not even a hint of these things.

> [@henryiii](#):
>
> I almost elaborated on that - I believe @pf_moore is _not_ saying that no package has a cap on Python version, I think he is saying adding a cap on Python version doesn’t _solve_ anything - it’s not really a solution to a problem. You can’t “fix” anything by capping Python. You can just break solves.

That’s probably correct however this is exactly the problem that these tools don’t offer anything. The argument above is that future versions **might** break something. To that we unfortunately say it **does** break something. I do read the post multiple times but you are not addressing this very point. We don’t want to maintain 9 versions for 4 major python releases all the time. Things should error out when the time comes so that we can nudge people forward. This is not upper capping this is deprecation.

Say `package` v2.0 released for Python 3.8. Everything was nice and dandy. Then came Python 3.9 and 3.10 this is going to be installed by users all the time and all will go to the repo of `package` and mention that version 2.0 is broken. Now you are leaving the package author under both obligation that nobody intended for (maybe it was meant only for Python 3.8 who knows) and also you are burdening everyone to keep up which is simply design by wishful thinking we don’t have that capacity anywhere. Things rot and should have the right to rot. As mentioned SciPy was broken with **every** new Python major release since 3.7 so this is not some arbitrary package complaint. And we spend a lot of time with every major release

Thus with some other extra reasons, I disagree with the sentiment and I also find it “horrific” to check the versioning backwards to find a working solution. I don’t know who ordered it but installing any version that doesn’t complain in a metadata tag is not really the solution to anything. I don’t know what it solves but just randomly trying versions out does not also constitute a solution to blame others for bad practice

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [December 20, 2021, 10:59pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/14 "2021-12-20T22:59:45Z")

</div>

> [@henryiii](#):
>
> What do you mean by “the current version of Python”? I don’t think it makes any sense if the “current” Python is the one running, so I guess you mean the latest Python available?

Sorry I wasn’t precise enough here. What I’m referring to is _the version of the Python runtime that the package was attempting to be installed under_; unlike conda, pip of course cannot upgrade or downgrade the Python version to satisfy such a dependency (EDIT: Which I now see you talk about later).

I think our confusion may be that I’m referring to the behavior of user-facing installation tools, not developer-facing build tools here. Did you mean to only apply the above to the latter?

> [@henryiii](#):
>
> It would actually be cute if this was time-dependent; that is, upper limits were allowed around the time of the next Python (that’s now date dependent, after all) - but due to the issues above, this would not be sufficient. Even though Numba’s \<3.10 is absolutely “correct” as far as conceptual metadata, it is also wrong - it can break a solve on 3.9 for 3.9 on either locking package manager because it will grab an old version of numba in order to make a lock file that allows Python 3.10+ (at least it thinks it does).
> 
> The idea would be _all_ upper limits would be disallowed. You’d have to have at least one `>=` or `>` . (For option 1)

Sorry if I wasn’t clear—I’m talking about idea 2, “Implement upper capping properly” here, as was the text of yours I was quoting. The title and the suggested text imply that a package installer would only raise an error if the upper cap _would not be satisfied_ for the version of the Python runtime that the package was attempting to be installed under, instead of back-searching, while still allowing installation if it was. But the text here:

> [@henryiii](#):
>
> Pip (and suggested for other solvers) would immediately error out if an upper bound is detected; that is, if all  
> future versions of Python are also excluded by Requires-Python.

Implies that tools would produce an error if any upper bound was used, even if the supported specifierset could be satisfied by the Python version the user is attempting to install under, which would mean that upper caps would not be usable at all (essentially a stronger version of Idea 1). What I was trying to clarify was precisely the circumstances under which you were suggesting tools raise an error versus install, and to follow up on that now, how you are suggesting this apply to build vs install tools.

> [@henryiii](#):
>
> Yes, wording was not great, it wasn’t meant to be final. I don’t like the double negative there, but it does specify the thing I was trying to say better. It could be explicit:
> 
> > This field specifies the earliest Python version(s) that the distribution is known to be compatible with, and must not have an upper bound.
> 
> (Again, option 1)

Again, this wording was quoting Idea 2, not idea 1, while the above wording appears to be for the latter. The issue was the proposed wording for idea 2 didn’t really say anything different to the current wording, as the statement and its inverse have fundamentally different meanings.

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [December 20, 2021, 11:16pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/15 "2021-12-20T23:16:31Z")

</div>

> [@ilayn](#):
>
> I don’t know who ordered it but installing any version that doesn’t complain in a metadata tag is not really the solution to anything. I don’t know what it solves but just randomly trying versions out does not also constitute a solution to blame others for bad practice

Just to note, @henryiii addresses this both in the linked blog post, and the Scipy issue you were part of. IPython originally asked for this in order to be able to drop old Python versions, and for this it works just fine, as @henryiii explains above and elsewhere. So long as the specifier is added with or before the first version to drop support for earlier Python versions, which is generally reasonable, if the user’s Python version (say, 2.7 or early 3.x) is unsupported by the most recent version of a package, `pip` will search back until it finds a package version that does support it, or lacks the tag. This _has_ been the solution to _this_ problem for a very long time, and generally speaking has worked as intended.

The whole point seems to be that this behavior breaks down and doesn’t make much sense when applied to _upper bounds_ on future Python, which it presumably wasn’t really intended for (but that wasn’t made clear), which it sounds like you actually agree on. He then proposes multiple practical solutions to address the issue for the upper bound case, with varying degrees of tradeoff, none of which are just “blam[ing]” downstream package authors, but rather either (1) clarifying the documentation as to the purpose of this field but retaining the existing behavior, (2) allowing upper bounds but _not_ triggering the behavior you object to in that case and rather erroring instead, as you presumably want, or (3) ignoring the upper bound in that field while pivoting toward other, more appropriate (package manager specific) methods of handling the situation.

---

<div class="post-metadata">

**Author:** ![henryiii](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/henryiii/32/3493_2.png) [@henryiii](https://discuss.python.org/u/henryiii)\
**Post date:** [December 20, 2021, 11:23pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/16 "2021-12-20T23:23:25Z")

</div>

> [@ilayn](#):
>
> I don’t know what it solves

Sorry, I thought I already answered this in the other thread. It solves a very important problem that SciPy is heavily using. Load up Python 2.7 and type `pip install scipy`. It works. Why? Because Pip checks scipy 1.73, and Requires-Python doesn’t match `"2.7.19"`. It then checks the version before, the version before, and finally ends up on whatever version was the last one to match 2.7 (don’t remember which one). It pip installs it, boom, working SciPy - even though the SciPy developers don’t bother with 2.7 anymore. Same thing for 3.5, 3.6, and very soon 3.7. **This is what Requires-Python was designed for, and it works wonderfully**. This is _why_ NEP 29 was able to specify such high limits - it’s not that everyone on Python 3.7 (which is the most popular version of Python still last I checked) is now broken with the PyData community dropping them, they just get older packages to go with older Pythons. It’s great!

It just was _not_ designed for _upper_ limits, and that’s exactly what this whole post is about and all three solutions listed cover. That’s why the behavior is so weird and terrible for upper limits. Please read the initial post.

> _the version of the Python runtime that the package was attempting to be installed under_

My point is that makes no sense to try to develop a package with Python 3.10 if you cap \<3.10, so I don’t see how you can have three variations here. I thought you were talking about solution 1; any Requires-Python that had an upper limit would produce a warning for solution 1. If you were referring to solution 2, I might not have been making much sense as I was replying to the wrong things. 🙂

> Implies that tools would produce an error if any upper bound was used

Sorry, yes, I think I thought you were talking about solution 1. For solution 2, if you were using Pip on 3.10.2 and tried to install a package, and then computed `requires_python.overlap("<3.10.2")` and that returned None (no overlap), then pip _would not_ check older versions, and would just return an error message stating “This package requires {requires\_python}”. If there is an upper cap _and_ that upper cap has no overlap with the versions above the current version, then this package is not valid for the “back search”.

---

<div class="post-metadata">

**Author:** ![ilayn](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ilayn/32/5769_2.png) [@ilayn](https://discuss.python.org/u/ilayn)\
**Post date:** [December 20, 2021, 11:25pm UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/17 "2021-12-20T23:25:42Z")

</div>

I should have mentioned my opening remark more strongly. I’m just carrying my confusion and the arguments here. Not necessarily repeating to be discussed again. My apologies

---

<div class="post-metadata">

**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\
**Post date:** [December 21, 2021, 12:06am UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/18 "2021-12-21T00:06:39Z")

</div>

> [@henryiii](#):
>
> Load up Python 2.7 and type `pip install scipy` . It works. Why? Because Pip checks scipy 1.73, and Requires-Python doesn’t match `"2.7.19"` . It then checks the version before, the version before, and finally ends up on whatever version was the last one to match 2.7 (don’t remember which one). It pip installs it, boom, working SciPy - even though the SciPy developers don’t bother with 2.7 anymore. Same thing for 3.5, 3.6, and very soon 3.7. **This is what Requires-Python was designed for, and it works wonderfully**.

I think another way to phrase this is `requires-python` says what _may_ work by excluding what is _guaranteed not_ to work. This is the same philosophy that wheel tags takes: tags express what _may_ be compatible and thus exclude what has no chance of working. Using an upper bound is prematurely guessing what may not work in the future and that goes against what `requires-python` is meant to express (under this definition).

This also makes the term “requires” accurate as you’re requiring Python _at least_ match the specified requirement to have any hopes of working. After that you hope you have what’s necessary to install and use the package (which is impossible to fully express with metadata for all scenarios w/o resorting to containers).

---

<div class="post-metadata">

**Author:** ![CAM-Gerlach](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cam-gerlach/32/3688_2.png) [@CAM-Gerlach](https://discuss.python.org/u/CAM-Gerlach)\
**Post date:** [December 21, 2021, 12:14am UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/19 "2021-12-21T00:14:42Z")

</div>

> [@henryiii](#):
>
> My point is that makes no sense to try to develop a package with Python 3.10 if you cap \<3.10, so I don’t see how you can have three variations here. I thought you were talking about solution 1; any Requires-Python that had an upper limit would produce a warning for solution 1. If you were referring to solution 2, I might not have been making much sense as I was replying to the wrong things. 🙂

Yeah, as mentioned (and you later figured out) I was referring to solution 2, and talking about user-facing install tools rather than developer-facing build tools. 🙂

> [@henryiii](#):
>
> For solution 2, if you were using Pip on 3.10.2 and tried to install a package, and then computed `requires_python.overlap("<3.10.2")` and that returned None (no overlap), then pip _would not_ check older versions, and would just return an error message stating “This package requires {requires\_python}”. If there is an upper cap _and_ that upper cap has no overlap with the versions above the current version, then this package is not valid for the “back search”.

Yep, this is the third interpretation, and the one I thought you meant. I was just confused by some of your wording, sorry.

---

<div class="post-metadata">

**Author:** ![ilayn](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ilayn/32/5769_2.png) [@ilayn](https://discuss.python.org/u/ilayn)\
**Post date:** [December 21, 2021, 12:52am UTC](https://discuss.python.org/t/requires-python-upper-limits/12663/20 "2021-12-21T00:52:55Z")

</div>

> [@brettcannon](#):
>
> This also makes the term “requires” accurate as you’re requiring Python _at least_ match the specified requirement to have any hopes of working. After that you hope you have what’s necessary to install and use the package (which is impossible to fully express with metadata for all scenarios w/o resorting to containers).

I have to disagree. This is a very confusing statement for anybody who has no idea what this tag is for. Everywhere in the ecosystem requirements are needed unconditionally. Even “requirements.txt” is an example of this behavior not “well at least with these might work”. Quite the contrary you **must** have these requirements. The word here, I repeat is unfortunate and should be explained better.

> [@brettcannon](#):
>
> Using an upper bound is prematurely guessing what may not work in the future and that goes against what `requires-python` is meant to express (under this definition).

I disagree again. This is not premature guessing. This has been practically demonstrated with all Python major releases both in NumPy and SciPy. We, well I should speak for myself, at least I do not share the faith or the optimism of backwards compatibility of the Python ABI and/or its standard modules as we have been actively struggling with these.

[Next page](https://discuss.python.org/t/requires-python-upper-limits/12663.md?page=2)