# Imposing a hard limit on Python integers

**URL:** <https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302>\
**Category:** Core Development\
**Created:** [August 29, 2024, 12:58pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302 "2024-08-29T12:58:42Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [August 29, 2024, 12:58pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/1 "2024-08-29T12:58:42Z")

</div>

Currently, the upper limit of Python integers is determined by the size of the block of memory that can be theoretically and practically allocated. On 32-bit platforms it is \frac{15}{2}(2^{31}-1) \< 2^{34} bits (2 bytes for every 15 bits). On 64-bit platforms the theoretical limit is \frac{15}{2}(2^{63}-1) \< 2^{66} bits. This requires insane amount of memory. This is much more than any existing computer have or can have in observable future. In both Intel64 and AMD64, the 52-bit physical address limit is defined in the architecture specifications. [64-bit computing - Wikipedia](https://en.wikipedia.org/wiki/64-bit_computing#Limits_of_processors)

I propose to impose hard limit of 2^{63}-1 or 2^{64}-2 bits for Python integers. With such limit (if forbid creation larger integers even if the size in bytes can fit in 64 bits) the number of bits can always be expressed as 64-bit integer (signed or unsigned), and some complex code in `longobject.c` and `mathmodule.c` could be simplified. Currently it is written to support integers larger than such limit, although this cannot be tested (and likely could not be tested in this century).

This can be implementation limit, not language limit.

I proposed this idea in the discussion about [gh-121485: Always use 64-bit integers for integers bits count by serhiy-storchaka · Pull Request #121486 · python/cpython · GitHub](https://github.com/python/cpython/pull/121486), and @mdickinson liked it. But this is a fundamental change and should be discussed among all core developers.

---

<div class="post-metadata">

**Author:** ![oscarbenjamin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/oscarbenjamin/32/1209_2.png) [@oscarbenjamin](https://discuss.python.org/u/oscarbenjamin)\
**Post date:** [August 29, 2024, 1:28pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/2 "2024-08-29T13:28:23Z")

</div>

> [@storchaka](#):
>
> this is a fundamental change

Is it a fundamental change?

If it is possible to change this now then it would be similarly possible to change it again in future if it somehow becomes desirable to exceed this limit. That is a purely theoretical concern for the foreseeable future though anyway as you say since at the least it would seem to require supporting some future architecture that does not even exist yet.

A more realistic concern: are there any compatibility implications for downstream extension modules resulting from this change?

I presume that this really only actually changes anything in a 32-bit build because otherwise `size_t` would already be 64 bits anyway right? So basically this is just making the type bigger on 32 bit system so that the code can be simpler?

Does there exist a supported platform where `size_t` is more than 64 bits?

---

<div class="post-metadata">

**Author:** ![notatallshaw](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/notatallshaw/32/10861_2.png) [@notatallshaw](https://discuss.python.org/u/notatallshaw)\
**Post date:** [August 29, 2024, 1:49pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/3 "2024-08-29T13:49:44Z")

</div>

Why is this in “Core Development” and not “Ideas”? It would be a big breaking change for many users (e.g. anyone who ever uses scripts to write Number Theory or Cryptography) and would presumably require a PEP?

I’ll note that limiting integers to be capped by default to printing 4,300 digits (approx 14,284 bits) without prior announcement caused significant backlash: [Int/str conversions broken in latest Python bugfix releases](https://discuss.python.org/t/int-str-conversions-broken-in-latest-python-bugfix-releases/18889).

Or am I misunderstanding, and this is an internal implementation detail that affects C code and not Python code?

**Edit:** Apologies, reading again, this is almost certainly “CPython implelentation integer” and not “Python Integers”. I hope you appreciate why I found the termanology confusing.

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [August 29, 2024, 1:57pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/4 "2024-08-29T13:57:00Z")

</div>

Hmm, that would make working with [Googolplex numbers](https://de.wikipedia.org/wiki/Googol#Googolplex) hard in Python 😄

If my math is correct, they need around 2^{334} bits to store in memory. But hey, it’s going to take a while before computers can manage such numbers down to the last bit with full accuracy, if ever.

Seriously, I think `size_t` is a good measure for what computers can realistically handle in the foreseeable future. That value should be 64 for most of today’s machines and we’re not even close yet to having that much RAM available.

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [August 29, 2024, 2:59pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/5 "2024-08-29T14:59:44Z")

</div>

There are only 10\*\*80 atoms in the observable universe.  
Use that as your limit on memory size.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [August 29, 2024, 4:41pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/6 "2024-08-29T16:41:58Z")

</div>

Definitely agreed with this proposal. It’s _extremely_ unlikely that it would bother anyone.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [August 29, 2024, 4:43pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/7 "2024-08-29T16:43:49Z")

</div>

> [@notatallshaw](#):
>
> It would be a big breaking change for many users (e.g. anyone who ever uses scripts to write Number Theory or Cryptography) and would presumably require a PEP?

I think you misread, this is capping the _size_ of integers to `2 **63 - 1` bits, not their value. Their maximum value would therefore be `2** (2**63 - 1)`.

---

<div class="post-metadata">

**Author:** ![gpshead](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gpshead/32/54_2.png) [@gpshead](https://discuss.python.org/u/gpshead)\
**Post date:** [August 29, 2024, 10:19pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/8 "2024-08-29T22:19:38Z")

</div>

This seems uncontroversial to me. We can never represent exact values such as `int` so large that the RAM to represent it can’t possibly exist.

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [August 30, 2024, 6:53am UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/9 "2024-08-30T06:53:36Z")

</div>

> [@storchaka](#):
>
> This can be implementation limit, not language limit.

The C API is kind of between that, as some other implementations re-implement or emulate it.  
I expect the public version of `_PyLong_NumBits` will return `int64_t`, rather than `size_t`. Which is fine: in a future where it’s needed, we can introduce a new function and let the old one fail.

---

<div class="post-metadata">

**Author:** ![storchaka](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/storchaka/32/217_2.png) [@storchaka](https://discuss.python.org/u/storchaka)\
**Post date:** [August 30, 2024, 7:17am UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/10 "2024-08-30T07:17:24Z")

</div>

> [@oscarbenjamin](#):
>
> I presume that this really only actually changes anything in a 32-bit build because otherwise `size_t` would already be 64 bits anyway right? So basically this is just making the type bigger on 32 bit system so that the code can be simpler?

No, it only affects 64-bit platforms. I just made bit counts 64-bit on all platforms, this solves issues on 32-bit platforms (it required only 0.5 GB to create an integer whose size in bits exceeds size\_t).

What I propose here, is to change the limit on 64-bit platforms. Currently it is about 2^{63} bytes or 2^{66} bits. I propose to reduce it to about to 2^{63} or 2^{64} bits (2^{60} or 2^{61} bytes). Both limits are insanely large and exceed the maximum size of addressable memory on most of current platforms.

> [@oscarbenjamin](#):
>
> Does there exist a supported platform where `size_t` is more than 64 bits?

Even if they exist, Python is not a right tool to work with exbibyte-size integers.  
Even adding two of such integers will take yeasr.

Well, it seems that there are no objections. I am not still sure about using signed or unsigned 64-bit integers, but this detail will be discussed on GitHub and in the C API Workgroup.

---

<div class="post-metadata">

**Author:** ![avi.gross](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@avi.gross](https://discuss.python.org/u/avi.gross)\
**Post date:** [August 30, 2024, 3:51pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/11 "2024-08-30T15:51:29Z")

</div>

\usepackage{mathtools}

The proposal is not outlandish if we look at the realities of implementing anything so large that might not be possible far enough into the future that python as a language will have been superseded.

But do people sometimes, perhaps inadvertently, use larger numbers? Sure.

Consider the typical formular when looking at permutations and combinations where if you want N objects arranged K at a time you blithely use the factorial function as in:

![Screenshot 2024-08-30 114625](https://sea2.discourse-cdn.com/flex016/images/transparent.png)

If some person just wonder how many ways the 10 to the 80th calculated particles in the universe can be arranged in a straight line, they run out of memory and computer time very quickly.

Of course, in some future, it may be possible for very networked computers to collectively share storage for larger numbers and do operations on them, or perhaps you could even somehow use quantum computers to more compactly store larger items.

As noted, a special-purpose extended type could be added in such eventualities.

---

<div class="post-metadata">

**Author:** ![kumaraditya303](https://avatars.discourse-cdn.com/v4/letter/k/a88e4f/32.png) [@kumaraditya303](https://discuss.python.org/u/kumaraditya303)\
**Post date:** [August 30, 2024, 4:16pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/12 "2024-08-30T16:16:56Z")

</div>

> [@storchaka](#):
>
> This is much more than any existing computer have or can have in observable future. In both Intel64 and AMD64, the 52-bit physical address limit is defined in the architecture specifications.

Note that with Intel level 5 paging it can use 57-bit virtual address space.

I agree that we should limit this based on how much RAM can represent nowadays.

---

<div class="post-metadata">

**Author:** ![franklinvp](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/franklinvp/32/11142_2.png) [@franklinvp](https://discuss.python.org/u/franklinvp)\
**Post date:** [August 30, 2024, 4:48pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/13 "2024-08-30T16:48:02Z")

</div>

Just a clarification, such that judgement of the idea is not done in terms of the following incorrect statement:

> [@gpshead](#):
>
> This seems uncontroversial to me. We can never represent exact values such as `int` so large that the RAM to represent it can’t possibly exist.

Every individual number, no matter how large, can be represented exactly and only one bit is needed for that. There is no such thing as

> `int` so large that the RAM to represent it can’t possibly exist

What requires space is a structure that can represent (has different states for) every one of a very large collection of numbers.

It is not inconceivable to have `int` represent every single integer up to some magnitude while after that switch to an encoding that is only able to represent some, but not all integers beyond that point. The same happens everywhere. We do not have representations for all integers. There are recipes (like positional system) that would do so, but all of them under the assumption “provided enough space” that eventually fails. But even past that failure, switching to different representations (2^{22222222}, 2^{2^{2^{2}}}, 2\uparrow\uparrow\uparrow\uparrow\uparrow\uparrow\uparrow2, \operatorname{TREE}(\operatorname{TREE}(2))) one can hit larger values, although not all, and do _some_ operations with them too, but not all.

I cannot judge if the same is expected/desirable in the near future of Python but all of these are phenomena that currently exist in Python:

1. Changing internal representation after some value. `int` does that.
2. Not all real numbers having representatives: `int` and `float` do that.
3. Not all operations are valid: `1 // 0` and `str` of large `int`, for example.

No opinion either way. I only wanted that choices are made based on correct premises.

---

<div class="post-metadata">

**Author:** ![oscarbenjamin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/oscarbenjamin/32/1209_2.png) [@oscarbenjamin](https://discuss.python.org/u/oscarbenjamin)\
**Post date:** [August 30, 2024, 5:47pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/14 "2024-08-30T17:47:54Z")

</div>

> [@storchaka](#):
>
> Well, it seems that there are no objections

I don’t think anyone is objecting but I do have a question that went unanswered:

> [@oscarbenjamin](#):
>
> A more realistic concern: are there any compatibility implications for downstream extension modules resulting from this change?

It is not clear to me from the PR diff whether it is all just changes to internal types that don’t touch any public (or non-public) API that downstream code might be using. It is not unheard of to access the internals of PyLong because it is slow ([fix here](https://github.com/python/cpython/pull/121339)) to convert large integers otherwise.

---

<div class="post-metadata">

**Author:** ![jeff5](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jeff5/32/1518_2.png) [@jeff5](https://discuss.python.org/u/jeff5)\
**Post date:** [August 30, 2024, 7:20pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/15 "2024-08-30T19:20:13Z")

</div>

> [@storchaka](#):
>
> This can be implementation limit, not language limit.

All the arguments do seem to be informed by implementation details, so I think it must be an implementation limit.

Obviously asking for the return of `sys.maxint` would be a mistake here 🤡. Do we need something like `int.maxbits` or `sys.maxintbits`?

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [August 30, 2024, 7:23pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/16 "2024-08-30T19:23:24Z")

</div>

Maybe `sys.int_info.maxbits`?

---

<div class="post-metadata">

**Author:** ![gpshead](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gpshead/32/54_2.png) [@gpshead](https://discuss.python.org/u/gpshead)\
**Post date:** [August 30, 2024, 8:27pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/17 "2024-08-30T20:27:32Z")

</div>

Why bother exposing the “limit” anywhere? Nobody can use `int` values even approaching that large anyways. The limit is _effectively_ already in place, it’s just that you currently potentially wind up getting a MemoryError or other not quite as explanatory exception.

---

<div class="post-metadata">

**Author:** ![tjreedy](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/tjreedy/32/137_2.png) [@tjreedy](https://discuss.python.org/u/tjreedy)\
**Post date:** [August 30, 2024, 9:35pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/18 "2024-08-30T21:35:00Z")

</div>

‘Integer’ is a mathematical abstraction denoting a particular doubly infinite ordered set. Unless ‘Python integer’ redundantly means either ‘integer’ or ‘Python int’', I think it should mean “Python object with an integral value”. And that should not have a hard limit, and indeed such could hardly be imposed on user classes.

Python ints are an implementation that can represent all integers between lower and upper bounds. The practical limits depend on the machine and currently available limits. The theoretical limits for the implementation as is on 64 bit machines is the address space size. The proposal is to divide by about 8 so the bit limit, rather than byte limit, will fit in an int64, so as to simplify internal C int code. So this would be an implementation limit.

Since this will have no effect on practical limits for decades, at best. I think go ahead. If any extensions dig deep enough into the implementation to be affected, I think they should adjust. But best to get this into .a1 to find out as soon as possible.

Unless there is already a sys.int\_bit\_limits or equivalent, there would need to be a good reason to expose one now.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [August 30, 2024, 9:46pm UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/19 "2024-08-30T21:46:44Z")

</div>

It is useful information that contributes to description of `int` and there is a natural place for it. All `<number>_info` define bounds if they exist, in both Python and `numpy` (possibly many others too).

How often do other values in `sys.int_info` get used in code? This would probably be one of the more useful ones in there.

Also, it would be a small safeguard against the unknown and unforeseen at minimal cost.

---

<div class="post-metadata">

**Author:** ![skirpichev](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/skirpichev/32/10996_2.png) [@skirpichev](https://discuss.python.org/u/skirpichev)\
**Post date:** [August 31, 2024, 3:45am UTC](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302/20 "2024-08-31T03:45:23Z")

</div>

> [@franklinvp](#):
>
> if the same is expected/desirable in the near future of Python

This looks more suitable for specialized computer algebra system and far beyond scope of a general purpose language like Python. Even algorithms used for integer arithmetic here are school-level (with few exceptions like Karatsuba multiplication). Hardly this will be changed, unless CPython switch to external bignum library.

> [@oscarbenjamin](#):
>
> It is not unheard of to access the internals of PyLong because it is slow ([fix here](https://github.com/python/cpython/pull/121339)) to convert large integers otherwise.

I think that this use case isn’t affected by OP PR.

[Next page](https://discuss.python.org/t/imposing-a-hard-limit-on-python-integers/62302.md?page=2)