Recursion limits

In 2016 I wrote a script which measures how many recursive operations can Python do before crash (Fastcall uses more C stack · Issue #73044 · python/cpython · GitHub, Reduce stack consumption of PyObject_CallFunctionObjArgs() and like · Issue #73056 · python/cpython · GitHub). It was used to find and fix some regressions in 3.7. It is now available at https://github.com/serhiy-storchaka/python-misc/blob/main/stack_overflow.py.

It is interesting to look how stack consumption was changed with Python versions. Here are results for the debug build on Linux for Python from 3.3 to future 3.13. Release builds have lesser stack consumption and allows deeper recursion. On Windows the stack size is smaller.

3.33.43.53.63.73.83.93.103.113.12main
test_ast_parse871541308041308771308771308041308771047311046581308772994 (104684)2992 (104684)
test_chain65397174459261762261616261762261762261762261762261762174459261762
test_compile32693104658130804130804130804872278722774771746982995 (87203)2995 (87251)
test_deepcopy13908176830565416234622967097072unlimitedunlimitedunlimited
test_eq2614252327523275236343650436134361340243402431498 (34909)1498 (34909)
test_filter1046581308771308041047311046581047317477174771104731104658104731
test_hash65397104731104658104731104731104731130950104658104731104731104731
test_islice1045851744591744591744591744591744591746051744591746057469874771
test_json_dump4017058149581485814858148581485814940243523631497 (52342)1497 (52340)
test_json_load4354058112581495814958149581495232758149476031497 (43646)1497 (43622)
test_map653971308771308777469874771747716547065397654336543365433
test_marshal_dump19991999199919991999199919991999199919991999
test_marshal_load19991999199919991999199919991999199919991999
test_partial5811287227872276543365433654707477165397654701497 (58151)1497 (58151)
test_pickle_dump326934760347604475674361343613402802908937380749 (32711)749 (32711)
test_python_call231488718176736878128180887187249874498 (8304)498 (8049)
test_python_call_keyword231488678176737678128176887587259875498 (8310)498 (8055)
test_python_function261613779137791308412463124631343014151unlimitedunlimitedunlimited
test_python_generator30792493420136227682179921799227681636218684unlimitedunlimited
test_python_getitem2294817675866978818091821189613084unlimited107070 (unlimited)107070 (unlimited)
test_python_iterator216170726460562769838445109081046915870749 (11897)749 (11897)
test_python_method261513779137701308412463124631342114151unlimitedunlimitedunlimited
test_repr5232765397654336539765433654695814965397654331496 (52342)1496 (52342)
test_yield_from309624934201362275021799217992275014535163627477165397

As a rule, stack consumption remained the same. In some cases, it got better with each version. I highlighted in bold cases when it got worse.

3 Likes

Many regressions in 3.12 are due to new hard limit (C_RECURSION_LIMIT). In parenthesis I added results for builds with removed hard limit.

1 Like

Thanks for compiling the table, those are interesting results. Some of those values appear to be very low for 3.12 and even in the range where normal applications (e.g. recursive parsers) could run into those limits. I wonder why C_RECURSION_LIMIT was set to such a low value (1500 in the current repo).

Aside: The table is cut off on the right in the normal browser view (it does show in the emails Discourse sends). You can see the full table by hovering over the post and then clicking the expand button on the upper right:

image

4 Likes

Here is a smaller table, for 3.10+

3.103.113.12main
test_ast_parse1046581308772994 (104684)2992 (104684)
test_chain261762261762174459261762
test_compile74771746982995 (87203)2995 (87251)
test_deepcopy7072unlimitedunlimitedunlimited
test_eq40243402431498 (34909)1498 (34909)
test_filter74771104731104658104731
test_hash104658104731104731104731
test_islice1744591746057469874771
test_json_dump40243523631497 (52342)1497 (52340)
test_json_load58149476031497 (43646)1497 (43622)
test_map65397654336543365433
test_marshal_dump1999199919991999
test_marshal_load1999199919991999
test_partial65397654701497 (58151)1497 (58151)
test_pickle_dump2908937380749 (32711)749 (32711)
test_python_call87249874498 (8304)498 (8049)
test_python_call_keyword87259875498 (8310)498 (8055)
test_python_function14151unlimitedunlimitedunlimited
test_python_generator1636218684unlimitedunlimited
test_python_getitem13084unlimited107070 (unlimited)107070 (unlimited)
test_python_iterator1046915870749 (11897)749 (11897)
test_python_method14151unlimitedunlimitedunlimited
test_repr65397654331496 (52342)1496 (52342)
test_yield_from14535163627477165397
6 Likes