Project run on Python 3.11.0b1 is 1.6x slower than Python 3.10?

Python Help
1

One of my projects available on GitHub (WolfBot) is roughly 1.6x slower when run on Python 3.11.0b1 vs 3.10.0, and I’m not sure why:

When I profile the application, I get the following results:

3.10.0: 6991718 function calls (6535674 primitive calls) in 3.761 seconds
3.11.0b1: 7293443 function calls (6837325 primitive calls) in 6.017 seconds

The app essentially runs a constraint satisfaction solver and primarily uses dataclasses and enums, but I don’t see any recent changes in the changelog that would cause this slowdown, especially since 3.11 is advertised as a performance boost on most benchmarks.

The code is available at: GitHub - TylerYep/wolfbot: One Night Ultimate Werewolf: AI Edition
I appreciate any pointers here!

Flamegraphs
3.10.0

log3.10
log3.101768×1589 371 KB

3.11.0b1

log3.11
log3.111568×1589 375 KB

2

What exactly is 1.6x slower?
What is the command you are running, and which builds of 3.10 and 3.11 are you using?

We are lacking benchmarks that use either dataclasses or enums, so if you would like to submit your program as a benchmark, that would be great.
github.com/python/pyperformance

3

The 1.6x slower part is running python profiler.py (runs CProfiler in the file) after cloning my repository and installing the requirements.

I am using pyenv’s builds of Python 3.10.0 and 3.11.0b1 (CPython).

Side discussion - how would I add my project as a benchmark? Is it possible to add my project as a submodule? Or do I need to copy everything over as a one-time export?

4

Interesting. Have you tried running without cProfile? It’s possible that running code under cProfile has slowed down in 3.11 (I can’t imagine by that much though), but if you disable cProfile and it’s still that slow, we may have other problems :).

To add a benchmark, you need to create a folder here, with the appropriate pyproject.toml and requirements.txt. For the code you could copy paste from one of the other benchmarks. It looks something like this.

5

@kj0 Oh, you’re totally right - Running the code without cProfile sees the expected speedup:

3.10.0
python main.py -n 100
Time Elapsed: 3.016459534992464
python main.py -n 500
Time Elapsed: 10.3476728789974

3.11.0b1
python main.py -n 100
Time Elapsed: 2.417181205993984
python main.py -n 500
Time Elapsed: 8.855358878034167

It looks like cProfile is the problem then.

6

The cProfile version is also faster on 3.11.0b1. You just need to compare the right numbers. Dismiss my comment; I misunderstood your post. Sorry for the noise.

7

Code run with cProfile performance regression in 3.11 · Issue #93381 · python/cpython · GitHub has been opened to attempt to investigate further and improve code running under cProfile’s performance. Thanks for this report Tyler.

4 Likes
8

Thanks to the excellent work by Mark and Dennis in this GitHub issue, the performance degradation in 3.11 is largely mitigated. The fixes are coming in 3.11b4.

I did a benchmark of the same program in 3.10 vs 3.11b4. It’s now only ~15% slower! (versus 60% slower in 3.11b3).

3.10 profiler.py
Time Elapsed: 2.1907681000002412

3.11b4 profiler.py
Time Elapsed: 2.4747707999995328
4 Likes