Diego from Arm Ltd here. I’m assessing the status of CPython and its ecosystem on aarch64 and during my investigation I bumped into speed.python.org and pyperformance package.
I really like integration between pyperformance and codespeed and I was able to setup an internal instance within Arm with aarch64 results.
Being Arm architecture present on all mayor cloud providers and Python a critical programming language for cloud workflows, it would be great if we can integrate aarch64 performance results on speed.python.org.
Pointless to say that we are more than happy to help out with both development and by providing infrastructure (in some form) to achieve this goal.
Happy New Year! I had in the back of my mind to write you about how ARM could help Python, and I got overwhelmed with other things on my TODO list. It’s great that you managed to set up an ARM instance of speed.python.org. Is there a chance you can make that public? It doesn’t have to be on a *.python.org URL for now. I’d like to see if we can learn something by comparing the ARM results to the x64 results.
Mostly just that speed.python.org has exactly one maintainer and he’s not got a lot of time for it, so anything that would require coordinating with him would just be extra red tape that I’d like to save you from.
Just a quick update. Things are progressing well but before making anything public, we’ve been running some internal CI infrastructure to cross validate some data.
I will be off for the next three weeks but as soon as I come back, I’ll continue working on this.
First of all I want to apologise the delay: I’ve been busy with work, EuroPython 2023 (organising it, sponsor it and presenting) and I took annual/parental leaves on the way. So the whole thing took more than expected.
I’m happy to announce that I was able to replicate the benchmark infrastructure that shows CPython performance across Arm CPUs: Neoverse-N1, Neoverse-N2, and Neoverse-V1.
Older branches should be more stable in terms of metrics so we preferred to focus on newer ones. If you think we need to include older, let us know and we will do it.
We’ve had this infra in place for more than a month now, (since August 15th) because we needed to solve a few of issues with the infrastructure. Until September 14th commits benchmarked on machines might be different depending on the machine: this because the machines were available at different times (with a few hours difference) hence picked the latest commit of the branch at the time of execution of the job.
Since September 14th you should see more consistent data: we first pick the commit and then we schedule the job on different machines with this specific commit. Data might be uploaded at different times but when it arrives it is consistent with other machines.
The web server and the database are running on an aarch64 machine sponsored by the Works on Arm project and the benchmarks are running on physical machines behind the Arm firewalls. Jobs are managed by our own Jenkins instance.
I think that’s it for now, have a look and please share your thoughts.
Between August 19 and August 27 the time this took shot up from 100ms to 800ms. Almost everything else was pretty stable. I didn’t see a similar jump for this benchmark on speed.python.org: Python Speed Center : Timeline
