About speed.python.org and aarch64

Core Development
21

Hello everyone!

I bring good news with me! @ambv and I have been working in the background to properly integrate AArch64 benchmark results to speed.python.org.

Today I can confirm that:

  • daily benchmarks are running on AArch64
  • commits since May 2023 have been backfilled
  • the speed-python and arm-neoverse environments have now the same commits since May 2023.
  • the hack that mapped main and master branches have been removed and existent data have been migrated to use only main branch.
  • cleaned up old and abandoned data in the database

speed.python.org is far from to be perfect and we have already a list of issues that we’d like to implement: Issues · python/codespeed · GitHub

Have a look and give your feedback. Of course contributions are more than welcome :slight_smile:

I want to thank @ambv for his support and patience in helping me enabling AArch64 metrics.

Cheers

4 Likes
22

Hi Diego,

Thanks for your work on speed.python.org! I have a request for readability of the comparison charts. Here’s a chart I’m interesting (comparing 3.12 to main, basically):

Unfortunately it looks like the histogram bars are only one pixel wide, which makes them basically impossible to read for me. (I’m not blind, but my eyesight is definitely declining as I age.) Could you tweak the plotting parameters to make them a bit more visible?

I also found that if I change the Environments selection from just arm-neoverse to just speed-python, while keeping the other selections the same (3.12 vs. main) I get an error about a missing baseline. Is that expected?

2 Likes
23

Issue created: Improve readability of the comparison charts · Issue #51 · python/codespeed · GitHub

I think this depends on the data available in the database. I’ve just checked and this is what’s happening.
Basically the two environments (speed-python and arm-neoverse) have results for different revisions in the 3.12 branch.
What it matters for the comparison page is that the latest commit of a branch has results attached for a specific environment.
These are the revisions we have for 3.12

58fdd15d5a70b0274ed5d4fc491700138aa51cdf CPython:3.12 Jan. 22, 2024, 8:08 a.m.
103728dc151e3797b1b94c635b0bcd9b52c03217 CPython:3.12 Nov. 24, 2023, 12:15 p.m.
86db104eebd4c3a914840f02e1ff2b5088180f15 CPython:3.12 Nov. 23, 2023, 5:07 p.m.
0fb18b02c8ad56299d6a2910be0bab8ad601ef24 CPython:3.12 Oct. 2, 2023, 6:48 a.m.
e5b6744f300553fe2591f73b59a88e8c6f5fa7e3 CPython:3.12 Oct. 29, 2023, 2:43 p.m.
8296b53027b15e688f028f4d7c44f3d76d283416 CPython:3.12 Oct. 2, 2023, 10:48 a.m.

arm-neoverse has results for the commit id 58fdd15d5a70b0274ed5d4fc491700138aa51cdf whilst for speed-python the latest commit with results is e5b6744f300553fe2591f73b59a88e8c6f5fa7e3 (which is not the latest of the 3.12 stored in the database)
For this reason it complains there is no baseline for it. Similar thing happens for branch 3.11.

The solution here is to sync benchmarks for release branch to be the same revisions. Whenever a tag is created we could run pyperformance and upload data there. In theory those revision should be tagged in codespeed so the user can make use of them in the timeline tab (see baseline)

On my side I can take all the tags of 3.11 and 3.12 and run benchmarks against them. The same should be done on the x86 machine but someone else (@ambv I invoke you) should run it.

1 Like
24

Thanks for looking into it.

Regarding the logic related to "latest revision, maybe there’s some improvement that can be made to the software there too, rather than just manually syncing the database?

25

Yes, indeed. Ideally we should be able to specify commits instead of implicitly always taking the latest one. This will allow also to compare random commits on the same environment as well.
At the moment this is possible only if the results are present against the latest commit on a branch.

There is an issue regarding the comparison page: Redesign the comparison screen · Issue #26 · python/codespeed · GitHub it would be nice to write down some requirements of what it would look like.