Buildbot statistics

I am gathering some statistics on the CPython buildbots to try to understand what are the platforms/builders/workers/files that are more likely to fail when running the tests. While I figure out how to obtain more interesting information (like logs, which the REST API makes painful to obtain) here is a graph of the failures in every stable builder in the last 60 days:

2 Likes

What makes FreeBSD 10.x such an outlier even compared to the other FreeBSDs?

I think that builder is particulary slow and is building master so it triggers almost all race conditions in socket, test_multiprocessing_spawn, queues…etc. Also it can be long standing issues that took a lot to fix or some bad configuration in the buildbot. For some reason, the logs are gone in the buildbot master so is difficult to tell. There is some information in the mailing list as we answer to the failure emails when we open a new issue.

This is the buildbot:

2 Likes

It seems like we need a core dev specialized in FreeBSD. Do you know any potential candidate? It would be great to have someone to handle FreeBSD specific issues. I’m not talking about random bugs spotted by the “FreeBSD 10.x shared” buildbot. But bugs which are obviously connected to FreeBSD. They are more and more of them. For example, I recall a test which started to fail because of a change in the FreeBSD kernel: the multiprocessing change using CMSG_SPACE().

Or maybe we should remove FreeBSD from the supported platform to reduce the burden on the Python maintenance? I mean to move FreeBSD to the list of “best effort support” platforms. https://pythondev.readthedocs.io/platforms.html

Probably one of them should happen, but as to which one I’m not sure. If someone doesn’t already obviously exist for this then we should probably drop it as a supported platform.