Coverage report in Github CI for standard library

[vstinner (Victor Stinner)]

@ammaraskar

Pinging @vstinner around https://bugs.python.org/issue40993

Looks like we recently disabled codecov for pull requests. It looks like it’s a feature request here and I’ve made some use of it in the past to show that a particular piece of C code was dead and unreachable.

Arguably we get pretty similar information from it running after merge but does anyone here have any strong opinions on running it on pull requests? It does add a fair bit of overhead in terms of CI resource usage and run-times.

Hi Ammar, I disabled coverage on pull requests because for an unknown reason, Travis CI started to take up to 50 min to report “tests passed” status to GitHub. Since Travis CI is now a mandatory CI, it prevents to merge a PR.

After I disabled coverage, it takes again less than 30 min to be able to merge a PR. The minimum time to merge a PR has a direct impact on our workflow performance.

See also Travis CI doesn't report its status or doesn't run on Python pull requests · Issue #371 · python/core-workflow · GitHub It seems like Travis CI behavior changed last weeks, and I got multiple issues with it.

Moreover, when I asked “who use coverage?” in April, nobody replied and nobody knew how it is supposed to work: https://bugs.python.org/issue40237

So IMO coverage can wait until a change is merged into master.