Coverage report in Github CI for standard library

How does everyone feel about adding a new CI job with coverage report using coverage.py?

It came to me after a bug report was filed against a piece of code I wrote but missed the tests for, mostly because other projects that I work on has a CI to remind me of missing coverage.

I don’t know if others would be okay with making it a mandatory check, but it would be nice to at-least have the information available for the interested module maintainers.

We could use a 3rd party service, like codecov.io, which other sub projects under Python seem to be using. I personally haven’t used them on any of my projects, but I don’t enjoy the very verbose output from their bot in every single PR. Not sure if it can be customized to just a regular Github check with URL to details or not.

I would prefer to just go with diff-cover library which can analyze the git diff of the PR with the coverage’s XML report to determine if any lines in the PR are missing the tests. We have been using diff-cover with Mailman project for quite some time and it has been great!

I am totally open to any other suggestion that y’all might have.

Thoughts?

FYI, we are using codecov.io already.
See https://codecov.io/gh/python/cpython

I think that a mandatory check might be a bit too obstructive, but I would definitely welcome some form of GitHub PR integration with the existing codecov.io, if that’s available. Otherwise, some other form of code coverage service with GitHub PR integration would work as well.

The coverage run takes a long time to execute.

1 Like

I did not know about this, thanks!