# Using Github problem matchers to catch warnings early **URL:** \ **Category:** Core Workflow\ **Created:** [May 23, 2020, 12:03am UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254 "2020-05-23T00:03:13Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ammaraskar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ammaraskar/32/32_2.png) [@ammaraskar](https://discuss.python.org/u/ammaraskar)\ **Post date:** [May 23, 2020, 12:03am UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/1 "2020-05-23T00:03:13Z")
Github semi-recently introduced a new feature to Action workflows that allows you to install little regex handlers for logs that create pull request review comments for matches. These are called [problem matchers](https://github.com/actions/toolkit/blob/master/docs/problem-matchers.md#problem-matchers) and are great for prominently surfacing things like linter failures, warnings and other issues that won’t necessarily fail the build. They seem to be a port of an existing feature in VSCode extensions: [https://code.visualstudio.com/docs/editor/tasks#\_processing-task-output-with-problem-matchers](https://code.visualstudio.com/docs/editor/tasks#_processing-task-output-with-problem-matchers) I’m particularly excited about the compiler warnings matchers since they seem to sneak in and then every once in a while someone makes a pull request to remove them. People that develop on Linux don’t tend to check the Windows warnings and vica-versa, and as long as the CI results are green no one really digs into the logs. This would allow them to be caught early in the review process. @csabella suggested I post about this to gauge interest and make it more discover-able. I’ve created three pull requests with problem matchers for: ## Sphinx ![image](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/d/d25d6f0805f1acafc6d5ef85109554ea1eb755ac.png) > Catches errors and warnings from sphinx documentation builds. As a bonus this allows for a workflow whereby people making and reviewing documentation pull requests can fully work entirely from Github. Between this problem matcher and the netlify doc preview, we can be fairly sure the changes are good to go. (This one is not as important since [https://github.com/python/cpython/commit/6aabb63d96845b3cb207d28d40bf0b78e171be75](https://github.com/python/cpython/commit/6aabb63d96845b3cb207d28d40bf0b78e171be75) which adds `-W`, warnings-as-errors to the sphinx build but still provides errors in a more convenient place). ## GCC ![image](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/f/f59345d1840720e27ef9cddf2717489e69d37529.png) > Catches warnings and errors generated from gcc. This is a straight forward port from the [vscode-cpptools matcher](https://github.com/microsoft/vscode-cpptools/blob/a8285cbc0efb5b09c2d2229b0e0772dcb3b602df/Extension/package.json#L76-L94). ## MSVC ![image](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/e/e57f8693f408062faffc176d2087f8348e176105.png) > This catches warnings and errors generated by our Windows builds. It is pulled straight out [vscode itself](https://github.com/microsoft/vscode/blob/d5386d323342076c82a0e2331df5c603d980f059/src/vs/workbench/contrib/tasks/common/problemMatcher.ts#L1208-L1216). # Implementation I’ve created three Github workflow actions on the marketplace for each of these matchers: - [https://github.com/marketplace/actions/sphinx-problem-matcher](https://github.com/marketplace/actions/sphinx-problem-matcher) - [https://github.com/marketplace/actions/gcc-problem-matcher](https://github.com/marketplace/actions/gcc-problem-matcher) - [https://github.com/marketplace/actions/msvc-problem-matcher](https://github.com/marketplace/actions/msvc-problem-matcher) This allows them to be used by the wider Github community and hopefully have a wider set of eyes looking at them to catch any issues. Despite the error formats being relatively stable, relying on me to maintain the actions might be a little queasy for some of the core developers. In this case, it’s possible to commit the matchers ([here’s an example of one](https://github.com/ammaraskar/gcc-problem-matcher/blob/master/gcc_matcher.json)) under the `.github` folder on our repo and just use the local versions, I’d be happy to change over to that method if people feel necessary. ---
**Author:** ![zware](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zware/32/58_2.png) [@zware](https://discuss.python.org/u/zware)\ **Post date:** [May 23, 2020, 3:08am UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/2 "2020-05-23T03:08:25Z")
I think this sounds good. If it’s really just the matcher `.json` files that would need to be added to the `.github` folder, I think I would opt for that just for discoverability. I’m not concerned about maintenance of these matchers; they appear simple enough to likely not need much if any for quite some time. However, if these warnings are going to start appearing on every pull request, we likely want to fix the any existing warnings and enable any warnings-as-errors options on the CI whose output these checks parse when we add them since otherwise people are likely to be confused about why their typo fix in a docstring is causing build warnings 🙂 ---
**Author:** ![pablogsal](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pablogsal/32/32246_2.png) [@pablogsal](https://discuss.python.org/u/pablogsal)\ **Post date:** [May 24, 2020, 12:57am UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/3 "2020-05-24T00:57:11Z")
I think this would be a great improvement. I have fixed many Windows and Linux compiler warnings in the past. I agree with @zware in that I think I would prefer to have these JSON files living in the repo. @ammaraskar Could you make a PR adding the relevant files so we know how many warnings we have currently? Feel free to add me as a reviewer to such pr 😉 ---
**Author:** ![EpicWink](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/epicwink/32/17968_2.png) [@EpicWink](https://discuss.python.org/u/EpicWink)\ **Post date:** [May 24, 2020, 9:23pm UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/4 "2020-05-24T21:23:13Z")
> [@zware](#): > > However, if these warnings are going to start appearing on every pull request, we likely want to fix the any existing warnings and enable any warnings-as-errors options on the CI whose output these checks parse when we add them since otherwise people are likely to be confused about why their typo fix in a docstring is causing build warnings 🙂 Would you want to check the difference between master’s warnings and the current PR’s warnings and only put the new warnings in the PR’s comment? An implementation is to commit (all of) the warnings as a text file with every PR, then the current PR can run `diff` on that file to get the text to display in the comment ---
**Author:** ![ammaraskar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ammaraskar/32/32_2.png) [@ammaraskar](https://discuss.python.org/u/ammaraskar)\ **Post date:** [May 24, 2020, 9:51pm UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/5 "2020-05-24T21:51:35Z")
> [@EpicWink](#): > > Would you want to check the difference between master’s warnings and the current PR’s warnings and only put the new warnings in the PR’s comment? Sorry, I was kinda being hand-wavey when I said comments, in reality they’re Github check annotations: [About status checks - GitHub Docs](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-status-checks#types-of-status-checks-on-github) This is actually handled in a much nicer way by Github whereby existing warnings in files that you didn’t touch show up like this: ![image](https://us1.discourse-cdn.com/flex002/uploads/python1/original/2X/5/542f3a86af3c561b23b65b9c8e47af947b12055a.png) That doesn’t mean we shouldn’t try to quash any existing warnings but it does make the experience not feel overwhelming for new contributors. My next steps will involve creating PRs checking in the matcher JSON files. ---
**Author:** ![ammaraskar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ammaraskar/32/32_2.png) [@ammaraskar](https://discuss.python.org/u/ammaraskar)\ **Post date:** [May 24, 2020, 10:53pm UTC](https://discuss.python.org/t/using-github-problem-matchers-to-catch-warnings-early/4254/6 "2020-05-24T22:53:19Z")
Alright I’ve updated all three PRs to check-in the matcher files: - MSVC - [https://github.com/python/cpython/pull/18532](https://github.com/python/cpython/pull/18532) - gcc - [https://github.com/python/cpython/pull/18567](https://github.com/python/cpython/pull/18567) - Sphinx - [https://github.com/python/cpython/pull/20325](https://github.com/python/cpython/pull/20325) The only warnings are from the MSVC build, which is what I anticipated since most core-devs exclusively build on Linux. You can see all the warnings for the MSVC build on the bottom, `Unchanged files with check annotations` part of [https://github.com/python/cpython/pull/18532/files](https://github.com/python/cpython/pull/18532/files)