How should coverage.py treat deferred annotations?

Core Development
1

In 3.14, annotations are deferred: they are compiled as separate code objects that will be run only when needed. Coverage.py sees that there is code there, but it usually isn’t run. It reports on missing lines of code, but for the most part it’s fine that the lines aren’t run. What can/should coverage do about this?

As an example:

class X:
    x: int

is reported as missing line 2, because int is compiled as a function but is never run.

The coverage.py issue:

4 Likes
2

It is right to say it’s uncovered code. In 3.14, an exception in evaluating the annotation is invisible by default.

class X:
    x: collections.Counter  # Forgotten import is no longer detected

Whether that’s actually meaningful feedback to the developer though is another question.

3

I would suggest having a simple way to ignore all such code objects (assuming they can be identified reliably), but with the option to check them in case a runtime type checker or other verify is available and in use. (Other verifier might mean a pytest plugin that verifies that all annotations at least evaluate to code objects or something)

4

Exactly. At runtime, the annotations are rarely evaluated. There is no error from running this code. There will be an error from a type checker, which is enough.

Perhaps what I need most is help with how to know what code came from annotations so it can be ignored.

5

I agree that most users of coverage probably won’t care if their annotate functions aren’t executed, so you should exclude them by default.

As for how to recognize them, they are in a function named __annotate__. If recognizing them by name is too fragile, perhaps we could add a bit to the code object flags.

7 Likes
6

It’s a reserved-to-Python name, so I guess it isn’t fragile.

7

It seems to work well to ignore code objects named __annotate__, thanks!

4 Likes
8

Perhaps unittest (or inspect) could gain an easy way to recursively check all code annotations in a module or package actually run?

Then coverage.py warning about missed evaluation could either be turned off if they’re only required to be valid under static analysis, or the extra test case to check them added if the project does want to support runtime execution.

2 Likes