Special semantics for generator context managers

[bswck]

@ImogenBits

Of course, you can define a new type of generators that yield exactly once and then have the contextmanager decorator take that type as an argument. But I don’t think that that’s really in the spirit of Python typing. We don’t have a type for generators that yield exactly 17 times either or one for functions that access global variables or one for iterators that never stop. That kind of program behaviour just isn’t something that the type system is trying to model.

Fair, I think I’m convinced.

To clarify, the reason why I believed this should be fixed by type checkers is because:

• What led me to creating this thread is incorrect reachability inference caused by something that leads to a runtime type error not caught by type checkers. I’ve described it in much more detail here: Support suppressing generator context managers. There are also other people arguing that this is a linter’s responsibility rather than type checker’s, which makes perfect sense.

  Looking at the symptoms of the problem I naturally assumed that this is type checker’s responsibility, and I haven’t been alone in this: just now, I’ve flicked through @Jelle’s repo that collects examples of unsoundness in the Python type system (i.e., code that produces a runtime error that the type system should catch, but that is accepted by type checkers), and found that exactly that same case has been reported and accepted some time ago: Add context manager examples by decorator-factory · Pull Request #18 · JelleZijlstra/unsoundness · GitHub

• Fixing that problem with a type checker would help detect that in existing codebases without having to migrate to any other solution such as specifying redundant keyword arguments just to exercise the right overload with no runtime benefit of using them.

These two things were why I concluded that special semantics for generator context managers would be a good idea in general, since type checkers don’t cover other aspects of control flow in generator context managers as well. I’ll admit that I’m not extremely experienced with type checker internal building blocks. I know that they would do some control flow analysis, and I think I found the right spot in mypy to start the patch to address the original problem. I understand though that I might be naive in thinking that this is the right place to address it.

I’d prefer a solution that would report an error on the generator suppressing context managers problem just as soon as someone affected updates their toolchain—not having to migrate the entire codebase. If linters can do that, that would be awesome.

CC @mikeshardmind