It came up recently that the conformance suite asserts that type checkers should not emit diagnostics in if not TYPE_CHECKING blocks (or the else clause of if TYPE_CHECKING blocks), but this is not clearly specified anywhere. I think it’s the correct behavior, and we should clearly specify it. if not TYPE_CHECKING expresses clear intent to do things the type checker should just ignore.
Bringing it up for discussion here since that’s the process for a spec change. Does anyone have strong use cases for wanting some kind of type diagnostics inside if not TYPE_CHECKING?
For a contrarian perspective: if not TYPE_CHECKING is useful as a block which hides the types of the symbols to be checked against from the rest of the scope, rather than something which completely turns off type-checking.
I find myself using its counterpart, if TYPE_CHECKING, far more frequently as a kind of .pyi-within-a-.py, and never actually wanted type checking to be off. Anything I’d like to hide under not TYPE_CHECKING frequently still benefits from type-checking for internal consistency within the same not TYPE_CHECKING block; if I wanted to actually stop type-checking something, I’d reach for one of the multiple ways which already effectively turns type-checking off at all sorts of scopes and situations, such as var: Any, @no_type_check, __getattr__(self, name: str, /) → Any, def f(*args: Any, **kwargs: Any), or just not even run a type-checker.
I sometimes reach for a mypy plugin to handle items under not TYPE_CHECKING that require special reporting. Although it is unlikely mypy will prevent error diagnostics under not TYPE_CHECKING with a plugin, it would be unfortunate if the spec mandates this and mypy follows suit in the future.
EDIT: To propose something concrete for the spec, I’d prefer it to say something along the lines of
Type-checkers must NOT account for symbols defined under not TYPE_CHECKING or the else clause of if TYPE_CHECKING: ...\nelse: ... for establishing symbol existence or symbol types in the scope:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
a: int = 1
else:
a: str = ""
b: str = ""
reveal_type(a) # Note: `builtins.int`
if not TYPE_CHECKING:
a: str = "" # Reassignment not seen by type-checkers
c: str = 1 # Spec should be silent on whether type-checkers do anything here
reveal_type(a) # Note: `builtins.int`
reveal_type(b) # Error: `b` is not defined
reveal_type(c) # Error: `c` is not defined
I don’t know about completely forbidding type checkers from emitting any diagnostics here. There are cases where they probably should be allowed to emit diagnostics still. The most obvious one is if there’s invalid syntax recognized by a type checker.
I think a better rule here would be for type checkers to suppress any type related diagnostics within that condition, but allow any other diagnostics a typechecker has chosen to provide at their own discretion.
I agree; I mentioned “type diagnostics” specifically in the final question in my OP, but I guess I didn’t clarify in the first paragraph.
I’m not sure if we can provide a precise definition for which diagnostics are “type diagnostics”, but it also maybe isn’t necessary, if we provide some concrete examples in the conformance suite.
Yes, I think this is clearly implied by the existing language (and conformance tests) in the spec. It’s a necessary consequence of treating TYPE_CHECKING as statically True. But it wouldn’t hurt to be extra clear about it.
It seems like currently all type checkers effectively “turn off type checking” in if not TYPE_CHECKING blocks. Until recently ty did not, and we had several users surprised that we would emit any type diagnostic in a region which (in their mind) they had clearly asked the type checker to ignore. So I’m not sure that yours is the majority expectation – and I think this is an area where we have to pick some behavior.
Clearly the conformance suite can’t mandate this, since it doesn’t use mypy plugins. So I think this would always remain up to mypy’s discretion. And if we clarify that it’s only “type diagnostics” which are disabled in these regions, that seems alone sufficient to clarify that it is not required to suppress every possible diagnostic.
I partially agree here; I would also not expect any (type-related) diagnostics to be shown under not TYPE_CHECKING either, for the baseline behaviour of a type-checker.
However, I would not expect the spec to detail whether the lack of diagnostic visibility is because (1) they are emitted-but-suppressed, (2) the code is simply not checked, or (3) the type-checker treats everything underneath not TYPE_CHECKING as typed with Any. It sounds like the proposal is for (2) to be the conformant way and others to be non-conformant, but I would expect an end-user to not think about this at all.
Maybe this is a non-issue now, since as you point out no major type-checker emits errors here, but I’m not sure about the spec change proposed, as I do not know the purpose or usefulness of declaring a future type-checker as non-conformant if they emit certain errors that would be currently considered type-related errors, like Name `var` is not defined under not TYPE_CHECKING.
I’d like to think we can word it loosely enough, while still making the intention clear. This one’s a case where I’m happy to leave it mostly up to type checkers unless it becomes an issue for users requiring reconciliation, which I doubt will happen if the intent is clear.
I think the main issue with this is up until now, there hasn’t been that much specification about other static analysis a typechecker might offer, or rules that some typecheckers offer that aren’t at all based in type theory, such as the choice for some typechekcers to error upon implicit any (eg. Any as a default for generic parameters)
If you’re wanting specification language here, then I think a path forward would require differentiating between classes of diagnostics. Based on the diagnostics typecheckers currently offer, I think that’s something along the lines of these four categories:
rules that are directly in specification or logical conclusions of type theory and the specification.
rules that catch things that are unambiguously an error, even when everything is typed as Any (such as invalid syntax)
rules that prevent accidental gradual typing where fully static typing might be possible.
rules that catch common issues statically, but that aren’t always a mistake (such as mutable defaults or non-initialized slots)
of those, the second is the category I would say should still be enabled under if not TYPE_CHECKING, while the fourth one could be left to typecheckers. The first and third should be supressed.
I actually made some changes to some of my public code as a result of ty’s prior behavior there. While it wasn’t strictly a fix, it did result in easier-to-read code.
Well, we had one user report an issue on our tracker. Were there any others? I don’t have a strong opinion on this change, but I also really don’t remember this coming up all that often for us
I would not support mandating (2) specifically. ty’s current approach is (1), not (2), and I don’t expect that to change. I expect the guidance here to be fairly loose and permit any of these approaches, as long as the result is that users don’t receive type diagnostics inside areas of code they’ve asked the type checker to pretend don’t exist.