All of pyright, mypy, and pyrefly currently say “yes”, regardless of whether the underscore-prefixed attribute is implicit (assigned to inside a method) or explicit (annotated on the class body).
I can’t find any text in the typing spec (or in PEP 695) to support this exception, but there is one conformance suite test that (implicitly? unintentionally?) requires it.
This exception is only sound if externally reading and writing underscore-prefixed attributes off an instance of the class is a type error. Pyright is the only type checker which enforces this.
Without this exception, it would be quite hard to write a generic class that exposes an immutable interface via properties and can be treated as covariant in its type variable; in practice, such a class likely must keep some state internally in mutable attributes.
None of the three type checkers apply this exception to underscore-prefixed methods, which seems inconsistent, but perhaps pragmatic given that external use of underscore-prefixed methods is widespread, maybe more so than use of underscore-prefixed attributes?
Should we add this carve-out for underscore-prefixed attributes explicitly to the variance inference rules in the typing spec? If so, should we also say that type checkers should prohibit external use of underscore-prefixed attributes?
If the leading underscore means that it’s a private field, then yes, I think it should not influence the inferred variance.
But with __dunder__ or _sunder_ attributes, that’s usually not the case. And even if we ignore those, then there are still exception to this rule. For example, I wouldn’t call the _fields attribute of NamedTuple instances private, because in practice it’s often used as if it’s public.
But even so, I think that in general, an attribute with leading underscore (and no trailing ones), can indeed be considered as private, and should therefore not influence the inferred variance IMO.
Also, in my experience, people that aren’t typing experts tend to expect that every type is covariant. Usually that looks something like def f(x: list[str | bytes | int]): ..., and it’s one of the most common mistakes I’ve seen. So that’s why I think it’s better to avoid inferring things as invariant, because I expect that it will cause more mistakes, even if it’s theoretically more type-safe.
To exclude a private attribute from variance checks, the attributes must only be accessed from self. This is a bit different from most languages’ definition of private, where a private attribute from another instance of the same class can be accessed. Scala makes this distinction, between class-private and object-private.
Sure, but that’s the general assumption type checking operates under: we can only hope to be sound as long as there is no code the type checker can’t see that does illegal things. I don’t think that should be a blocker here.
My personal opinion is that soundness with respect to contravariance is in practice “best effort” in Python, and that’s probably a reasonable tradeoff for usability. While the type systems enthusiast in me would love to see a truly sound type system applied to Python, on the balance I’m mostly just happy to see more and more people enjoy the benefits of a mostly-sound system.
That said, if there is a building consensus that type checkers should be strict about contravariance of attributes, then I’m happy to see it. We might also reconsider unsound contravariant Self types in that case.