We already have some warnings in the typing-module docs for @runtime_checkable, but I think they could be clearer and more prominent. I’ve just filed Typing docs: increase prominence of warnings regarding `@runtime_checkable` by AlexWaygood · Pull Request #127985 · python/cpython · GitHub for that. (Here’s a preview for what the docs would look like if that PR were merged: typing — Support for type hints — Python 3.14.0a2 documentation)
2 Likes
I’ve PR’d a change to some wording brought up in this thread, and to include __init_subclass__
It also went unnoticed to me that @erictraut is working on support for this PEP, hope that this doesn’t cause much disturbance 
For __init_subclass__, shouldn’t it also allow a similar alternative type check approach as instance variables, allowing assignment to any type[TheClass] instead of specifically cls? Otherwise it’d imply type checkers can take that simplification for instance vars, but have to implement it anyway for class vars.
There’s no alternative for __init__. Do you mean the alternative for __new__?
__new__ has the alternative rules to permit what is done in immutable classes (fractions.Fraction, yarl.URL), where self can be sourced from classmethods.
I believe __init__ and __init_subclass__ are on pair in that there’s no ambiguity what object they should be able to initialize, as it is always the first positional argument.
Allowing assignment to arbitrary type[TheClass] would allow reassigning an already initialized read-only class attribute:
class Base:
foo: ReadOnly[ClassVar[int]] = 0
def __init_subclass__(cls) -> None:
Base.foo += 1
class A(Base): ...
class B(Base): ...
print(Base.foo)
…defeating the point of using ReadOnly there.
Ah yes, I was thinking that rule was more to allow a checker which found it difficult to special-case only certain variables for this, not just say anything with Self.
I’m working on implementing experimental support for PEP 767 in pyright. I found a number of issues in the PEP that require additional discussion.
- The PEP indicates that a read-only attribute can be overridden by a property or other descriptor object. I don’t think that’s sound because of the covariance of
type. Consider the following code. Regardless of whetherHasName.nameisReadOnly, it is unsound to override it with a descriptor.
from typing import Protocol
class HasName(Protocol):
name: str
def func(h: type[HasName]):
reveal_type(h.name) # Both pyright and mypy reveal str here
print(h.name.upper())
class NamedProp:
@property
def name(self) -> str: ...
# The following call will crash. This should be a type violation,
# and currently mypy and pyright generate an error here. Making
# `name` ReadOnly should not change this.
func(NamedProp)
This affects at least three parts of the spec:
- In the “Rationale” section, the PEP says that “A subclass of
Membercan redefine.idas a writable attribute or a descriptor”. I think you need to delete “or a descriptor”. - The “Subtyping” section says “Read-only attributes can be redeclared as writable attributes, descriptors or class variables”. For the same reason here, “descriptors” cannot be allowed.
- In the “Subtyping” section, one of the examples (
NamedProp) should be deleted because this is unsound. Similarly,NamedDescriptorshould be deleted.
-
The PEP is not clear on whether a “bare”
ReadOnlyannotation is allowed or should be considered an error. One could argue that this should be an error condition sinceReadOnlyis a type qualifier, so it should have a corresponding type to qualify. However, one could also make the argument that an implicit type ofAnycould be assumed if a type annotation is omitted. One could also make the argument that if a bareReadOnlyis coupled with an assignment, inference should be used (as it is with theFinalqualifier); however, inference rules would get tricky to define here because (unlikeFinal), it’s inadvisable to infer literal values because this could cause problems for overrides within subclasses. In any case, additional clarity is needed here. -
The “Initialization” section includes a comment “cannot assign to a read-only attribute of a base class” even though the code is located within an
__init__method. This seems inconsistent with the fact that a subclass can override a read-only attribute. If it’s able to override, why can’t it reassign a new value in the__init__method (or__new__or__init_subclass__) without generating an error? -
There is an “Open Issues” section that still needs to be resolved. My take on the open issue is that third-party initialization hooks should be out of scope. The initialization rules in this PEP are already pretty onerous for type checkers to implement. Extending these rules to arbitrary third-party initialization mechanisms is not feasible IMO.
I think removing assignability to descriptors from the PEP would be a big mistake and make this PEP a lot less useful. Since a lot of the motivation stems from being able to express “I don’t care whether this attribute is a descriptor or a regular attribute, it just needs to be readable”. Being able to declare a covariant attribute is certainly nice, but I don’t think it’s the main motivator behind this PEP, since a read-only property will already get you covariance.
If we’re truly worried about this safety hole with type I think I would prefer to treat accesses to a ReadOnly attribute on type as potentially unsafe and flagging those uses, rather than disallow descriptors entirely. That’s a far lower penalty to pay for type safety.
Especially since you could get around that limitation by declaring the attribute as a ClassVar in addition to it being ReadOnly. Doing that would remove assignability to descriptors, unless they happen to return the correct type for class access as well as instance access.
4 Likes
Isn’t the unsafety in that example the access of name on the class object? The protocol only says that instances need to have a name attribute that is a string, not their class. This code also has the same issue:
from typing import Protocol
class HasName(Protocol):
name: str
def func(h: type[HasName]):
reveal_type(h.name) # Both pyright and mypy reveal str here
print(h.name.upper())
@dataclass
class NamedDataclass:
name: str
class NamedInit:
def __init__(self) -> None:
self.name = "abc"
func(NamedDataclass)
func(NamedInit)
Neither mypy nor pyright currently error on this, but both function calls will throw runtime errors. I’m not sure why the h.name access should be legal and safe? An object of type type[HasName] is not guaranteed to have a name attribute or even an annotation for it in its class body.
2 Likes
h.name in the above examples are “possibly present”, and if present, should be str. I think this is a result of another prior mistake, where typecheckers can treat bare annotations in the class body as either classvars or instancevars without any explicit mention of it being a classvar and without an assignment in the class body making it one.
Fixing this would require modeling this more accurately as something similar to a typed dict’s NotRequired field on the type.
Allowing a property replacement might be fine if we do that, and then say in the presence of ReadOnly, it’s optionally a string or a descriptor which returns a string on the type.
Nope, this would still be an issue with covariance of type because of the fact that ReadOnly[T] can be replaced with T in subclasses because ReadOnly doesn’t mean not compatible with settable, it means Readable. This doesn’t work, and @erictraut is right that this isn’t sound.
1 Like
Oh, right, here’s a concrete example of that being a problem.
from __future__ import annotations
class A:
x: 'ReadOnly[str]'
class B(A):
x: 'ReadOnly[str]' = property(lambda self: '"only" is a misnomer')
class C(B):
# if we claim it only applies to instance, this would only add
# functionality to x, not an LSP issue without negation of writable
x: str = "covariance trap"
def mod_b[T: type[B]](typ: T) -> T:
typ.x = property(lambda self: "change")
return typ
def use_c():
mod_b(C).x + "boom" # TypeError: unsupported operand type(s) for +: 'property' and 'str'
if __name__ == "__main__":
use_c()
To go even further than that, if A is a protocol and not in the MRO, and mod_b becomes mod_a, then this isn’t even something that can be fixed with patchwork of inspecting the mro and collecting the clash as a union on the type.
It’s just unsafe to have all of these:
- ReadOnly doesn’t mean incompatible with Settable
- types are covariant
- descriptors are valid as a ReadOnly version of the type they get.
- class annotations can mean instance variables
These issues aren’t new, and they’re all related to the type model not properly representing instance-level attributes. It doesn’t seem smart to cripple this feature due to past mistakes.
My proposed solution of flagging all accesses to non-ClassVar ReadOnly attributes on a type[Protocol] as unsafe instead[1], should work as a band-aid until we decide we want to properly represent which attributes are only accessible on an instance of a type, but not the type itself.
@mikeshardmind Shouldn’t your example flag the assignment in mod_b? I don’t see why that would be allowed. Structurally matching a type with a descriptor doesn’t mean you’re allowed to assign a descriptor to a type[Protocol]'s read-only attributes, that would still be a type violation[2].
I think we should. These issues aren’t new, but we shouldn’t make these issues worse either. We can revisit this if we fix the underlying problems, but we shouldn’t create new problems.
I think you missed the comment that was included on that about if it was being treated as only on the instance as requested. It doesn’t matter though because of this example I was talking about when pointing out the protocol case.
class A(Protocol):
value: 'Readonly[str]'
class B:
value: str = "..."
def mod_a[T: type[A]](x: T) -> T:
x.value = property(lambda self: "...")
return x
def issue():
mod_a(B).value + "nope, this isn't okay"
I must be missing something, because that example has exactly the same problem:
def mod_a[T: type[A]](x: T) -> T:
x.value = property(lambda self: "...") # should be disallowed
return x
A.value is ReadOnly so you can’t assign anything to it, not even a read-only property.
I think there’s definitely various levels of concern depending on whether we’re talking about structurally matching a Protocol or nominal subtyping. The assignability on a subtype is far more problematic, since you can produce an issue with a much simpler example:
class A(Protocol):
value: ReadOnly[str] = "..."
class B:
@property
def value(self) -> str:
return "..."
def foo(x: type[A]) -> str:
return x.value + "..."
foo(B) # boom
I agree that this is problematic enough, without also having access to something like InstanceVar, that it should probably be disallowed.
However for structural type matching I think we can get away with instead treating ReadOnly attributes as instance-only[1]. The potential benefits here far outweigh any concerns about safety IMHO.
Without that, I don’t see myself ever using this feature much in protocols, since property with its special-casing in a Protocol already mostly behaves like this nerfed ReadOnly would and showcases similar problems when interacting with type.
as long as the type doesn’t explicitly subclass the
Protocol↩︎
That doesn’t follow. If the logic for it being allowed to be replaced with a descriptor is that it doesn’t describe what exists on the type, only the instance, then the ReadOnly requirement can’t be part of the type, only the instance.
I think descriptor overrides need to be decoupled from this PEP. The current draft is inconsistent with assumptions made by the major type checkers with regard to descriptors. As others have pointed out, it’s probably not a good idea to introduce new functionality that is spec’ed to work inconsistently with existing (unspec’ed) functionality.
This is an area that has not yet been tackled in the typing spec, but it’s an area where I’d welcome additional clarity. You’ll notice that “Instance and class variables” appears in the list of spec priorities that I proposed last year. I’ll note that I predicted this topic would be contentious and potentially difficult to spec. 
I think there are two paths forward. Both involve decoupling this topic from the PEP.
Path 1: This PEP could move forward by removing mention of descriptor overrides as I suggested above. We could subsequently work to clarify the typing spec with regard to class and instance variables. This clarification would then apply to ReadOnly and non-ReadOnly attributes in a consistent manner.
Path 2: We could put the PEP on hold and first focus on clarifying the typing spec with regard to class and instance variables. Once the behavior has been clarified for non-ReadOnly attributes, work on this PEP could resume in a way that extends the functionality for ReadOnly attributes in a consistent manner.
I guess there’s also a Path 3 where we attempt to tackle all of this as part of the same effort, but that sounds unnecessarily complicated.
@Eneg, as the PEP author, do you have a preference?
Does anyone want to volunteer to drive the effort in clarifying behaviors related to class and instance variables?
2 Likes
This is not particularly high on my priority list unless you see it as feasible that this could include the following features:
NotRequiredfields on types as a result of instance fields that are required (required for what some people want here)- Descriptors being generic, including property (required for what some people want here)
objectandtypegaining some base definitions corresponding to the data model since dunder methods are explicitly reserved for use by python, and these have documented semantics.- Stricter rules for the type of the
typefunction at runtime.
class HasName:
name: str
According to PEP-526, this means that instances of HasName have the name attribute.
I think it’s wrong for type checkers to allow access to name from the class.
class HasName:
name: ClassVar[str]
With ClassVar, we can access name on the class.
As this unsafe type-checker behavior already exists unrelated to this PEP, I don’t see a reason for it to hold up this PEP.