Expanding `ReadOnly` to normal classes & protocols

Python lacks a concise way to spell “this attribute is read-only”.
For normal classes, the current two solutions are:

  • marking the attribute Final
class Foo:
   bar: Final[Bar]

   def __init__(self, bar: Bar) -> None:
       self.bar = bar
  • using @property as a proxy
class Foo:
    _bar: Bar

    @property
    def bar(self) -> Bar:
        return self._bar

The former overspecifies the type, as Final marks an attribute non-overridable by subclasses.
The latter requires extra boilerplate, and doesn’t work well for dataclasses (as now __init__ will be generated with _bar as the param name). Freezing a dataclass works only if it’s desired that all attributes are read-only.

For protocols, the currently most permissive solution is to mark the name as a @property. However, this rejects custom getters and class variables.
Final attributes additionally reject @property too.
(My observations here are based on what pyright reports)

from dataclasses import dataclass
from typing import ClassVar, Final, Protocol

class HasFooFinal(Protocol):
    foo: Final[int]

class HasFooProp(Protocol):
    @property
    def foo(self) -> int: ...


class FooImpl:
    foo: int

    def __init__(self, foo: int = 123) -> None:
        self.foo = foo

class FooProp:
    @property
    def foo(self) -> int:
        return 123

@dataclass
class FooData:
    foo: Final[int] = 123

class FooCls:
    foo: ClassVar[int] = 123

class CustomGet:
    def __get__(self, obj: object, cls: type | None) -> int:
        return 123

# pyright reports foo as (variable)
class FooCustom:
    foo = CustomGet()

def custom_get(fn: object) -> CustomGet:
    return CustomGet()

# pyright reports foo as (property)
class FooCustom2:
    @custom_get
    def foo(self) -> object: ...


def read_foo_final(foo: HasFooFinal) -> int:
    return foo.foo

def read_foo_prop(foo: HasFooProp) -> int:
    return foo.foo


read_foo_final(FooImpl())    # ok
read_foo_final(FooProp())    # err: "property" is not assignable to "int"
read_foo_final(FooData())    # ok
read_foo_final(FooCls)       # ok
read_foo_final(FooCls())     # err: "foo" is not defined as a ClassVar in protocol
read_foo_final(FooCustom())  # err: "CustomGet" is not assignable to "int"
read_foo_final(FooCustom2()) # err: "CustomGet" is not assignable to "int"

read_foo_prop(FooImpl())    # ok
read_foo_prop(FooProp())    # ok
read_foo_prop(FooData())    # ok
read_foo_prop(FooCls)       # ok
read_foo_prop(FooCls())     # err: "foo" is not defined as a ClassVar in protocol
read_foo_prop(FooCustom())  # err: "CustomGet" is not assignable to "int"
read_foo_prop(FooCustom2()) # err: "CustomGet" is not assignable to "int"

I think this is a good opportunity for ReadOnly to fill in the gap.
From pragmatic standpoint I’d expect

class HasName(Protocol):
    name: ReadOnly[str]

to accept anything that supports .name access, though I’m +0 whether this should also cover ClassVariables.

For normal classes, it should forbid writing to like Final does, but permit subclasses to override how the name is defined.

6 Likes

This has come up a number of times and I think there’s a general sense it’s a good idea. However, somebody needs to actually write up a PEP about the idea with a precise specification. If you’d like this feature in the type system, I’d encourage you to write that PEP.

3 Likes

I think this would also be useful with ClassVar. Right now mypy allows unsafe covariant overrides of class variables, since there doesn’t seem to any safe way to support this. Use cases like this are somewhat common:

from typing import ClassVar

class X: pass
class Y(X): pass

class A:
    x: ClassVar[X] = X()

class B(A):
    x: ClassVar[Y] = Y()
4 Likes

I also have need for this with ClassVar
Here’s some material you can put in a PEP if you want:

class B:
    x: ClassVar[int] = 1

class C(B):
    x: ClassVar[Literal[2, 3]] = 3  # error - unsafe override

a demonstration of why it’s unsafe:

def spam(b: type[B]) -> None:
    b.x = 4

spam(C)

assert_type(C.x, Literal[2, 3])  # this passes
assert C.x in {2, 3}  # this fails

It’s safe for C to narrow the type of x as long as x in superclasses is read-only.

class B:
    x: ClassVar[ReadOnly[int]] = 1

class C(B):
    x: ClassVar[Literal[2, 3]] = 3

def spam(b: type[B]) -> None:
    b.x = 4  # error - x is ReadOnly

spam(C)

assert_type(C.x, Literal[2, 3])
assert C.x in {2, 3}

Why not x: ReadOnly[ClassVar[int]] ?

I was on the fence about this until this discussion came up: spec: clarify interaction of Final and dataclass by carljm · Pull Request #1669 · python/typing · GitHub

Before this change, the typing spec said:

Final may only be used as the outermost type

But it changed to be allowed inside ClassVar:

x: ClassVar[Final[int]] = 3

Following that same pattern and for the same reasons, ClassVar[Readonly[int]] seems better than ReadOnly[ClassVar[int]]

I’d prefer to allow both orders in general when multiple type qualifiers are added to a type. I don’t think one is obviously better than the other, and I’d rather not require users to remember which order should be used. This is consistent with the behavior of ReadOnly in combination with Required/NotRequired in TypedDict (PEP 705 – TypedDict: Read-only items | peps.python.org).

There may be runtime issues with dataclasses (which introspect for the presence of ClassVar at runtime), but we can fix that by changing the runtime code.

3 Likes

I understand the desire for this, but I don’t particularly like that this is a typing-only construct that doesn’t accurately reflect runtime. (to be clear, I’m not a fan of readonly in typeddicts either)

That said, as long as the rules can be scoped in a way that doesn’t conflict with future attempts at immutable fields of objects of fully immutable objects in the future[1], and doesn’t complicate existing issues, I don’t have a reason to oppose this, just think it’s an area where some caution is warranted.


  1. I have a suspicion that more people are going to want immutable/frozen objects going forward with the nogil work as well as subinterpreters ↩︎

Oh well. I’ve searched the typing forum and didn’t see anything on the topic, but now I see it was brought up in ideas and here

Reading through PEP 1/PEP 12, do I gather right that the starting step would be to create a draft pep-NNNN.rst file and present it here (this thread)?

Sure, that should work. I’d recommend looking at other recent typing PEPs for the expected style too.

1 Like

Is it ok to mention PRs in the PEP? Talking specifically about typing#1669 while covering dataclass + Final interaction

Yes, it’s good to mention such background information.

Here goes nothing: PEP 763[1]
I’ve written Abstract and Motivation, and filled out some of the lesser fields.
Hoping to get some feedback on the soundness of the above.
Rationale/Specification TBD.
Some observations:

  1. I don’t have a sponsor.
  2. I’m not convinced the Motivation is strong.
  3. I guess I can remove Rejected Ideas/Open Issues at the time being?
  4. If this gets implemented, it might be an interesting idea for @dataclasses to look for ReadOnly attributes and rewrite them as private + property combo. Can/should I include this anywhere in the PEP?
  5. Is it really mandatory that I use my real name?
  6. It might be worth linking to discussions#1525 somewhere… the Protocols section already draws from it

  1. I don’t think I can make it viewable in the html form? ↩︎

Depending on when the next PEP is submitted, you might not end up with 763; the usual procedure is to use a placeholder number 999 until you get to the stage of a PR to the peps repo.

I’ve sponsored many of the typing PEPs for the last few years but I’d like more people to step in in that area; perhaps another Typing Council member is interested.

It reads pretty good to me! For me, the strongest motivation is in protocols, where it would be nice to have a concise way to spell “this protocol must have a readable attribute of a particular name and type”.

These sections can be omitted if there’s nothing to add.

It’s definitely worth considering whether we could make dataclasses give ReadOnly a runtime effect (that would also address part of @mikeshardmind’s concern above). Using a synthesized property may not be the best way to do it, however.

It does seem mandatory according to the current template. I opened Should authors use their real name? · Issue #4052 · python/peps · GitHub about potentially dropping this requirement.

Always useful to link back to old discussions for reference.

2 Likes

Thanks for the mention on that Jelle, I might have missed clarifying something that shouldn’t be seen as a detraction till much later otherwise.

I think there’s a pretty strong motivation for both read only fields of instances and standalone immutable values. While only one of those appears to be covered by this, I think that’s a reasonable scope as it keeps it inline with the behavior where ReadOnly is currently allowed, and it would be possible to go even further in the future if this is shown to have a positive impact. It’s generally my preference that the type system and the actual runtime align with eachother, but in places where it’s only ever going to add expressiveness (for example, marking a private field as read only matching developer intent even where the runtime doesn’t enforce it) is still a positive. My prior concern was just making sure you don’t write yourself into something that conflicts with any other work here, especially where runtime might be concerned in the future.

Having now read the draft that you’ve put together since, I’ve gone from cautious but not inherently opposed, to in favor. You’ve laid out everything well, and while there may be things to bikeshed about the wording, the motivation seems strong, and using ReadOnly in this way, while not the end of what I want for python developers certainly seems useful and non-conflicting with future runtime things like object freezing.

1 Like

Thanks for the reassuring words to the both of you.

Ah, I couldn’t know that. Both PEP 1 and 12 say to just use the next available number. I recall also reading somewhere it’s the duty of the sponsor to assign the number ¯\_(ツ)_/¯

That’s fine; how do I get their attention though? I’m wary of mentioning people.

Ok, I’ll leave the implementation details out of the idea; which section would be the most appropriate for it? I’m going to phrase it along the lines of

This feature opens up a way for class transformation tools, like dataclasses, to rewrite fields marked ReadOnly to also reflect it at runtime.

A TODO to my own benefit:

Classes

  • add a footnote to the “pros” of “prevents runtime mutation” as not being governed by the pep
  • consider rephrasing or clarifying “mutation”

Protocols

Backwards Compatibility

  • dataclasses might need to be adjusted if the current implementation would wrongly treat ReadOnly[ClassVar[...]] as an instance attribute
  • dataclasses in older versions using backported ReadOnly may need to enforce it being the inner type [1]

  1. this might be true for Final[ClassVar[...]] as well, since the change made by typing#1669 ↩︎

I don’t know if it should be part of this PEP to dictate what should or shouldn’t be reported by type-checkers around this.

I could imagine an interpretation:

1 class C:
2     x: Readonly[int]
3
4     def __init__(self) -> None:
5         self.x = 3  # error (assignment only allowed on line 2)

For that example, I don’t anticipate a lot of disagreement that that isn’t how it should work.
But I can imagine other cases where there might be more disagreement.

What about this?

class C:
    x: Readonly[int] = 2

    def __init__(self) -> None:
        self.x = 3  # error?

or this?

class C:
    x: Readonly[int]

    def __init__(self) -> None:
        if something:
            self.x = 3
        else:
            self.x = 4  # error?

or this pattern that I think might be common?

class C:
    x: Readonly[int]

    def __init__(self) -> None:
        self.x = 3
        if something:
            self.x = 4  # error?

I think it’s best if all assignments in __new__ or __init__ or in the class scope are accepted, and all others rejected.

For ReadOnly[ClassVar] (or ClassVar[ReadOnly]), only the class scope and metaclass assignments should be accepted, and others rejected.

I believe that the semantics of ReadOnly should closely resemble that of Final, so:

  1. fine, that’s only 1 assignment
  2. it’s an error - there are two assignments, and x is also inferred to be a ClassVar
  3. fine, any code path this can branch into results in only 1 assignment
  4. error, two assignments

However, I agree that it might be needlessly strict to restrict initialization to only one assignment.
In the name of pragmatism, we could deviate from Final and adopt C#'s semantics:

A readonly field can’t be assigned after the constructor exits

Which implies the name can be assigned to an unlimited number of times within __new__ or __init__.

At the time being I’ll continue to write the PEP without this deviation.

I think it’s important to specify this. Some existing type checker features are underspecified, but for new features we should strive to specify them fully so users can rely on consistent behavior across type checkers.

I’m not sure yet what the right behavior is here. Users will likely want to assign ReadOnly variables also in methods invoked from __init__ (a common pattern in some codebases). For example, should it be possible to assign to a ReadOnly in a dataclass __post_init__ method?

I don’t think this should be allowed - it requires a mechanism which would guarantee/could prove that the method is called exactly once during the lifetime of the object; coincidently, such mechanism is exactly what __init__ has implied. [1]

I would like to say yes. However, I don’t think the special dataclass methods [2] are treated specially by type checkers.
I believe this feature would require support from the dataclass_transform side (means to tell a type checker what set of method names are additionally involved in the initialization process)


  1. It’s very unnatural to manually invoke __init__ on an already existing object, and doing so has potential for greater problems than a single read-only attribute being mutated ↩︎

  2. also note that attrs defines its own set of methods ↩︎

1 Like

This doesn’t seem like a good idea to allow this.

I often have __init__ call another method, often called reset or __reset, but the reason I do that is so that I can call reset at a later time also.

There’s nothing in the type system to say that your __post_init__ shouldn’t be called by other methods at any time later. So that would effectively make it not read-only.