My wrapper to @dataclass that requires/enforces attribute declarations and constness

First, quick introduction. I’m a not-so-young physicist who programs a good bit, mostly C++. I only just looked at Python a few weeks ago. Decorators are very neat. __setattr__ and/or properties are just super to allow one to just get on with the coding.

But I like clear interfaces, declared/self-documented up top, and not having assignment to typos go silently unnoticed. I noticed dataclasses, which provide a great format for something like that, and even saw discussions on abusing slotted dataclasses for some amount of declaration enforcement, with some inheritance limitations I believe.

Anyway, to the point:

The readme gives the short version. A test script is included. The wiki gives some related thoughts on encapsulation, along with an included demo script for that as well.

This repo represents half of my total python portfolio, so I expect that shows. I know of a couple of issues, mentioned in the readme or issues. But, it does work (and already caught a couple of my typos in “production” code!), so I thought I’d share. Is the concept useful for python core? I don’t know. I think a more comprehensive version could deal with some of the encapsulation things in a more clean pythonic way, but I doubt I’ll be adding that personally. I think there have been some discussions and developments on that aspect already.

If you use a static type checker like mypy, then you can catch such errors as well.

With this code

from dataclasses import dataclass, field
from typing import Final

@dataclass
class A:
    CONST: Final[int] = field(default=10, init=False)
    x: int

a = A(x=3)
a.y = "foo"
a.CONST = 5

mypy reports:

test.py:10: error: "A" has no attribute "y"
test.py:11: error: Cannot assign to final attribute "CONST"
Found 2 errors in 1 file (checked 1 source file)
1 Like

Nice, does this work for preventing declaration or misspellings within methods? Still, it does clearly get most of the point, and without a run-time penalty.

Edit: Related, does it prevent class variable shadowing by instance variables? Anyway, sounds like I should learn the linters and then consider.

I meant to say thanks for the very detailed reply.

I’m realizing that I shouldn’t assume your comment should be taken as particularly dismissive of a runtime version. It seems that can have use anyway, particularly in less formal environments where one wants to inject a little protection/protection-habits without discussion. Static typing seems still optional and maybe unconventional and relies on less visibly imitable developer habits. I probably should use the annotation spec for Final instead of caps, probably.

So dataclasses and Final seem not, in general, PEP compatible, and I think it’s (close to) impossible to thus define a spec-compliant implimentation, and mypy doesn’t seem very close.

#!/bin/python3
from dataclasses import dataclass
from typing import Final

@dataclass
class MyClass:
	CONST : Final[int] = 5
	    
	def SetConst(self,val):
	    self.CONST=val

x=MyClass(CONST=20)
assert(x.__class__.CONST == 5)
assert(x.CONST == 20)

y=MyClass()          # First assignment to class attribute
assert(y.CONST==5) 
y.SetConst(20)       # First assignment to instance attribute
assert(y.CONST==20)
y.SetConst(30)       # Reassign the instance attribute!!
assert(y.CONST==30)

The asserts all pass and mypy allows it all. I have runtime versions that don’t allow any shenanigans not involving a 5, and one version (not pushed) that just blocks the last change, simply blindly allowing a single assingment to anything.

But PEP 591 explicitly says:
“There can be at most one final declaration per module or class for a given attribute. There can’t be separate class-level and instance-level constants with the same name.” My bold.

And it also seems to imply that assignment in methods other than __init__ are not allowed (and __post_init__?), although the wording isn’t quite tight on that.

It alsmost seems mypy shouldn’t allow Final with dataclasses, or dataclass needs to remove the class variable (in this case). That might be possible.

Technically mypy could simply require this syntax for a class-only const:

	CONST : Final[int] =field(init=False,default=5)

or this syntax for a keyword initializable instance-only constant:

        CONST : Final[int] =field(init=True)

and reject other combinations.

Those do create the right things, maybe not obviously. At present adding a default to the latter syntax results in producing a class attribute as well as the instance attribute, so there is no syntax to create an instance-only initializable constant with a default value that is PEP compliant, and maybe there shouldn’t be. Basically the original syntax is a shorthand for that, and a user might not expect the value 5 to be modifiable at all. It’s more questionable with the field syntax.

I’m not sure any of this can be cleaned up without asking what expected/intuitive behavior would be for a typical user. The present mypy behavior surely isn’t it.

Am I missing something here?

mypy 0.761
python 3.8.10

mypy does block a direct assignment to y.CONST at main scope.

Note that mypy will detect your assignment to final if you annotate val in SetConst:

@dataclass
class MyClass:
	CONST : Final[int] = 5
	    
	def SetConst(self,val: int):
	    self.CONST=val

EDIT: To be more precise: mypy will detect the error as soon as you add any annotation to SetConst.
This would also lead to the detection of the assignment to final:

	def SetConst(self,val) -> None:
	    self.CONST=val

If a method has no annotations, mypy will not consider the method at all for type checking (see No errors reported for obviously wrong code in the mypy docs).

You can change this behavior by running

mypy --check-untyped-defs
1 Like

Nice, but I could also check my code for assignments to CAPS. Was trying to get something else to check things, not more things to check.

You are quite probably missing a fair amount of context around Python’s “gradual typing” approach, then. It’s a key design principle that type annotations and checking are optional, and in general runtime behaviour should not depend on type information from annotations.

You should remember that PEP 591 is a typing PEP, stating how annotations should be interpreted by type analyzers. The dataclass decorator is not a type analyzer, and indeed the documentation explicitly states that

With two exceptions described below, nothing in dataclass() examines the type specified in the variable annotation.

IMO, this is both deliberate and reasonable. One of the key points of gradual typing is that unless you use a type checker, types annotations aren’t enforced. It’s up to type checkers such as mypy to deal with ensuring that the type annotations are respected. I wouldn’t be surprised if it has trouble doing that for a highly dynamic feature like dataclasses which generates code at runtime (indeed, I believe type checkers have special case code to recognise dataclasses and handle them).

Isn’t this a simple consequence of the fact that type checkers, by design, don’t check functions that aren’t annotated? Maybe it’s surprising to someone who normally uses type annotations everywhere, but again, it’s a deliberate consequence of the principle that type checking is opt-in.

I’m not sure what your point here is. If you accept that dataclasses ignore annotations except to treat their existence as the trigger for treating a class variable as a field, then all of the above seems fine to me. The SetConst cases are OK because, as noted, they are not type checked due to lack of function annotations. And setting the instance variable in the constructor seems perfectly fine to me. I’d hope that mypy treats it as Final, and indeed if you annotate SetConst it seems to do so.

So at best, you seem to be suggesting there’s a bug in mypy, but I can’t actually see what that bug is…

On the other hand, if you want a variant on @dataclass that enforces type annotations at runtime, I see no problem with that. And your strictclasses may be a good implementation, I don’t know as I haven’t looked at it. But the fact that you want it and implemented it doesn’t mean the stdlib version is in any way wrong. You may want to publish your strictclasses library on PyPI, if you think others might find it useful, though. But you might need to choose a new name, as there’s already a strictclasses library on PyPI…

Yes, you are totally right. I edited my answer to mention this.

Thanks for the info about PyPl and the mypy options.

Yes, to me, having well enforced dataclasses in casually-distributable runtime scripts makes python a winner, and it seems the mypy suggestion isn’t panning out to change that, and is a few key steps from being able to. All fine. I certainly never thought or said that dataclass is broken by not having runtime checks. Stdlib doesn’t have to get it, or improved dataclass property compatibility, or modifications to help with this typing issue, and no dataclass isn’t required to even produce any code compatible with any typing PEP’s. It’s an ideas sub-forum.

The “bug”
Yes, the applying the “gradual” typing fixes a lot of this. Thank you. My oversight for sure in spite of actually having heard about this I’m afraid. Maybe the remaining bug aspect, is less practically important, but mypy does still allow for both the class and instance versions of a Final-annotated variable, and that’s against PEP 591, for what it’s worth. It seems it may never be possible to see the class value through the instance variable name, which definitely limits the practical impact.

The question
But the related/resulting conundrum is what should be done with this and how strictly should PEP even be followed (by checkers, not python)?

@dataclass
class foo:
    CONST: Final[int] =5
    CONST2: Final[int] = (init=True, default=5)

One should read these as instance constants really but keyword initializable in __init__ with a default of 5.

The first at least doesn’t intuitively look like an attribute that can be not 5, but it can. Should we just be happy it’s technically illegal because of shadowing and fix up the checkers? Or should we want this and ignore the technical shadowing violation?

Ok, maybe the point is I should ask in a mypy forum, as python doesn’t care what checkers do. Fair enough. Obviously I can do what I want with my checker, but I guess I was curious if there’s any thought on what is right.