Surprising failure when overriding a property with a field in dataclass

Python Help
1

(related but NOT the same: Allow overriding (abstract) properties with fields )

This code:

from dataclasses import dataclass

@dataclass
class B:
    @property
    def p(self) -> int:
        ...


@dataclass
class C(B):
    p: int
    q: int

gives this failure:

Traceback (most recent call last):
  File "/tmp/exec_project_3120100107/code/main.py", line 10, in <module>
    @dataclass
     ^^^^^^^^^
  File "/usr/lib/python3.12/dataclasses.py", line 1275, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/usr/lib/python3.12/dataclasses.py", line 1265, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/dataclasses.py", line 1063, in _process_class
    _init_fn(all_init_fields,
  File "/usr/lib/python3.12/dataclasses.py", line 585, in _init_fn
    raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'q' follows default argument

The issue is that for some reason the dataclass apparatus stores the property (function object) as a default value for p in the parent class.

Is there a reason for this? This is certainly surprising, and I’d say qualifies as a bug. Without the dataclass decorator python has no problem overriding a property with an attribute (they’re supposed to be interchangable).

2

Type annotations alone don’t replace the class attribute and dataclasses uses getattr(cls, ...) when checking for field default values, this can be seen with inherited default values:

from dataclasses import dataclass
@dataclass
class B:
    a: int = 42
    
@dataclass
class C(B):
    a: str
    
print(C())  # C(a=42)

So in your example, your C class is essentially equivalent to this:

@dataclass
class C(B):
    p: int = B.p  # the inherited property
    q: int

There’s no special casing for properties so you get the TypeError. If you remove the ‘q’ attribute you can see that dataclasses.fields(C) shows that the ‘p’ field has a default of the property itself.

You can work around this initial issue by declaring p as a field explicitly so that there is a value for p on C, preventing it from falling through to B.p.

from dataclasses import dataclass, field

@dataclass
class B:
    @property
    def p(self) -> int: ...

@dataclass
class C(B):
    p: int = field()
    q: int

However, this class still isn’t usable as the field() is removed after processing while the property is still in place.

>>> C(1, 2)
...
AttributeError: property 'p' of 'C' object has no setter

You can make a usable class like this if you use slots=True. Then the class attribute is replaced by the slot so there is no fall through to the property on the parent class.

from dataclasses import dataclass, field

@dataclass(slots=True)
class B:
    @property
    def p(self) -> int: ...

@dataclass(slots=True)
class C(B):
    p: int = field()
    q: int

>>> C(1, 2)
C(p=1, q=2)

All of that said, I’m not sure what the correct behaviour here is. The error is unhelpful, but I’m not sure there’s a good working behaviour. If you wrote the class by hand you’d also end up with the same AttributeError issue if you didn’t also create a class attribute and dataclasses behaviour is defined as only leaving default values in class attributes.

1 Like
3

Perhaps the root cause can be pinned to dataclasses inheriting default-arguments. I can see no analogue to it in regular python classes, nor in any other language I’m aware of.

Removing this inheritance could break some existing code, but would make dataclasses more sensible.

4

I think it’s more precise than that, the issue is that dataclasses uses getattr(cls, ...) to decide if there’s a default value and not cls.__dict__.get. If the intent is to inherit default values then it should check for them in fields and not through inherited attributes. An implementation that worked this way would also work the same way for slotted classes, which it currently doesn’t.

from dataclasses import dataclass
@dataclass(slots=True)
class B:
    a: int = 42
    
@dataclass(slots=True)
class C(B):
    a: str
    
print(C())  # TypeError: C.__init__() missing 1 required positional argument: 'a'

This feels more like an implementation quirk than an intended feature.

It’s worth noting that attrs even with slots=False[1] doesn’t inherit default values in this way and behaves the same as dataclasses do with slots=True.

  1. in attrs, slots=True is the default ↩︎

1 Like
5

I can make a PR with the fix suggested. Would you consider it?

6

While I have contributed to (and unintentionally broken) the dataclasses library before - I’m not the one to consider it and I think jumping to a PR is premature. This is a breaking change and not just a ‘fix’ at this point. It’ll definitely need an issue on github before making a PR at the very least.

People may be relying on the existing behaviour, even though it is undocumented and probably undesirable. Due to some other guaranteed dataclasses behaviour you also can’t just switch it to using __dict__.get now without breaking things that are documented (doing so breaks descriptor behaviour and one really strange test I don’t quite understand).

It’s worth noting that even if this were changed, you still wouldn’t be able to make instances of your original example as the property will still be present and has no setter.