Why do __setattr__ and __delattr__ raise an AttributeError in this case?

Python Help
1

What is the rationale for which object.__setattr__ and type.__setattr__ raise an AttributeError during attribute update if the type has an attribute which is a data descriptor without a __set__ method? Likewise, what is the rationale for which object.__delattr__ and type.__delattr__ raise an AttributeError during attribute deletion if the type has an attribute which is a data descriptor without a __delete__ method?

I am asking this because I have noticed that object.__getattribute__ and type.__getattribute__ do not raise an AttributeError during attribute lookup if the type has an attribute which is a data descriptor without a __get__ method.

Here is a simple program illustrating the differences between attribute lookup by object.__getattribute__ on the one hand (AttributeError is not raised), and attribute update by object.__setattr__ and attribute deletion by object.__delattr__ on the other hand (AttributeError is raised):

class DataDescriptor1:  # missing __get__
    def __set__(self, instance, value): pass
    def __delete__(self, instance): pass

class DataDescriptor2:  # missing __set__
    def __get__(self, instance, owner=None): pass
    def __delete__(self, instance): pass

class DataDescriptor3:  # missing __delete__
    def __get__(self, instance, owner=None): pass
    def __set__(self, instance, value): pass

class A:
    x = DataDescriptor1()
    y = DataDescriptor2()
    z = DataDescriptor3()

a = A()
vars(a).update({'x': 'foo', 'y': 'bar', 'z': 'baz'})

a.x
# actual: returns 'foo'
# expected: returns 'foo'

a.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(a)['y'] == 'qux'

del a.z
# actual: raises AttributeError: __delete__
# expected: ('z' in vars(a)) is False

Here is another simple program illustrating the differences between attribute lookup by type.__getattribute__ on the one hand (AttributeError is not raised), and attribute update by type.__setattr__ and attribute deletion by type.__delattr__ on the other hand (AttributeError is raised):

class DataDescriptor1:  # missing __get__
    def __set__(self, instance, value): pass
    def __delete__(self, instance): pass

class DataDescriptor2:  # missing __set__
    def __get__(self, instance, owner=None): pass
    def __delete__(self, instance): pass

class DataDescriptor3:  # missing __delete__
    def __get__(self, instance, owner=None): pass
    def __set__(self, instance, value): pass

class M(type):
    x = DataDescriptor1()
    y = DataDescriptor2()
    z = DataDescriptor3()

class A(metaclass=M):
    x = 'foo'
    y = 'bar'
    z = 'baz'

A.x
# actual: returns 'foo'
# expected: returns 'foo'

A.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(A)['y'] == 'qux'

del A.z
# actual: raises AttributeError: __delete__
# expected: ('z' in vars(A)) is False

I would expect the instance dictionary to be mutated instead of getting an AttributeError for attribute update and attribute deletion. Attribute lookup returns a value from the instance dictionary, so I am wondering why attribute update and attribute deletion do not use the instance dictionary as well (like they would do if the type did not have an attribute which is a data descriptor).

2

If __set__ is missing then it’s actually not a data descriptor but a non-data descriptor. As such there isn’t anything to mutate so I’m not surprised it raises AttributeError. And since descriptors are called before anything relating to the attribute occurs, trying to do something that isn’t legal for the descriptor should raise an exception.

What are you expecting to be the result instead?

3

A data descriptor does not actually need a __set__ method, a __delete__ method is enough, as stated in the Descriptor HowTo Guide:

If an object defines __set__() or __delete__() , it is considered a data descriptor.

I would expect the instance dictionary to be mutated instead of getting an AttributeError. Attribute lookup returns a value from the instance dictionary, so I am wondering why attribute update and attribute deletion do not use the instance dictionary as well (like they would do if the type did not have an attribute which is a data descriptor). In other words:

a.x
# actual: returns 'foo'
# expected: returns 'foo'

a.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(a)['y'] == 'qux'

del a.z
# actual: raises AttributeError: __delete__
# expected: 'z' not in vars(a)

and

A.x
# actual: returns 'foo'
# expected: returns 'foo'

A.y = 'qux'
# actual: raises AttributeError: __set__
# expected: vars(A)['y'] == 'qux'

del A.z
# actual: raises AttributeError: __delete__
# expected: 'z' not in vars(A)
4

Right, forgot about that quirk!

Right, if there’s an instance attribute with the same name as the descriptor.

Without diving into the C code, my guess is no one thought about it that way. Descriptor logic is already complicated as it is between data descriptors, instance attribute, non-data descriptors, and class attributes. Wouldn’t surprise me that someone didn’t think to mirror the lookup order for setting and deleting things.

5

I have also asked the question on Stack Overflow and user2357112 seems to share your opinion and provides an analysis of the C code showing how the AttributeError happens and could be avoided. If you know who designed or implemented the descriptor protocol it might be interesting to have him join this discussion.

6

Are you looking only for the reasons why, or are you hoping to change it?

7

If this AttributeError was not intended by the original author of the descriptor protocol (PEP 252 suggests this is @guido) but an accident from the C implementation, I am hoping to change it, if this is backward compatible and people agree.

8

After all of these years, I don’t see how it could be honestly.

9

@brettcannon @stoneleaf I have just opened a corresponding issue here and a pull request here solving the issue and adding the above test case. The changes do not break any tests in the Python test suite, which is good news. Could I request your review?

10

Did you read the documentation for descriptors in the data model?

https://docs.python.org/3/reference/datamodel.html#invoking-descriptors

I think your example involves the function described in class binding. According to PEP 252 in that case calling delete on it, should result in an attribute error. I think that is quite logical, because if you delete a descriptor in a class, the expected effect would be that in any subtype and any instance of the type and its subtypes you would not have access to the described data any more. It would just disappear. Very disconcerting. Or am I missing something?

11

It’s good that the stdlib passed, but it is only a tiny fraction of all the Python code out there. The backwards compatibility issue is the code out in the wild that uses and expects the current behavior.

12

Yes, I did.

If you are referring to the example del a.z or del A.z, none of them involve the call A.__dict__['z'].__get__(None, A). The former involve the call type(a).__dict__['z'].__delete__(a), the latter involve the call type(A).__dict__['z'].__delete__(A).

An AttributeError is raised from these calls because the __delete__ method does not exist (not because these calls would affect any subtypes or instances like you said, since when __delete__ exists no AttributeError is raised anyway). Now since __delete__ does not exist, why does not CPython proceed with the instance by trying to delete a.__dict__['z'] for the former, and trying to delete A.__dict__['z'] for the latter, as it would have done if a non-data descriptor attribute, a non-descriptor attribute, or no attribute at all had been found on the type? That is the point of this thread.

13

You write:

In the descriptor how-to the call to get is described as:

If binding to a class, A.x is transformed into the call: A.__dict__['x'].__get__(None, A) .

As a corollary I would say that the deletion would be:

If binding to a class, del(A.x) is transformed into the call: A.__dict__['x'].__delete__(None, A) .

PEP 252 doesn’t mention this for delete, but it does for set and continues to warrant an exception to be thrown:

If the attribute is read-only, this method may raise a TypeError or AttributeError exception (both are allowed, because both are historically found for undefined or unsettable attributes). Example: C.ivar.set(x, y) ~~ x.ivar = y.

So it was probably not by accident that you cannot replace or remove a descriptor by replacing it through the instance dictionary of the class.

I do not know if this is really a problem or that I am just not right. I am still confused, I would expect that when defining a descriptor that misses a setter or a deleter, you would not be able to refute those definitions by creating an attribute that is not a descriptor.

14

I’ve been studying this lately in the interests of a Java implementation. I have the luxury of being able to define __set__ and __delete__ distinctly, so the logic is a little clearer. The question of missing definitions is more obvious. Even so, I had missed that a descriptor is one even if it has no __get__. (Now fixed.)

Descriptors are defined so we can take control of the “dot” (attribute access). What seems odd to me about the original example is that doing this in type A, the example yet manipulates attributes through the instance dictionary directly.

The existing semantics seem sensible to me: if the programmer creates a descriptor with no __set__, I think they intend the attribute not to be set by attribute access. The error message is not very helpful. (Mine says the attribute is read only.)

It is a little surprising that absent a __get__, attribute access returns the descriptor itself, or an instance value, but it is what the data model says. And an attribute you can’t get, wouldn’t be very useful.

15

Actually not, because:

  1. __delete__(instance) does not have an owner parameter like __get__(instance, owner=None).
  2. The attribute is first searched on the type of the originating object (so 'z' is searched on type(A) for del A.z) and if it is found and is a data descriptor, its method is called (so type(A).__dict__['z'].__delete__(A) is called). (Cf. the Descriptor HowTo Guide: “If an instance’s dictionary has an entry with the same name as a data descriptor, the data descriptor takes precedence.”) You can experiment in Python or read the C source code to check this.

The PEP 252 quote states that you can implement the __set__ method of the descriptor as raise TypeError or raise AttributeError for signaling that the overridden instance attribute is read-only. This is not related to the topic of this thread where the __set__/__delete__ method does not even exist.

The above examples do not expect the descriptor of the owner to be replaced/removed. They expect the attribute of the instance to be replaced/removed. So I think this is where you were confused.

16

No, they intend not to override the update of the instance attribute. If they intended the instance attribute not to be updatable i.e. to be read-only, they should explicitly define a __set__ method which raises an AttributeError, as specified in PEP 252:

__set__(): a function of two arguments that sets the attribute value on the object. If the attribute is read-only, this method may raise a TypeError or AttributeError exception (both are allowed, because both are historically found for undefined or unsettable attributes).

and in the Descriptor HowTo Guide:

To make a read-only data descriptor, define both __get__() and __set__() with the __set__() raising an AttributeError when called.

17

No, they intend not to override the update of the instance attribute. If they intended the instance attribute not to be updatable i.e. to be read-only, they should explicitly define a __set__ method which raises an AttributeError , as specified in PEP 252

Good point. I agree. There is that explicit way to achieve read-only character.

I am clearly too steeped in my reverse-engineered CPython. I looked at property too for clues to the expected behaviour of defective[1] data descriptors, where not specifying a fset/fdel means you can’t do that operation. I wonder if that also ought to change under this proposal.

I read somewhere that PEPs are proposals, not specifications: it is clear they are not maintained when overtaken by their implementation: PEP 252 still has an unfinished air about it and doesn’t mention __delete__ at all, so it can’t consider the possibility of defective data descriptors.

The data model ought to be definitive, but is incomplete in not considering the full implications of a data-descriptor with only __set__ or only __delete__.

[1] as in defective verb

18

That is an interesting remark. But no, property should not be changed to allow the creation of “defective” descriptors under this proposal (for example data descriptors with a __delete__ method but without a __set__ method), since according to the language documentation, it is designed to create read-only properties (i.e. data descriptors with a __set__ method raising an AttributeError) when initialized without an fset argument:

This makes it possible to create read-only properties easily using property() as a decorator:

class Parrot:
    def __init__(self):
        self._voltage = 100000

    @property
    def voltage(self):
        """Get the current voltage."""
        return self._voltage
1 Like
19

I have been looking into this further and it is indeed not so that you propose what I was afraid of.

Sorry for the noise.

20

No problem, thanks for joining this thread :slight_smile:.