NotImplemented in inplace dunders?

In what circumstances, if any, can and should inplace dunders (__iadd__, __isub__, __imul__, __imatmul__, __ior__…) return NotImplemented ?

The doc about inplace dunders says “If a specific method is not defined, the augmented assignment falls back to the normal methods.”, but it does not say that fallback happens if and when NotImplemented is returned, as it does with r-dunders : “to evaluate the expression x - y, where y is an instance of a class that has an __rsub__() method, type(y).__rsub__(y, x) is called if type(x).__sub__(x, y) returns NotImplemented”.
Does that mean that i-dunders does not support NotImplemented ? If so it’s perfectly understandable (since you can return a + b as a fallback in a.__iadd__), but I think it should be written more clearly.

This doesn’t seem hard to test:

>>> class x:
...     def __init__(self, name):
...         self._name = name
...     def __str__(self):
...         return f'x<{self._name}>'
...     def __iadd__(self, other):
...         print('iadd', self, other)
...         return NotImplemented
...     def __add__(self, other):
...         print('add', self, other)
...         return NotImplemented
... 
>>> 
>>> a, b = x('a'), x('b')
>>> a += b
iadd x<a> x<b>
add x<a> x<b>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +=: 'x' and 'x'

Wow, that’s definitely not documented :eyes:
That means there can be two fallback calls when the second operand implements radd ! (yes, I tested)
Thanks.

I think it’s just poorly explained, and this is probably a good issue to raise in #documentation .

Yes, I think I’ll PR it on github directly. The behavior itself is good, it’s just not at all what I expected.

Just to note, the docs for NotImplemented itself actually do in fact document that NotImplemented may be returned by the I* dunder methods (I in fact just looked up this very question a couple weeks ago):

A special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) for the same purpose.

The docs for the i* methods you mentioned could also be improved by properly cross-referencing NotImplemented in the spot you mention so users can easily find and navigate to these relevant details of how NotImplemented works; I can mention that on your PR.

3 Likes

I committed the reference in the PR, thanks.
I think the explanation you quote is (was) not enough : it says NotImplemented can be returned by idunders, but not what happens then. It could very well be turned into a TypeError without any fallback.
That’s not unheard of, it’s what happens in the following example :

class A:
    def __add__(self, other):
        return NotImplemented
    def __radd__(self, other): # won't be called
        raise Exception

A() + A()

Since those are the same type the rdunder is not tested.

That quote was only an except from the linked description—I originally included the full portion where the behavior of NotImplemented is discussed, but as I thought you were only asking about the “idunders” supporting NotImplemented and not the mechanics of how NotImplemented fallback works, I elided all but the portion relevant to the former.

Immediately after that paragraph that I quoted from, there’s a Note call-out describing the fallback process in some detail:

When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return NotImplemented, the interpreter will raise an appropriate exception. Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code.

See Implementing the arithmetic operations for examples.

The linked document has additional detail and examples of how that works in practice.

1 Like