NotImplemented and operator overloading

Documentation
1

Note: Originally I included a lot of links to the documentation in this post, but as a new user it wouldn’t allow me to post more than two links, so I had to remove them. I hope even without those links, it’s clear what pages I’m referring to in this post.

I think the documentation about NotImplemented, object.__eq__, and operator overloading in general is just a little bit unclear.

For me, it started with this note:

Built-in Constants

[…]

NotImplemented

[…]

Note: When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator). If all attempts return NotImplemented, the interpreter will raise an appropriate exception. Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code.

See Implementing the arithmetic operations for examples.

I interpreted this to mean that if “both directions” of a comparison return NotImplemented, then some kind of exception will always be raised.

However, in the case of __eq__, it seems Python will always fall back to object identity equality instead of raising any kind of exception:

>>> class A:
...     def __eq__(self, other):
...         print(f'A.__eq__({other!r})')
...         return NotImplemented
...
>>> class B:
...     def __eq__(self, other):
...         print(f'B.__eq__({other!r})')
...         return NotImplemented
...
>>> A() == B()
A.__eq__(<__main__.B object at 0x7fca76e3b8e0>)
B.__eq__(<__main__.A object at 0x7fca76e1beb0>)
False

So I guess falling back to an is check here is just one of those “other fallbacks, depending on the operator” that the note mentioned. But surely that’s documented somewhere?

Digging deeper I found this:

6.10.1. Value comparisons

[…]
Because all types are (direct or indirect) subtypes of object, they inherit the default comparison behavior from object. Types can customize their comparison behavior by implementing rich comparison methods like __lt__() , described in Basic customization.

But it looks like that’s not really true:

3.3.1. Basic customization

[…]

object.__eq__(self, other)

[…]
By default, object implements __eq__() by using is , returning NotImplemented in the case of a false comparison: True if x is y else NotImplemented .

If it were falling back to object.__eq__, then the falsy is check would return NotImplemented (according to the documentation on object.__eq__) and then an exception would be raised (according to the note on NotImplemented). But that’s not what happens, we just get the False result of the is check.

Furthermore, this doesn’t even seem to be describing the behavior of object itself:

>>> object() == object()
False

But trying this was the epiphany for me:

>>> object.__eq__(object(), object())
NotImplemented

So the documentation for object.__eq__ is describing the behavior of object.__eq__ (duh!!) and not the behavior of a == b (not duh!!).

Indeed, digging in the documentation some more, I find:

3.3. Special method names

A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names. This is Python’s approach to operator overloading , allowing classes to define their own behavior with respect to language operators. For instance, if a class defines a method named __getitem__(), and x is an instance of this class, then x[i] is roughly equivalent to type(x).__getitem__(x, i) . Except where mentioned, attempts to execute an operation raise an exception when no appropriate method is defined (typically AttributeError or TypeError).

There’s a suggestion here that the expression containing the actual operator is only “roughly equivalent” to the corresponding magic method call. But I couldn’t find any further information on this - I would have really liked to know exactly how the two expressions differ for a particular operator. More on this later.

Back to the example: we’re overriding object.__eq__, so its behavior is entirely irrelevant. We ignore that paragraph I quoted from 6.10.1, and look at the next one:

6.10.1. Value comparisons

[…]
The default behavior for equality comparison (== and != ) is based on the identity of the objects. Hence, equality comparison of instances with the same identity results in equality, and equality comparison of instances with different identities results in inequality. A motivation for this default behavior is the desire that all objects should be reflexive (i.e. x is y implies x == y ).

This seems to mean a == b actually falls back to a is b, and not to the “is check, but only if it’s True” behavior documented for object.__eq__. It looks like we’ve finally found the documentation for the behavior we’re seeing at runtime.

It turns out it’s all in the docs somewhere (mostly) - but the problem is the path I took to get there:

  1. I found a bug in my code. IncompatibleTypeA() == IncompatibleTypeB() evaluated to False instead of raising the exception I expected.
  2. Both my __eq__ methods are returning NotImplemented for this case. Clearly I don’t understand something about how this is supposed to work. Let’s check the docs to see what’s going on.
  3. Found the note on NotImplemented, and started looking around for documentation on the fallback behavior for __eq__.
  4. Found “6.10.1 Value comparisons”, which looks like it says it inherits object.__eq__. Weird, since I’m not calling super() - but sure, let’s see what that means.
  5. Checked the documentation for object.__eq__. Sounds like I should be getting NotImplemented for a == b when a is not b. But I’m getting False, so something’s not right.
  6. Referred back to 6.10.1 and found the next paragraph that says the default is to compare the identity of the objects. This seems consistent with what I’m observing at runtime, but it seems inconsistent with what the previous paragraph just said.
  7. The documentation is just getting confusing, so it’s time to switch to “hands-on mode”. Tried out a bunch of example cases in the REPL.
  8. Finally discovered by tinkering that there is a subtle difference in meaning between a == b and a.__eq__(b).
  9. Dug around in the docs some more, and found the “roughly equivalent” wording in “3.3 Special method names”.

Understanding that subtle difference between the expression a == b and the magic method call a.__eq__(b) was critical for understanding why overriding __eq__ behaves the way it does here. But as far as I can tell, that subtle difference isn’t actually spelled out anywhere in the documentation. At best there’s a suggestion that this subtle difference might exist. I think this is the critical missing piece in the documentation here.

As a whole, the documentation strongly suggests that my example above would raise an exception, and it’s very surprising that it doesn’t. It certainly surprised me, and it seems that others found it surprising too:

Only after much closer inspection, piecing together information found on different pages on very different topics, was it possible to form an understanding of the runtime behavior. And I’m still not exactly clear on how this affects other operators, if it affects them at all.

So I’ll suggest some rough (i.e. terrible) changes, and (if you agree that anything needs to change in the first place) we can bikeshed them into something much better.

First I suggest adding a link to the “6. Expressions” page in the NotImplemented note, since that seems to be where those “other fallbacks” are actually documented:

Built-in Constants

[…]

NotImplemented

[…]

Note: When a binary (or in-place) method returns NotImplemented the interpreter will try the reflected operation on the other type (or some other fallback, depending on the operator - see (link to "6. Expressions" page) for details on the fallback behaviors for some operators). If all attempts return NotImplemented, the interpreter will raise an appropriate exception. Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code.

See Implementing the arithmetic operations for examples.

As a documentation reader, this was my “entry point” to the docs for this whole topic, and having this link would have directed me to exactly the right place to further my understanding of the operator fallback behavior.

The more important piece is to somehow document that there is a difference between an operator-expression and the corresponding magic method call (e.g. a == b vs a.__eq__(b)). I’m not actually sure where this would best fit. In most contexts it’s going to be unnecessary detail, but in other contexts it’s going to be exactly what you need to know.

My best guess is to place a footnote in “3.3 Special method names” next to “roughly equivalent”, that leads to a separate page with details on how the expressions differ:

3.3. Special method names

A class can implement certain operations that are invoked by special syntax (such as arithmetic operations or subscripting and slicing) by defining methods with special names. This is Python’s approach to operator overloading , allowing classes to define their own behavior with respect to language operators. For instance, if a class defines a method named __getitem__(), and x is an instance of this class, then x[i] is roughly equivalent [1] to type(x).__getitem__(x, i) . Except where mentioned, attempts to execute an operation raise an exception when no appropriate method is defined (typically AttributeError or TypeError).

[…]

[1]: See (link to another page) for more details.

But this still feels fairly well-hidden. And it doesn’t really connect to the path I took through the documentation starting from NotImplemented, so it wouldn’t have actually been very helpful to the single “user story” I just described.

Does anyone else feel like this needs to be improved? Does anyone have any better suggestions for how this can be made more clear?

4 Likes
2

I don’t have any specific suggestions on what you mention. But I share your concern that the official docs are woefully lacking when it comes to clearly and coherently documenting the specific, detailed operation of things like this. The relevant information is distributed across different sections of the docs, and there aren’t enough links connecting the pieces. And for some behaviors I’m not sure a complete specification is extractable from the docs at all.

1 Like
3

a == b is a binary operator that invokes special methods on its operands and returns a Boolean representing their result.

If you can point out somewhere in the docs that explicitly says it returns the result of one or the other’s magic method, we’ll fix it, because it shouldn’t say that.

The same goes for the comparison operators. __lt__ and friends return the value, so that these operators can be chained, but the final result of an expression involving them is either True or False. To get the value back, you have to call the implementation directly.

4

Right, that’s the understanding I eventually arrived at. That quote is from the numbered list where I’m outlining what I was thinking while I was still trying to figure out what was going on. (Perhaps that wasn’t very clear, sorry - I was trying to describe my interaction with the docs as I attempted to solve a problem)

Well, the problem I’m encountering with the documentation is that it doesn’t seem to explicitly say what it actually does (beyond invoking the magic method and using the result somehow). That’s the part that I feel is missing from the docs.

Well said. The Python docs are in general pretty spectacular, but unfortunately with specific details like this, they’re either scattered across multiple pages or missing entirely.[1]

  1. Perhaps I’m too used to “heavily-specified” languages like C++ where, for pretty much every behavior you might encounter, you can find a line of incomprehensible gibberish somewhere in the spec that claims to describe that behavior. :slight_smile: ↩︎

5

That’s not correct. Or at least, I hope it’s not, as it would break the whole Python scientific stack. :upside_down_face: The result of np.array([1, 2, 3]) == np.array([8, 2, 0]) is array([False, True, False]). It’s the same for __lt__ and the others. (Comparison chaining doesn’t work with numpy arrays because it explicitly tries to do a bool() on the results of the individual comparisons, and numpy arrays won’t let you bool() them.)

3 Likes
6

You’re right. I guess it’s just the NotImplemented singleton that has special behaviour.

7

There’s something else that’s interesting about the behavior of the equality dunder : when you compare two instances of the same type, e.g B(left) == B(right), you’d expect it to try it once, and give up if it returns NotImplemented, like what happens with B(left) + B(right) : if __add__ fails, __radd__ is not called.
But no, if the left call fails it still calls B(right).__eq__, treating __eq__ exactly as “its own reflection”. Kudos for word-for-word applying what’s documented, I guess :upside_down_face:

Anyway, it seems that the behavior for == (and for !=) is :

  • Try left.__eq__(right)
  • if it returns NotImplemented, try right.__eq__(left)
  • if it returns NotImplemented, return left is right

Should we add that info (rephrased of course) to the documentation ?

1 Like
8

Personally, I’d probably expect it to do what it does anyway (or at least not be surprised that it does). But making the docs more explicit[1] doesn’t really have any downsides, so sure.

  1. as long as you don’t obscure the main narrative in the process ↩︎

9

Here it is : gh-72971: Clarify the special no-TypeError behavior for equality by Gouvernathor · Pull Request #110729 · python/cpython · GitHub
I tried to be as straightforward as possible, but of course I take suggestions on rephrasings.

If I may, here is another PR I authored on a very similar subject - the role of NotImplemented in in-place math dunders : gh-104219: Document that idunders can return NotImplemented by Gouvernathor · Pull Request #104220 · python/cpython · GitHub

1 Like