A Bug? in matmul and rmatmul

Hi there :wave:

When I looked at how the parser resolved the left and right @ operators, I found a bug-like behavior.
Before reporting this to the bug tracker, I’d like to hear opinions… thank you.
Issue
Python tries to parse x @ y as x.__matmul__(y), and if it isn’t found, will try y.__rmatmul__(x).
But in the symmetric case, i.e. b @ b, Python will try b.__matmul__(b), but does not try to find b.__rmatmul__(b), resulting in TypeError.

class A:
    def __matmul__(self, argv):
        return argv
    def __rmatmul__(self, argv):
        return argv

class B:
    def __rmatmul__(self, argv):
        return argv

class C(B):
    pass
>>> a @ a
<__main__.A object at 0x000001899E53EE80>
>>> a @ b
<__main__.B object at 0x000001899E53EE50>
>>> b @ a
<__main__.B object at 0x000001899E53EE50>
>>> b @ b
TypeError: unsupported operand type(s) for @: 'B' and 'B'
>>> b @ B()
TypeError: unsupported operand type(s) for @: 'B' and 'B'
>>> b @ c
<__main__.B object at 0x000001899E53EE50>
>>> c @ b
<__main__.C object at 0x000001899E53EE20>
>>> 

Seems bug-like to me. Maybe it’s not actually a bug, it seems worth opening a bug report, if for no other reason to get core dev attention. They can pronounce one way or the other.

Not a bug. The reflected version is only ever tried for different types. It’s both an optimization, and logical: if type Foo doesn’t know how to multiply itself, why would it know how to multiply itself the other way?

Please see the documentation here:

https://docs.python.org/3/reference/datamodel.html#object.__radd__

These methods are called to implement the binary arithmetic operations ( + , - , * , @ , / , // , % , divmod() , pow() , ** , << , >> , & , ^ , | ) with reflected (swapped) operands. These functions are only called if the left operand does not support the corresponding operation 3 and the operands are of different types. 4

Footnote 4 explains:

For operands of the same type, it is assumed that if the non-reflected method – such as __add__() – fails then the overall operation is not supported, which is why the reflected method is not called.

2 Likes

Not a bug. The reflected version is only ever tried for different
types. It’s both an optimization, and logical: if type Foo doesn’t
know how to multiply itself, why would it know how to multiply itself
the other way?

No, that’s a design bug. It assumes that the overloaded operator is
symmetric, and will be commutative (like multiplication: ab == ba) or
anticommutative (like division, or subtraction).

In the symmetric cases, like multiplication and division, it makes sense
to ask the question as you do: if the object doesn’t know how to
multiply when it is on the left of the operator, why would it know how
to multiply when it is on the right?

But not all binary operators are symmetric. For example, the in
operator doesn’t have a reflected dunder:

[] in mylist  # calls mylist.__contains__([])

Remember, in this case, the dunder is actually the right-hand reflected
version of a non-existent __contained_by__ dunder.

“If doesn’t know how to search for itself in mylist, how can mylist
know how to search for in itself?”

That reasoning is obviously wrong when it comes to the in operator. It
remains wrong if we overload another operator (even the * multiply
operator!) to behave like in.

Concrete example: a “push” operator:

obj >> database

Only the database decides how to handle having an object pushed into it.
The object being pushed should never get a say in it. And that seemingly
works fine:

>>> class DB:
...     def __rrshift__(self, other):
...             return "DB accepted %r" % (other,)
... 
>>> 42 >> DB()
'DB accepted 42'

Or at least, it works fine until you try to push a database into another
database. And then it breaks. That’s a bug.

I hit send too early. I forgot my last comment. Sorry.

Whether you agree with me that this documented short-cut is a design bug
or not, it seems unlikely to be changed unless there is compelling
use-case for only calling the reflected dunder that cannot be worked
around by defining a non-reflected dunder. E.g. in my early “push”
example, have the DB class define a __rshift__ method that
delegates to the right-hand operand:

def __rshift__(self, other):
    if isinstance(other, DB):
        return type(other).__rrshift__(other, self)
    return NotImplemented

Assuming that works (I haven’t tried it) then I guess we have a work-
around for the operator shortcut for such in model overloaded
operators.

1 Like

Thank you very much for your replies.
It was good for me to post here before reporting this to the bug tracker!

I understand that this is intended behavior.

It’s truly logical.
if A.__matmul__(A) => (A @ A) is failed, A.__rmatmul__(A) => (A @ A) must be failed.
There seems to be no room for objection!
But if I dare say, programming is not always mathematical.
If it were, even distinguishing matmul and rmatmul doesn’t make sense. :upside_down_face:

In my use case, I wanted to use @ only as a postfix operator (I really don’t want to define left-hand side @, therefore my @-ops is neither symmetric nor reflective). So, @steven.daprano’s solution is helpful to me.

class B:
    def __matmul__(self, argv):
        if type(argv) is B:
            return argv.__rmatmul__(self)
        return NotImplemented
    def __rmatmul__(self, argv):
        return argv

Only when both operands are the same, the left-hand side operation can be deduced from the right.
This is also instructive. I think this behavior could be a default action in future python.

Actually, the following C++ code works fine.

struct B {
    template<typename T>
    friend B operator * (const T& x, const B& r) {
        cout << "B::right(*) is called for " << typeid(x).name() << endl;
        return B(r);
    }
};
    B b;
    1 * b; // -> B::right(*) is called for i
    b * b; // -> B::right(*) is called for 1B

but

error: no match for 'operator*' (operand types are 'B' and 'int')
     b * 1;
     ~~^~~