Define __add__, __radd__, __iadd__ using C

I am trying to use Argument Clinic for __add__, __radd__, and __iadd__ methods, but it turns out that these special methods cannot be defined using Argument Clinic.
Here is the output of the Argument Clinic:

Error in file '.\\Modules\\_foo.c' on line 42:
'__add__' is a special method and cannot be converted to Argument Clinic!

I am wondering the correct way of defining these special methods using the C API.

Assuming you are not using the limited API.

Create a PyNumberMethods object and fill in its fields lije nb_add etc.
Then set the type’s tp_as_number to that object.

1 Like

The stable ABI way is to #include <typeslots.h> (source) and use the constants in there with PyType_FromSpec to create a type with those methods filled in.

The docs for each slot (e.g. nb_add) will tell you what the function signature should look like. These don’t use the normal Python calling conventions - they are much more direct.

3 Likes

Yes, except: <typeslots.h> is included from Python.h, you don’t need to include it directly.

1 Like

Is that not the one that you have to include manually? Which one am I thinking of?

1 Like

You perhaps think about structmember.h which was needed for PyMemberDef. It is no longer needed if you use the Py_ prefixed constants (Py_T_INT instead of T_INT).

3 Likes

Thank you so much for your help.

I noticed that the slot sq_concat is responsible for only __add__ but not __radd__. nb_add does support __radd__ but it cannot tell the order of the operands. I was also wondering if it is possible to explicitly define __radd__, which might be useful in some cases, for example, concatenating two sequences of different types (and similarly for other operators).

I appreciate your help with this question.

For a mutable class, you can define __radd__ like in Python: construct a callable object, and set it on the class using PyObject_SetAttr. But this way, you’ll lose the performance benefits of the C slots.

So it’s not usually done that way. A typical nb_add function checks the types of both of its arguments, and do Py_RETURN_NOTIMPLEMENTED if it can’t handle them.

Argument Clinic was created for PyMethodDef only; type slots/special methods are deliberately not supported[1]. @corona10 recently added basic support for PyGetSetDef methods, and @colesbury wants to enable support for some type slots:


  1. except for __new__ and __init__ ↩︎

The main purpose of Argument Clinic – to generate signatures for builtin functions and methods. After the start of the implementation, a second goal arose – reducing the overhead of calling builtin functions and methods. Argument Clinic can use the most efficient calling convention and generate the more efficient code for converting positional and keyword arguments to C values.

But special methods like __add__ has fixed signature. They use fixed calling convention which is already efficient, and they need original argument values, to chose the behavior depending on their type, and because they usually support larger range of values than represented by simple C types. I do not see how Argument Clinic is applicable here.

The same is true for getters and setters.

1 Like

See also PEP-436 The Argument Clinic DSL for more information about the rationale and goals.

Quoting the PEP :wink:

Future goals of Argument Clinic include:

  • providing signature information for builtins,

“Generate signatures” and “providing signature information” (implying to one other than the C compiler) are different things.

The main and original purpose is indeed to streamline use of PyArg_ParseTupleAndKeywords within our own sources, particularly making sure conversions were applied more consistently. Chances are that the optimized call conventions that came later would not have been attempted without all the parameter handling being streamlined already.

1 Like

+1

Continuing the digression regarding getters/setters and type slots: with free-threading, it has been obvious that we can use Argument Clinic to help reduce boilerplate code when introducing critical sections. Much of the free-threading work that has been done has been greatly simplified by the fact that we now have the @critical_section directive and PyGetSetDef support. I support the idea of adding type slot support to Argument Clinic.

Note also that the Faster CPython team has some ideas for Argument Clinic: