Docstrings for dunder methods

The __add__ method Numpy arrays performs elementwise addition. For Python lists, it performs concatenation. However, the docstrings don’t tell us about this. The docstrings describe how __add__ is called rather than what it actually does. I’ve seen this be a problem for beginners over and over again.

>>> from numpy import ndarray
>>> help(ndarray.__add__)
Help on wrapper_descriptor:

__add__(self, value, /)
    Return self+value.

>>> help(list.__add__)
Help on wrapper_descriptor:

__add__(self, value, /)
    Return self+value.

Currently, pure Python classes can provide custom docstrings for dunder methods. Offhand, I’m not sure how to do this for C types. It might need new argument clinic magic to create an alternate docstring and a way for a wrapper_descriptor or inspect to find it.

class A:
    def __add__(self, other):
        'Concatenate two A instances to make a new A.'

>>> help(A.__add__)
Help on function __add__ in module __main__:

__add__(self, other)
    Concatenate two A instances to make a new A.
4 Likes

Related:
The brand new @critical_section Argument Clinic directive is currently being used to help implement free-threading. @colesbury has floated the idea of adding type slot support to Argument Clinic, to further ease implementation of PEP-703. With type slot support in place, it would be easy to add custom docstrings for dunder methods in C types.

2 Likes

IIUIC, it’s impossible right now. All wrapper descriptors (i.e. the __add__ dunder for different types) share pointer to same wrapperbase struct.
We could extend the PyWrapperDescrObject with a doc field to support this kind of overloading.

Quick draw
>>> help(int.__add__)
Help on wrapper_descriptor:

__add__(self, value, /) unbound builtins.int method
    Return self+value.

>>> help(list.__add__)
Help on wrapper_descriptor:

__add__(...) unbound builtins.list method
    Boo!

diff --git a/Include/cpython/descrobject.h b/Include/cpython/descrobject.h
index bbad8b59c2..25545fc9e4 100644
--- a/Include/cpython/descrobject.h
+++ b/Include/cpython/descrobject.h
@@ -55,6 +55,7 @@ typedef struct {
     PyDescr_COMMON;
     struct wrapperbase *d_base;
     void *d_wrapped; /* This can be any function pointer */
+    const char* doc;
 } PyWrapperDescrObject;
 
 PyAPI_FUNC(PyObject *) PyDescr_NewWrapper(PyTypeObject *,
diff --git a/Objects/descrobject.c b/Objects/descrobject.c
index df546a090c..cabd01e8d9 100644
--- a/Objects/descrobject.c
+++ b/Objects/descrobject.c
@@ -687,7 +687,7 @@ static PyObject *
 wrapperdescr_get_doc(PyObject *self, void *closure)
 {
     PyWrapperDescrObject *descr = (PyWrapperDescrObject *)self;
-    return _PyType_GetDocFromInternalDoc(descr->d_base->name, descr->d_base->doc);
+    return _PyType_GetDocFromInternalDoc(descr->d_base->name, descr->doc);
 }
 
 static PyObject *
@@ -695,7 +695,7 @@ wrapperdescr_get_text_signature(PyObject *self, void *closure)
 {
     PyWrapperDescrObject *descr = (PyWrapperDescrObject *)self;
     return _PyType_GetTextSignatureFromInternalDoc(descr->d_base->name,
-                                                   descr->d_base->doc, 0);
+                                                   descr->doc, 0);
 }
 
 static PyGetSetDef wrapperdescr_getset[] = {
@@ -1019,6 +1019,7 @@ PyDescr_NewWrapper(PyTypeObject *type, struct wrapperbase *base, void *wrapped)
     if (descr != NULL) {
         descr->d_base = base;
         descr->d_wrapped = wrapped;
+        descr->doc = base->doc;
     }
     return (PyObject *)descr;
 }
@@ -1402,7 +1403,7 @@ static PyObject *
 wrapper_doc(PyObject *self, void *Py_UNUSED(ignored))
 {
     wrapperobject *wp = (wrapperobject *)self;
-    return _PyType_GetDocFromInternalDoc(wp->descr->d_base->name, wp->descr->d_base->doc);
+    return _PyType_GetDocFromInternalDoc(wp->descr->d_base->name, wp->descr->doc);
 }
 
 static PyObject *
@@ -1410,7 +1411,7 @@ wrapper_text_signature(PyObject *self, void *Py_UNUSED(ignored))
 {
     wrapperobject *wp = (wrapperobject *)self;
     return _PyType_GetTextSignatureFromInternalDoc(wp->descr->d_base->name,
-                                                   wp->descr->d_base->doc, 0);
+                                                   wp->descr->doc, 0);
 }
 
 static PyObject *
diff --git a/Objects/object.c b/Objects/object.c
index df14fe0c6f..4e1de930a7 100644
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -2337,6 +2337,11 @@ _PyTypes_InitTypes(PyInterpreterState *interp)
         }
     }
 
+    PyObject *dict = PyType_GetDict(&PyList_Type);
+    PyObject *descr = PyDict_GetItem(dict, &_Py_ID(__add__));
+    Py_DECREF(dict);
+    ((PyWrapperDescrObject *)descr)->doc = "Boo!";
+
     // Must be after static types are initialized
     if (_Py_initialize_generic(interp) < 0) {
         return _PyStatus_ERR("Can't initialize generic types");

Probably, first there should be a way to do this without AC…

1 Like