Py_CLEAR and Py_SETREF without UB

sergey-miryanov · February 15, 2025, 8:01pm

Can someone explain to me why Py_CLEAR and Py_SETREF don’t have UB (ifndef _Py_TYPEOF)? Am I missing something from the C standard, or memcpy from NULL allowed?

MegaIng · February 15, 2025, 8:24pm

The relevant lines of Py_CLEAR are this:

            PyObject *_null_ptr = _Py_NULL; \
            memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*)); \

This doesn’t memcpy from a null pointer, this copies a null pointer, i.e. we are passing in a pointer to a NULL pointer to memcpy. Py_SETREF is similar. Watch for the extra &.

sergey-miryanov · February 16, 2025, 5:57am

Oh, I see now! I should sleep at night and not ask dumb questions!
Thanks for explanation!

pitrou · February 16, 2025, 9:56am

For those who were wondering about the use of memcpy here, this is the explanation as outlined in the source comments:

github.com/python/cpython

Include/refcount.h

798f8d3ea


      
          *
          * If available, use _Py_TYPEOF() to use the 'op' type for temporary variables,
          * and so avoid type punning. Otherwise, use memcpy() which causes type erasure
          * and so prevents the compiler to reuse an old cached 'op' value after
          * Py_CLEAR().
          */

And the reason _Py_TYPEOF might not be available is that it relies on non-standard C:

github.com/python/cpython

Include/pyport.h

798f8d3ea


      
          // _Py_TYPEOF(expr) gets the type of an expression.
          //
          // Example: _Py_TYPEOF(x) x_copy = (x);
          //
          // The macro is only defined if GCC or clang compiler is used.
          #if defined(__GNUC__) || defined(__clang__)
          #  define _Py_TYPEOF(expr) __typeof__(expr)
          #endif

(however, C23 has typeof which will probably be usable for this in CPython in a few years, while C++ has decltype and auto which are likely alternatives for this use case )

Edit: even recent MSVC seems to have __typeof__