Choice of complex buffer protocol format intentional break with PEP?

Core Development
16

I think you should convince SC on this, not just me :wink:

No, the discussion was about whether this “specification” was implemented somewhere. But the text doesn’t look as a specification at all. It just happened that some proposals from that section are implemented in some way across the Python ecosystem.

Lets not use this argument, unless SC reconsider their position.

Sorry, I don’t see much difference:

>>> import inspect
>>> import numpy as np
>>> a = np.array([1-1j, 1j])
>>> a.dtype.char
'D'
>>> m = a.__buffer__(inspect.BufferFlags.FULL)
>>> m.format
'Zd'

Both type codes are visible for me as end user. It just happens to be that the later code is not documented in the NumPy, besides reference to the PEP. BTW, the memoryview docs says about format attribute:

A string containing the format (in struct module style) for each element in the view. A memoryview can be created from exporters with arbitrary format strings, but some methods (e.g. tolist()) are restricted to native single element formats.

I like this approach, as it much more local: we need to change the memoryview and the array module, which uses buffer protocol.

So, in meanwhile, we will accept importing with either 'D' or 'Zd' format codes. And the array module will export complex arrays with the 'Zd' format code. Ditto for 'F'. Should this be a part of transition to 'D'/'F' type codes or will stay forever?

Edit:

A patch to play with.

This should fix Issues · numpy/numpy · GitHub

diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
index a86a7561271..d9389797a71 100644
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -2960,6 +2960,12 @@ array_buffer_getbuf(PyObject *op, Py_buffer *view, int flags)
         if (sizeof(wchar_t) >= 4 && self->ob_descr->typecode == 'u') {
             view->format = "w";
         }
+        if (self->ob_descr->typecode == 'F') {
+            view->format = "Zf";
+        }
+        if (self->ob_descr->typecode == 'D') {
+            view->format = "Zd";
+        }
     }
 
     self->ob_exports++;
diff --git a/Objects/memoryobject.c b/Objects/memoryobject.c
index 4cbbb7eb7cd..ca31ec8e5a2 100644
--- a/Objects/memoryobject.c
+++ b/Objects/memoryobject.c
@@ -1220,6 +1220,21 @@ get_native_fmtchar(char *result, const char *fmt)
     case 'D': size = 2*sizeof(double); break;
     case '?': size = sizeof(_Bool); break;
     case 'P': size = sizeof(void *); break;
+    case 'Z':
+        {
+            if (fmt[1] == 'f' && fmt[2] == '\0') {
+                size = 2*sizeof(float);
+                *result = 'F';
+            }
+            else if (fmt[2] == 'd' && fmt[2] == '\0') {
+                size = 2*sizeof(double);
+                *result = 'D';
+            }
+            else {
+                return -1;
+            }
+            return size;
+        }
     }
 
     if (size > 0 && fmt[1] == '\0') {
@@ -2196,6 +2211,14 @@ adjust_fmt(const Py_buffer *view)
     const char *fmt;
 
     fmt = (view->format[0] == '@') ? view->format+1 : view->format;
+    if (fmt[0] == 'Z' && fmt[1] != '\0') {
+        if (fmt[1] == 'f' && fmt[2] == '\0') {
+            return "F";
+        }
+        if (fmt[1] == 'd' && fmt[2] == '\0') {
+            return "D";
+        }
+    }
     if (fmt[0] && fmt[1] == '\0')
         return fmt;

So, it looks that “letter-saving” convention doesn’t save us letters at all? :slight_smile: