Buffer Protocol with PEP3118 struct string syntax

[ngoldbaum (Nathan Goldbaum)]

@encukou

Since you posed in Core Development: Yes, I’m willing to mentor someone who’d want to put in the work of researching and implemening something that would suit struct, ctypes and numpy (and others?).

If there’s interest in improving the buffer protocol implementation in Python, something that would make this much more attractive from the NumPy side is to make it easier to work with arbitrary data types. See this thread for the last time NumPy developers broached this.

While the Arrow C Data interface would definitely help in many use cases, it’s not quite what NumPy needs (e.g. >1D data, strides, etc), and the buffer protocol has the nice advantage that it’s already used throughout the ecosystem.

It would also be nice to fix the thread safety issues inherent in the design of the buffer protocol. Producers of buffers ought to have a way to do borrow checking on the buffer.

Sorry if this thread is actually unrelated to coming up with a sucessor to the buffer protocol. I just thought I’d throw it out there because it would unlock a lot of cool functionality in the future to allow zero-copy data sharing with arbitrary data layouts.