Since this was mentioned as still relevant:
Could you start writing a PEP? The recommended sections should guide you – why we need a solution, what we need from the solution (what exactly are the use cases? what do we need from CPython?), and then how it can be implemented.
My thoughts on this: it might be useful to separate the underlying “C” struct from mapping to Python types. The buffer protocol might be extendable to do the former well, but for things like “these 8 bytes are a DateTime” or “this uint8
should be treated as a Python bool
”, up to “wrap this struct in a custom class”… I’m not convinced that a universal bytecode-based format can work.
Ideally, whatever needs to be in CPython would be tightly scoped, so we could call it done rather soon, leaving further iterations to external projects.
Perhaps CPython can “simply” reserve prefixes like [np:...]
or [arrow:...]
for external specifications? I mean, it already doesn’t really care, maybe we just need to document it and add nice error messages.