Access the buffer protocol using ctypes

Ideas
1

I’m trying to write code to access the internal buffer of any object passed by the user (without copying the data to another location).

After scouring the internet for a way to do this, the following ways seem to be recommended:

1- Using from_buffer():

buffer = (ctypes.c_ubyte * length).from_buffer(data)

This seems to only work for writable buffers. For example, It will fail if you pass a bytes object.

2- Direct Pass:

If the function expects a c_char_p or a c_void_p, then bytes objects can be passed directly, but objects can’t be passed this way.

I’m able to get the length of the buffer through memoryview objects, but I’m unable to find a way to access the address of the buffer. Is there a way to do that? Maybe there should be an easier way to interact with buffer protocol through ctypes?

2

I was hit by this too. Best I could get was to use PyObject_GetBuffer().

It’s slightly over embellished but you should be able to get the gist from here. Note that you have to specify whether you want a C contiguous or arbitrary array and beware that you also need to release the buffer when you’re done with it or it’ll be a memory leak. It’s also surprisingly slow so you need to be spending some time in C execution to make it worthwhile.

I wanted to request that this become more accessible ages ago but found some old mailing list where the idea was already rejected (too raw and pointerey).

3

I found this issue posted back when the behavior of from_buffer() was changed to only allow access to writable buffers.

This is my attempted solution which does copy the data. There might some syntax errors in the code:

with memoryview(data) as view:
    buf = (ctypes.c_ubyte * view.nbytes).from_buffer_copy(view)
    some_c_func(buf, view.nbytes)

But yeah, the performance penalty is not acceptable. I ended up going back to my Cython solution.

I do think that ctypes should have a way to implement const (read-only) pointers as mentioned in the old issue, plus a way to work with the buffer protocol. Though, I don’t know if there is much interest among the devs in doing so.