How to pass a python function to a C function as a function pointer in Cython

Python Help
1

Hi all,
I am writing a cython class like this.

cdef class Button:
    cdef CButton* ptr
    cdef object _onClick
    
    def __cinit__(self, ...):
        self.ptr = make_btn_ptr_in_C()

    @onClick.setter
    def onClick(self, pfunc):
        self._onClick = pfunc
        setClickHandler(self.ptr, self._onClick)

This setClickHandler function’s signature is here.

ctypedef void (*EventHandler)(CControl*, EventArgs*)
void* setClickHandler(void* ctl, EventHandler pFunc)

Both CControl and EventArgs defined in pxd file.
So basically, I want to execute a python function in Button’s wndproc function. The wndproc of this button is in C world. So I want to pass a python function from cython world to C world.

2

If pfunc is an arbitrary Python function (with EventArgs as arguments), then I think you need a bridge between that Python object (pfunc) and the EventHandler type. One way might be to add a C-level wrapper function (with EventHandler signature) that when called executes the python level function using PyObject_CallObject.

Turns out that Cython has a little demo of how to define/use a callback.
https://github.com/cython/cython/tree/master/Demos/callback May need to be tweaked a little to make it compile (callback should be cast as cheesefunc).

1 Like
3

Look at the ctypes module. (but I cannot answer questions about it.)

1 Like
4

I would be kind of wary of using ctypes for this, because of this warning

Make sure you keep references to CFUNCTYPE() objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.
Also, note that if the callback function is called in a thread created outside of Python’s control (e.g. by the foreign code that calls the callback), ctypes creates a new dummy Python thread on every invocation.

1 Like
5

Thanks for the reply. I bookmarked this page

But I am not sure how to use this. Anyhow, let me check that.

6

Thank you for the reply. I did that once but faced a dead end at WINFUNCTYPE. But yeah, I think I can use ctypes directly from python for this purpose. setClickHandler can be called with a WINFUNCTYPE object. But it’s burden for my users. They need to set the argtypes and restype for this function in their scripts.

7

Yes, that’s good to read anyway - also if you’re working with Cython or ctypes.
I’ve written my own extension module code in the distance past (even for Python 1.6 when Python didn’t have nice macros yet for dealing with the Python thread state). The main thing for asynchronous callbacks from C to Python (as here) was that the C-level code needed to create a fake PyThreadState, and then also needed to get hold of the GIL before it could safely execute Python code. (Not sure how this works in the current CPython, but those are the things you still need to look out for, I think.)

I would worry about ctypes if that creates a new thread for every button click… (But might be good as initial prototype code.)

1 Like
8

I would like to see your extension module code for learning purpose. If you don’t mind, can you please share it ?

9

That code was actually proprietary, so I cannot share it (and no longer have access). But let me see if I can come up with a skeleton for this (I’ll have to think a bit and double-check.)

Or perhaps someone else has something available: What is wanted is an asynchronous callback from C into Python that executes an arbitrary python function (with fixed argument types). So, this also implies either a registration function to register the python function for the C code or another way to retrieve the function (registration is in principle better, since it allows more control).

1 Like
10

This code seems confusing.

cdef extern from "cheesefinder.h":
    ctypedef void (*cheesefunc)(char *name, void *user_data)
    void find_cheeses(cheesefunc user_func, void *user_data)

def find(f):
    find_cheeses(callback, <void*>f)

cdef void callback(char *name, void *f):
    (<object>f)(name.decode('utf-8'))

find is called from python. And f is the user’s python function. But f is converted to void* and passed as user_data in find_cheeses. In my case, I need to pass f as user_func parameter. That’s confusing me.

11

I was just reminding myself about that. So, typically when a library supports callbacks, it will add an extra void* argument (to the registration function). The design of any kind of generic callback almost makes this unavoidable. The extra argument can be used to pass user data, or extra context, so the user can actually do sth with that. In this case, you need to be able to pass the actual Python function (Python object) into the C code - and it needs to be converted into a void* just to satisfy the function signature (the C code in principle doesn’t need to known about Python).
The void callback(...) function here is the C-level wrapper for the inner Python function.
In this case, find is not called asynchronously though - it will be called from the main Python thread.

1 Like
12

Yeah, registration is what I am trying to do. I can easily convert a Cython cdef function to an EventHandler. But I can’t execute a python function inside it.

13

Yeah, but Cython is not allowing to convert a python object to a void* with the <> casting. Is there any other way ?

14

I didn’t have problems with that when compiling the demo code. I only needed to cast

def find(f):
    find_cheeses(<cheesefunc> callback, <void*>f)
15

Oh I see. So you are converting a python function to void pointer. Okay, then let me try that.

16

I tried this but didn’t worked

cdef class Button:
    cdef void* _onclick

    cdef void callback(self, void* p1, void* p2):
        (<object>self._onclick)(<object> p1, <object>p2)

    def onMouseClick(self, pfunc):
        setClickHandler(self.ptr, <EventHandler>self.callback)

I think it failed because it is a class method.

17

Then I tried this and it worked.

cdef void* gonclick # global variable
cdef void ClickHandler( void* p1, void* p2):
    print("cython func handled click event")
    (<object>gonclick)(<object>p1, <object>p2)

cdef class Button:

    def onMouseClick(self, pfunc):
        global gonclick
        gonclick = <void*>pfunc
        setClickHandler(self.ptr, <EventHandler>ClickHandler)

This worked nicely, But I need to store the function pointers outside the class.

18

tkinter.init _register functions register user-written Python callback functions with C-coded tcl/tk for use with tk objects. They usually wrap the callback with a tkinter CallWrapper, then create a cbname as a handle, then create a tcl function with tk.createcommand, which must be somewhere in _tkinter.c. Perhaps these code pieces can help.

2 Likes
19

Ah, that’s a great help. Thank you. :slight_smile: