A New (C) API For Extensions that Need Runtime-Global Locks

It is similar to Petr’s idea (so we should not scratch that), but it simplifies the API for the extension modules by providing convenient PEP 489 module slots.

Pseudo-code without error checking, partly borrowed from PEP 489:

def PyModule_ExecDef(module, def):
    # ...

    exec = None
    g_init = None
    for slot, value in def.m_slots:
        if slot == Py_mod_exec:
            exec = value
        if slot == Py_mod_global_init:
            g_init = value  # In Petr's example, this would be setup_my_global_state()

    if g_init:
        if global_state_refcnt(def) == 0:
            acquire_global_lock()
            g_init(module)
            global_state_incref(def)
            release_global_lock()

    if exec:
        exec(module)

# Called when module is unloaded (or dealloc'd), for example at interpreter shutdown
def unload_module(module, def):
    # ...

    for slot, value in def.m_slots:
        if slot == Py_mod_global_exit:
            g_exit = value  # In Petr's example, this would be teardown_my_global_state()
            if global_state_refcnt(def) == 1:
                acquire_global_lock()
                g_exit(module)
                release_global_lock()
            global_state_decref(def)

The nice thing with such an API, is that the extension modules can get rid of possibly-hard-to-get-right boilerplate locking code; they can simply provide functions for setup and tear-down of global state. The runtime will make sure to call these functions when needed; that responsibility is not on the extension module author.

Unless I’m misreading you, I believe you should be able to solve that using a global lock API and the ordinary Py_mod_exec and m_free/m_clear coupled with the proposed Py_mod_global_ slots.

1 Like