How to share module state among multiple instances of an extension module?

I see two main ways:

  • Simple but still safe: Make the module fail to import more than once per process. There’s a simple recipe in the HOWTO, but it’ll need an additional lock to work with multiple GILs.
  • Use lock-protected process-global state, with its own reference counting to ensure it’s cleaned up when no longer needed. Don’t treat one interpreter specially.

If you have shared state that needs to be correctly cleaned up when everyone’s done with it, you should do one of these now – even with a single GIL.

(Next thing to think about: how do you safely share per-process state with other people’s modules, and perhaps even non-Python wrappers? But to solve that you’d probably need improvements in the library you’re wrapping.)


@moderators, the thread starting with Mark-André’s first message could be extracted into its own topic.