PEP 788: PyInterpreterRef: Interpreter References in the C API

I appreciate the effort that’s gone into this PEP, but I’m somewhat concerned about the overall complexity we’re introducing here. While the reference counting approach is sound in principle, we’re essentially asking extension authors to learn and correctly use an entirely new paradigm for interpreter lifecycle management.

My main worry is that we’re replacing one set of footguns (PyGILState_Ensure issues) with a different set (managing strong vs weak references, understanding when to use Dup vs Close, getting the refcount semantics right). The cognitive load feels significant, especially for maintainers who aren’t deeply embedded in CPython internals.

Additionally, the deprecation of PyGILState_Ensure is going to create a substantial migration burden across the ecosystem. There are thousands of extensions using these APIs, and many are no longer actively maintained. We risk fragmenting compatibility or leaving orphaned code that won’t work with newer Python versions.

I wonder if we’ve fully explored simpler alternatives - perhaps something that keeps more of the existing mental model but just fixes the specific issues with subinterpreters and thread safety? The current proposal feels like we’re doing a fairly large redesign when a more surgical fix might suffice.

That said, I understand the desire to get this right for the long term, and I don’t want to block progress. Just wanted to raise these concerns.

Hi, thanks for the feedback. Please take a look at the new revision of this proposal, which abandons reference counting entirely. (Please also reply on the new thread.)

That was the original proposal. It was not received well; making PyInterpreterState * implicitly “strong” or “weak” was very confusing.

3 Likes