I’m looking for guidance on whether CPython should expose a public C API to allow extension modules to query whether the GIL is currently enabled, for example:
int PyInterpreterState_IsGilEnabled(PyInterpreterState *interp);
Context:
A Python-level diagnostic API, sys._is_gil_enabled(), has already been added.
Some C extensions may want to make similar diagnostic decisions without calling back into Python (e.g. via PySys_GetAttr + PyObject_CallNoArgs).
The use case is primarily reporting, diagnostics, and choosing reasonable defaults (e.g. threading vs multiprocessing backends), not controlling GIL behavior.
Can you go into a little bit more detail about why the Python API for this isn’t usable? What are you doing specifically in your extension such that that the Python overhead is important for peformance?
To clarify, the Python API is usable, and this isn’t about tight inner-loop performance. The motivation is more about layering, ergonomics, and correctness for C extensions.
Some examples of where a direct C API would be preferable:
C extensions that want to make early or one-time decisions (e.g. choosing a threading vs multiprocessing backend, or enabling certain code paths) during initialization, where calling back into Python is awkward or undesirable.
Diagnostic or reporting code in C extensions (similar in spirit to Py_IsInitialized or other runtime-introspection APIs) that would prefer not to depend on Python-level attribute lookups and calls.
Situations where extensions are already operating at the C runtime/interpreter layer and want to query interpreter state directly, rather than re-entering Python and managing references, error handling, etc.
So this isn’t primarily about saving a few cycles, but about providing a clear, supported way for C code to query interpreter state, analogous to other CPython-specific introspection APIs.
That said, if the consensus is that the existing Python-level API is sufficient and a public C API would add too much surface area, I’m fine with that conclusion as well.
I think it would make more sense to make _PyEval_IsGILEnabled public rather than add PyInterpreterState_IsGilEnabled. Passing an interp parameter is more friction for users.
Exposing an existing helper like _PyEval_IsGILEnabled does seem simpler and avoids the need for extensions to handle an interpreter pointer explicitly.
Would the intent be to make this a public, CPython-specific C API (i.e. without the leading underscore), or to expose it as-is in public headers? And assuming there’s agreement on that direction, would it be OK for me to work on the necessary changes (headers, docs, tests)?
This should be an unstable C API, since this is pretty specific to CPython and we might want to change/remove it.
That said, to add C APIs, you need to open an issue with the C API working group. Please do that and get their approval (via a vote) before opening any PRs.
I’ll open an issue with the C API working group to propose this as an unstable, CPython-specific API and will wait for their approval before proceeding further.
Just to confirm: should any next step here (such as initiating a CAPWG vote) be handled by a core dev, or should contributors wait unless explicitly asked?
It’s a bit of a gray area. Generally speaking, all new C APIs should go through the WG, but there are sometimes exceptions. PyUnstable_Object_Dump and PyUnstable_Object_EnableDeferredRefcount were both APIs that went through them, for example. Sometimes the WG is skipped if there’s a rush to get the function out (such as with PyUnstable_Object_IsUniqueReferencedTemporary).
I’m inclined to point to the WG here, because there’s plenty of time before the 3.15 beta freeze, and as far as I can tell, there’s some ambiguity about the actual use case (I’m pretty sure the examples provided above were generated by an LLM).
Yeah - at least from what I’ve found, knowing if the GIL has been enabled is much less useful than you’d initially think. It only really tells you the situation at a single moment in time so you almost can’t do anything with it because it might have changed by the time it matters. You basically have to have a really short block that you know can’t be interrupted.
I suspect the function probably should exist. But it also should have a big note in the documentation saying “this is useless - don’t use it”.
No, they don’t need one, though it doesn’t hurt to request one.
But, the discussion here is still important.
With PySys_GetAttr+PyObject_CallNoArgs available, I don’t think we need a C function.
Do you have any real-world example?
If you’re doing this, you probably want to compile two versions of the library – one for free-threaded builds and non-free-threaded ones. That is, use Py_GIL_DISABLED.
If you do need to do it at runtime, there’s PySys_GetAttr+PyObject_CallNoArgs.
That seems somewhat plausible. Do you have any real-world example, though?
As far as I know, enabling the GIL in a free-threaded build is a non-reversible operation that already comes with a big warning for the user, and it’s only done on module import.
I wonder if a PySys_Audit event would serve these use cases better.
This isn’t really a use case; it’s just a longer way to say “Situations where I want to call PyInterpreterState_IsGilEnabled rather than the Python API”.
Arguably if you’re trying to avoid the possibility of the interpreter being interrupted and the value changing before you can use it, then a C API is better than a Python function call.
I think there’s three states: gil disabled, gil temporarily enabled, and gil permanently enabled. I think it may go through the temporary state on single phase imports (until it knows for sure what they require).
The temporary state probably makes it even harder to do anything useful with the information.
“Early or one-time decisions”: The temporary state could be treated as GIL disabled.
“Diagnostic or reporting code”: For that we should expose all 3 states. Or even a human-readable string that we can change in the future if some other state is added.
Nah. If the GIL is irreversibly enabled, there’s no race any more. In other cases, another thread can change the value at any time, so C vs. Python doesn’t matter.
The main reason we’ve been hesitant to add these APIs is because we think you should code as if the GIL is not there (or alternatively, your non-freethreaded build assumes the GIL is there, primarily through the two sets of macros being defined differently).
Code that tries to handle both cases at runtime is going to be harder to write and much harder to get correct. I’m not calling it impossible, mind, which is why a clearly marked “this is not intended semantics” function is probably okay to have.
Something like choosing between a thread pool or process pool really ought to be a user-configurable option (in code) anyway, which means the Python function is likely to be accessible. But even then, defaulting in a free-threaded build to a thread pool is a totally fine choice - the user has already indicated their intent, and you’re just following that.
Otherwise you end up in a situation where previously imported modules change your semantics - perhaps breaking you. And we end up in a situation where changing the core behaviour (e.g. making it more likely to have no GIL there) changes your library’s semantics, and those often force us to not make the improvement we want.
So it really is best to depend on intent (in this case, the mode of the target runtime) rather than depending on the current status. And by not offering an obvious API, we encourage/force more people to choose correctly here.