# A new API for ensuring/releasing thread states

**URL:** <https://discuss.python.org/t/a-new-api-for-ensuring-releasing-thread-states/83959>\
**Category:** C API\
**Created:** [March 10, 2025, 9:51pm UTC](https://discuss.python.org/t/a-new-api-for-ensuring-releasing-thread-states/83959 "2025-03-10T21:51:51Z")\
**Posts on this page:** 1\
**Showing post:** 20

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [March 12, 2025, 7:30am UTC](https://discuss.python.org/t/a-new-api-for-ensuring-releasing-thread-states/83959/20 "2025-03-12T07:30:13Z")

</div>

> [@steve.dower](#):
>
> So if you’ve got a scenario in mind where your native code starts running on an existing thread and you either want (a) a new Python thread state or (b) to interrupt an existing one without coordination, and can’t determine ahead of time which it’ll be, I’d love to learn more about it.

It actually comes quite easily when you have some code that can either be run synchronously (in an existing Python thread) or in the background / in parallel (in a potentially non-existing Python thread), depending on various execution specifics that _the function itself_ doesn’t control.

For example in Arrow C++ we have IO abstractions that can be implemented for different backends:

> <https://github.com/apache/arrow/blob/2316ce88d08bb9063ab095c5ecc3ab92ae8348af/cpp/src/arrow/io/interfaces.h#L208-L245>

And in PyArrow (the Python bindings for Arrow C++) we have an implementation of these IO abstractions that delegate to a Python file-like object. This is so that PyArrow users can use Arrow C++ functionality with arbitrary file-like objects, including their own (for example you could probably call the Arrow CSV or JSONL reader on a `ZipFile` entry).

Since the C++ IO abstractions can be called in any context for the purpose of reading/writing data, whether they are called from a Python thread or, say, a C++ thread pool thread, entirely depends on what functionality is being called and how. This is not under control of the IO routines themselves.

So, for example, the `Tell()` implementation for Python file-like objects:

> <https://github.com/apache/arrow/blob/2316ce88d08bb9063ab095c5ecc3ab92ae8348af/python/pyarrow/src/arrow/python/io.cc#L214-L216>

wraps its underlying functionality in the `SafeCallIntoPython` wrapper that ensures that Python APIs can be safely called from that point:

> <https://github.com/apache/arrow/blob/2316ce88d08bb9063ab095c5ecc3ab92ae8348af/python/pyarrow/src/arrow/python/common.h#L149-L166>

… and `PyAcquireGIL` there is just a RAII wrapper around the `PyGILState_Ensure`/`PyGILState_Release` 🙂

(yes, at some point we noticed that taking the GIL is not sufficient and you also need to ensure you don’t have an error status set)

---

_[View the full topic](https://discuss.python.org/t/a-new-api-for-ensuring-releasing-thread-states/83959)._