A new API for ensuring/releasing thread states

pitrou · March 12, 2025, 7:30am

It actually comes quite easily when you have some code that can either be run synchronously (in an existing Python thread) or in the background / in parallel (in a potentially non-existing Python thread), depending on various execution specifics that the function itself doesn’t control.

For example in Arrow C++ we have IO abstractions that can be implemented for different backends:

github.com/apache/arrow

cpp/src/arrow/io/interfaces.h

2316ce88d


      
          class ARROW_EXPORT InputStream : virtual public FileInterface, virtual public Readable {
           public:
            /// \brief Advance or skip stream indicated number of bytes
            /// \param[in] nbytes the number to move forward
            /// \return Status
            Status Advance(int64_t nbytes);
          
            /// \brief Return zero-copy string_view to upcoming bytes.
            ///
            /// Do not modify the stream position.  The view becomes invalid after
            /// any operation on the stream.  May trigger buffering if the requested
            /// size is larger than the number of buffered bytes.
            ///
            /// May return NotImplemented on streams that don't support it.
            ///
            /// \param[in] nbytes the maximum number of bytes to see
            virtual Result<std::string_view> Peek(int64_t nbytes);
          
            /// \brief Return true if InputStream is capable of zero copy Buffer reads
            ///

This file has been truncated. show original

And in PyArrow (the Python bindings for Arrow C++) we have an implementation of these IO abstractions that delegate to a Python file-like object. This is so that PyArrow users can use Arrow C++ functionality with arbitrary file-like objects, including their own (for example you could probably call the Arrow CSV or JSONL reader on a ZipFile entry).

Since the C++ IO abstractions can be called in any context for the purpose of reading/writing data, whether they are called from a Python thread or, say, a C++ thread pool thread, entirely depends on what functionality is being called and how. This is not under control of the IO routines themselves.

So, for example, the Tell() implementation for Python file-like objects:

github.com/apache/arrow

python/pyarrow/src/arrow/python/io.cc

2316ce88d


      
          Result<int64_t> PyReadableFile::Tell() const {
            return SafeCallIntoPython([=]() -> Result<int64_t> { return file_->Tell(); });
          }

wraps its underlying functionality in the SafeCallIntoPython wrapper that ensures that Python APIs can be safely called from that point:

github.com/apache/arrow

python/pyarrow/src/arrow/python/common.h

2316ce88d


      
          // A helper to call safely into the Python interpreter from arbitrary C++ code.
          // The GIL is acquired, and the current thread's error status is preserved.
          template <typename Function>
          auto SafeCallIntoPython(Function&& func) -> decltype(func()) {
            PyAcquireGIL lock;
            PyObject* exc_type;
            PyObject* exc_value;
            PyObject* exc_traceback;
            PyErr_Fetch(&exc_type, &exc_value, &exc_traceback);
            auto maybe_status = std::forward<Function>(func)();
            // If the return Status is a "Python error", the current Python error status
            // describes the error and shouldn't be clobbered.
            if (!IsPyError(::arrow::internal::GenericToStatus(maybe_status)) &&
                exc_type != NULLPTR) {
              PyErr_Restore(exc_type, exc_value, exc_traceback);
            }
            return maybe_status;
          }

… and PyAcquireGIL there is just a RAII wrapper around the PyGILState_Ensure/PyGILState_Release

(yes, at some point we noticed that taking the GIL is not sufficient and you also need to ensure you don’t have an error status set)