PEP 703: Making the Global Interpreter Lock Optional (3.12 updates)

[gpshead (Gregory P. Smith)]

@markshannon

For the purposes of this discussion, let’s categorize parallel application into three groups:

1. Data-store backed processing. All the shared data is stored in a data store, such a Postgres database, the processes share little or no data, communicating via the data store.
2. Numerical processing (including machine learning) where the shared data is matrices of numbers.
3. General data processing where the shared data is in the form of an in-memory object graph.

Python has always supported category 1, with multiprocessing or through some sort of external load balancer.
Category 2 is supported by multiple interpreters.
It is category 3 that benefits from NoGIL.

How common is category 3?
All the motivating examples in PEP 703 are in category 2.

While understandable, I don’t think this is a wholly accurate categorization.

TL;DR - We do see demand from the numerical processing / ml side for free threaded 703-style Python:

We hear from our data science & ML teams at work that the CPython bottleneck of any form is always what must be worked around in their software stacks. It is the computation between calls into specialized-kernels and in between offloading mass computations to TPUs and GPUs where they want a throughput boost aka latency reduction. As that in-Python wall time often means the dedicated special xPU resource sits allocated to them but idle.

Thread parallelism for numerical computation, what you call (2), would be very much appreciated. They’ll write code that bridges the (2)-(3) categorization that could benefit latency wise by being parallelized across the N-NNN available cores on a machine.

Multiple interpreters (be it processes or be it our upcoming per-interpreter-gil subinterpreters that nobody can actually use yet) lack a good shared data from Python story so they don’t support that case as well as threads should. (regardless, I’m interested to see what use they can make of per-interpreter-gil subinterpreters or not - work done there will likely be useful in a potential future free-threaded pep703 python as well)