@kumaraditya303 just published a post on the Quansight blog I thought might be of interest here. It summarizes work he led and I helped out on to improve multithreaded scaling of NumPy. I suspect there will be a lot of debugging stories like this across the ecosystem to chase down scaling bottlenecks under threads so I hope the anecdotes and explanations of all the different bottlenecks we ran into are helpful.
https://labs.quansight.org/blog/scaling-numpy-on-free-threaded-python
Very interesting read and good that there is no longer a degradation beyond 18 threads.
Was the benchmarking on a 32 core machine? If so improvement over single core, factor of 4, not that great? Also if 32 core, what happens if I ask for 64 threads?
Asking these questions because it is a topic I’m interested in and I use NumPy!
Thanks.
I think you misread the results. The final version is 4x faster than a multiprocessing implementation (also using 32 cores). The graph shows there is more like a 32x speedup (approximately, I can’t see it exactly) compared to single core.
Thanks, I think you’re right
Initial Sam Gross Nogil presentation in 2022 was scaling to 8 cores, now Python-3.15 / Numpy-2.5 may scale up to 32x for everyone, it’s amazing. But when does it stop scaling with more cpus ? Is there a theorical limit ? in fact at 40’29" it was showing no gil performance on Hanabi test till 80 threads, going down after 50 threads at that early version of free-threading, on the “Hanabi test”