With free-threading becoming ready, there’s going to be more and more interest in ThreadPoolExecutor.
However it seems the default (python-based) implementation’s speed could be improved. Likely with custom C code.
Here is a simple Python code to highlight the issue:
from concurrent.futures import ThreadPoolExecutor
from cymade.threadpool import ThreadPool
import gc
import time
gc.disable()
pool = ThreadPoolExecutor(max_workers=1)
def f():
def g():
e = 0
for i in range(1000):
e += 1
return e
a = time.monotonic()
for _ in range(1000000):
fut = pool.submit(g)
b = time.monotonic()
fut.result()
c = time.monotonic()
print(b-a, c-b)
f()
pool.shutdown()
pool = ThreadPoolExecutor(max_workers=4)
f()
pool.shutdown()
pool = ThreadPool(max_workers=1)
f()
pool.shutdown()
pool = ThreadPool(max_workers=4)
f()
Where cymade’s ThreadPool is a Cython made ThreadPoolExecutor that can be found here cymade/cymade/threadpool.pyx at main · axeldavy/cymade · GitHub.
The code, run on Cpython 3.13 freethreaded build (with PYTHON_GIL=0) prints on my computer:
6.369051308000053 46.233525454999835
15.782200638999711 16.54086810300032
0.3838555649999762 44.35501626699988
0.9758839780006383 25.223620621998634
The first column is the time to submit one million functions to the thread pool. The second column is approximately the time after that submission for the work to finish (note the actual time running the work is actually longer as the first submission starts before the second timestamp)
The first row is with the default threadpool, max_workers=1, the second with max_workers=4. The third and fourth row are the same result but with a custom Cython-made threadpool (which is there mainly for demonstrating the issue but should not be considered a solution).
The results seem to indicate that there is significant overhead with the default ThreadPoolExecutor for work submission. The cost of running the function g is rather small, but not zero either. Yet just submitting the function can be in the same order of cost as running it. In my own usage of ThreadPoolExecutor I can have submissions of even smaller functions. I believe it would be worthwhile for CPython to improve the speed of this feature.
I assume if work was to be done on this topic, it would be as a C custom implementation in CPython. Would contributions on this topic be accepted ? Or, if you think it is a hard contribution, is there interest from someone more experimented ?