I’ve been thinking about the proposal in Enabling thread_inherit_context and context_aware_warnings by default on both builds - #30 by ngoldbaum today with an AI model’s help. As they’re wont to do, it pointed out a related issue in ThreadPoolExecutor that I think we should fix.
The problem
ThreadPoolExecutor spawns workers lazily from inside submit(), and work items run directly in the worker’s ambient thread context.
With thread_inherit_context=1 (the free-threaded default), a spawned thread gets a snapshot of the spawning thread’s context when it is lazily started. A pool worker permanently captures whatever context was active at whichever submit() call happened to create it.
To see what I mean, consider the following script:
import contextvars
import sys
from concurrent.futures import ThreadPoolExecutor
var = contextvars.ContextVar("var", default="<unset>")
def read():
return var.get()
flag = getattr(sys.flags, "thread_inherit_context", "n/a")
print(f"thread_inherit_context={flag}\n")
# Pool A is "warmed" before var is set: its single worker is spawned here.
warm = ThreadPoolExecutor(max_workers=1)
warm.submit(lambda: None).result()
var.set("A")
print(f"warm pool, submitted while var='A' -> worker sees {warm.submit(read).result()!r}")
var.set("B")
print(f"warm pool, submitted while var='B' -> worker sees {warm.submit(read).result()!r}")
# Pool B is created now, so its worker is spawned during the first submit,
# while var='B' happens to be set.
cold = ThreadPoolExecutor(max_workers=1)
print(f"cold pool, submitted while var='B' -> worker sees {cold.submit(read).result()!r}")
var.set("C")
print(f"cold pool, submitted while var='C' -> worker sees {cold.submit(read).result()!r}")
Output on 3.14.6, GIL build, defaults (thread_inherit_context=0):
warm pool, submitted while var='A' -> worker sees '<unset>'
warm pool, submitted while var='B' -> worker sees '<unset>'
cold pool, submitted while var='B' -> worker sees '<unset>'
cold pool, submitted while var='C' -> worker sees '<unset>'
Threads don’t inherit context state, so every thread sees the default value. On 3.14.6 with -X thread_inherit_context=1:
warm pool, submitted while var='A' -> worker sees '<unset>'
warm pool, submitted while var='B' -> worker sees '<unset>'
cold pool, submitted while var='B' -> worker sees 'B'
cold pool, submitted while var='C' -> worker sees 'B'
This has materialized in the wild as pytest-dev/pytest#14077: on 3.14t, @pytest.mark.filterwarnings fails to suppress warnings raised in dask worker threads if any earlier test warmed dask’s worker pool.
Dask’s fix was to ship its own ContextAwareThreadPoolExecutor.
In my opinion, nobody intentionally designed “capture at the moment a lazily-created worker happens to spawn”. Looking back at old discourse threads, Thomas Grainger realized this would be an issue:
Bear in mind that loop.run_in_executor would still need to be changed with this new behaviour because otherwise the context will be from where the thread in pool is spawned not from where the task is submitted
As far as I can tell, no one tried to patch CPython in time to fix this before 3.14 shipped.
This issue is probably a bigger deal than the context variable issue I raised last week. I, unfortunately, simply failed to notice that this is an issue until it was pointed out me today. I also failed to realize when I first heard about the work Dask did to work around this problem that the problem lies in ThreadPoolExecutor’s behavior.
The 2018 decision, and why its premise no longer holds
This same idea was proposed in June 2018: bpo-34014 / gh-78195 with PR #8035 (and successor PR #9688). @yselivanov rejected it, entirely on executor-type symmetry grounds:
I considered enabling that, but in the end decided not to. The reason is that it’s possible to use a ProcessPoolExecutor with run_in_execuror(), and context cars currently don’t support pickling (and probably never will). We can’t have a single api that sometimes works with contextvars and sometimes doesn’t.
Two things have changed since:
-
The symmetry argument was already abandoned.
asyncio.to_thread()(added in 3.9) does exactly this for threads only, with no process-pool equivalent, and closing bpo-34014 in 2022 pointed to it as the resolution. The stdlib already has “an api that sometimes works with contextvars and sometimes doesn’t”; it’s just attached to asyncio instead of to the executor, where sync users can’t reach it. -
The status quo got worse. Under
thread_inherit_context=1, the alternative is the warmth-dependent nondeterminism above.
This matters for free-threaded ergonomics specifically: the pitch of the free-threaded build is “use threads for parallelism”, and ThreadPoolExecutor is the front door. Right now the front door has the least predictable context semantics of any concurrency primitive in the stdlib.
Proposal sketch
I think this could be done in two ways:
- Opt-in only: a keyword, e.g.
ThreadPoolExecutor(inherit_context=True), defaulting to current behavior. Zero compatibility risk; libraries stop reimplementing it. - Tie the default to
sys.flags.thread_inherit_context: the flag then means one coherent thing instead of today’s “context follows work onto threads, except through the API most people actually use, where you get a spawn-time lottery.” The free-threaded build gets deterministic semantics immediately, and the flag’s eventual default flip (per the other thread) carries executors with it.
My preference is 2 (with the kwarg from 1 for explicit control): on the GIL build with the flag unset, nothing changes; on the free-threaded build, I’d argue the warmth-dependent behavior is an unintended regression of the 3.14 feature that we can simply fix — spawn-time capture for pool workers was never documented or promised anywhere.
Code deliberately using worker-ambient context as worker-local state would need to opt-out. I’d expect this to be rare, it only works dependably today with max_workers=1.
Whatever the outcome here, we need to document the behavior we land on in the ThreadPoolExecutor docs.