I’ve been thinking about a problem that many of us might have run into in practice: a piece of code runs for too long, and you don’t know whether it is the os problem or the code logic problem or it is just stuck. Hence, I’d like to propose a built-in cooperative timeout for synchronous code. The idea is a with timeout(seconds): context manager that works on any thread, introduces negligible overhead when not used, and provides a clear pattern for C extensions to become interruptible.
Current gap
The existing tools are either restricted to the main thread (signal), only for async code (asyncio.timeout), or don’t actually stop the work (thread pool futures).
More fundamentally, there is no standard cancellation check that C extensions can call to cooperate with Python-level timeouts.
What I have so far?
I have a working prototype(maybe a bit ugly) which leverages CPython’s existing “eval breaker” mechanism:
A new per‑thread field timeout_block (a linked list for nested timeouts) is added to PyThreadState.
with timeout(seconds): pushes a deadline onto this list and sets a dedicated _PY_EVAL_TIMEOUT_BIT in the thread’s eval_breaker.
The interpreter loop already calls _Py_HandlePending at safe points (between bytecodes, back‑edges, function calls). My patch adds a tiny check there: if the timeout bit is set, it compares the current time against the innermost deadline and, if expired, raises TimeoutError.
When the with block exits, it pops the deadline. If the list becomes empty, the timeout bit is cleared and the thread goes back to zero‑overhead execution.
Key properties:
Zero overhead when not in use: the eval_breaker bit stays 0, so the interpreter skips the time check entirely.
No background thread: everything is driven synchronously by the bytecode loop, making it simple and efficient.
Works with any thread: the state is thread‑local; no GIL assumptions required.
Fully compatible with the free‑threaded build: each thread checks its own timeout_block, no cross‑thread coordination is needed.
Pure Python loops are automatically interrupted at the bytecode boundary. For C extensions that perform long computations (e.g. regular expressions, compression, JSON parsing, etc.) to become timeout‑aware, they need to cooperatively callPy_CheckTimeOut(tstate) at regular intervals.
Similar to
C++20: stop_token.
Go: context.WithTimeout + ctx.Done() checks
C#: CancellationToken
I’d love to get some feedbacks on this before I open an Issue/PR. Is this a feature you’d support? Are there design concerns or use cases I’m missing?
On my Fedora 44 laptop, calling PyTime_PerfCounterRaw() takes 22.0 ns +- 1.1 ns, it’s not cheap. For example, calling len("abc") takes 19.5 ns +- 0.9 ns.
Calling frequently PyTime_PerfCounterRaw() can introduce a significant slowdown.
But gettimeofday() is very fast as it only has to read out of shared memory on Linux and I think something similar on Windows.
e.g. no syscall over head.
If the timeout is based on the time of day calls then you get 1ms accuracy I think.
I agree that calling PyTime_PerfCounterRaw() frequently would be too expensive, given the numbers you showed.
However, When no timeout is active, no time check is performed. As for in the timeout block, we can reduce the frequency to check the time (e.g., every N bytecodes or every function call, not every single instruction). C extensions like _sre can check every K steps or every M bytes of input, instead of every state transition.
So the bottom line is: no overhead without a timeout, and tunable overhead when a timeout is active.
If the performance impact can be kept negligible, do you think this would be a worthwhile feature to have?
_PY_EVAL_TIMEOUT_BIT seems to simply indicate whether timeout_block has a non-zero length. Is the expense of checking the length of the list enough to justify caching it in the timeout bit, or is there another reason for the bit?
Small poi: $ man 3p gettimeofday regards it as obsolete
APPLICATION USAGE
Applications should use the clock_gettime() function instead
of the obsolescent gettimeofday() function.
though it may be one of those things like signal(3p) vs sigaction(3p) where the old version never truly dies Python only exposes signal as signal.signal and not sigaction at a user-facing level.
if (_Py_atomic_load_uintptr_relaxed(&tstate->eval_breaker) & _PY_EVAL_EVENTS_MASK) {
return _Py_HandlePending(tstate);
}
In the bytecode dispatch loop, we already test a single word tstate->eval_breaker(atomic) against a set of bits for pending calls, async exceptions, etc. Adding _PY_EVAL_TIMEOUT_BIT means the timeout check is just another bit in the same test—effectively zero overhead on top of what’s already there. Not setting that means not only lose atomic test but also an additional condition after the above if statement(though it may be negligible to test if it is a nullptr or not).
I’ve collected some new data that might address the earlier concerns about per-bytecode overhead.
Raw C loop — calling Py_CheckTimeOut directly in a tight loop:
– Slow path (clock read every call): 10.2ns ± 1.2ns per call
– Fast path (skip interval active): 1.1~1.3ns per call
This shows that the fast path (just a counter decrement and branch) is essentially free, while the more expensive clock read is amortized to negligible levels.
Let me know if you have some other ideas or concerns.
I don’t understand well what you are measuring. Can you provide the benchmark script?
From what I saw in your PR, you’re setting _PY_EVAL_EVENTS_BITS during a with timeout(...): block. It means that ceval has to stop at every single instruction to call _Py_HandlePending(), rather than only stopping when there is a pending event. It sounds inefficient.
Thanks for taking a closer look. I should have included the benchmark script upfront. Here is the script,
import timeit
import _timeout
from contextlib import timeout
_timeout._test_timeout_init(3600)
number = 100_000_000
for si in range(0, 513, 2):
t = timeit.timeit(f"_timeout._test_benchmark_timeout_check_raw_loop({si}, {number})", globals=dict(_timeout=_timeout), number=1)
print(f"Py_CheckTimeOut raw loop (wt interval {si}): {t / number * 1e9:.1f} ns")
# when si = 0: it gives around 10ns
# as si increases it diminish down to around 1~2ns
_timeout._test_timeout_cleanup()
number = 100_000_000
code = "x = [i * i for i in range(1000)]"
t1 = timeit.timeit(code, number=number, globals=globals())
t2 = timeit.timeit(f"with timeout(3600):\n {code}", number=number, globals=globals())
print(f"overhead: {((t2/t1)-1)*100:.2f}%") # this gives around 20~30% as of current implementation
You’re right that my current PR sets a dedicated bit (_PY_EVAL_TIMEOUT_BIT) inside _PY_EVAL_EVENTS_MASK, which forces check_periodics to call _Py_HandlePending on every bytecode when a with timeout block is active.
But the expensive part (clock read) is only done once every N invocations. Inside _Py_HandlePending, the actual check Py_CheckTimeOut uses a per‑block skip counter. For most of the calls it only does an increment and returns immediately with no clock/syscall.
The remaining overhead is the fixed cost of entering _Py_HandlePending itself
This is what shows up as the 20~30% in the list‑comprehension benchmark.
But still “always entering _Py_HandlePending” is not ideal, any suggestion on this?
Since your post is actually exactly the same that was proposed in another thread some days ago, I repeat my concerns:
You emphasize that the mechanism would be “cooperative”. But the details of the proposal reveal that it is cooperative only at the C level (like signal handling; so C extensions can be hardened), but it is not cooperative at the Python level (so Python code cannot be not be hardened). So, the following would also be true if a timeout has fired:
Python only exposes