A lightweight cooperative timeout mechanism for synchronous code `with timeout(seconds):`

[vstinner (Victor Stinner)]

@zang-langyan

The remaining overhead is the fixed cost of entering _Py_HandlePending itself
This is what shows up as the 20~30% in the list‑comprehension benchmark.

IMO it’s too expensive. I don’t think that we should accept such design.

@zang-langyan

But still “always entering _Py_HandlePending” is not ideal, any suggestion on this?

Well, the alternative is to not implement with timeout(...):.

[Lucas_Malor]

You can use an unsigned integer and add to it, instead of using a signed integer for skip_interval. This is the same “trick” that `regex use. Furthermore, it does the check every 8192 iterations: mrab-regex/src/_regex.c at e57d185bb711729091907b23edac5dcba0426243 · mrabarnett/mrab-regex · GitHub

[zang-langyan (Langyan)]

I agree that an automatic timeout mechanism shares the same limitations as KeyboardInterrupt. IMO, using such a mechanism means accepting the risk that the program may be left in an inconsistent state (e.g., broken class invariants), just like with signal.alarm or similar tools. For users who need stronger guarantees, a python-level timeout check function could be provided to be called explicitly at safe points.

[zang-langyan (Langyan)]

@vstinner

IMO it’s too expensive. I don’t think that we should accept such design.

By inlining Py_CheckTimeOut directly into the check_periodics function, the overhead in the same benchmark drops to 3.1%–5.2% using the same benchmark script as above. Note that this benchmark is nearly a worst case, since every JUMP_BACKWARD is immediately followed by check_periodics. In typical mixed workloads, the impact is even smaller.

@vstinner

Well, the alternative is to not implement with timeout(...):.

I do not think that is a constructive suggestion. I’d much rather focus on concrete technical issues than on dismissing the idea altogether.

[blhsing (Ben Hsing)]

One situationally more efficient approach is to use a scheduler thread to set an eval-breaker bit only after, rather than before, a deadline expires. This avoids keeping the eval breaker hot for the whole duration of an active timeout block. I am calling it the deadline-scheduler approach below, where I guided codex with gpt-5.5 xhigh to implement a prototype that passes all CI tests plus new ones that cover the new code paths. It’s also made safe in a free-threaded build or subinterpreters with its state and lifecycle runtime-managed and locked.

CI passed here:

Make timeout test output checks newline agnostic

Test PR here for easy viewing of diffs:

Cooperative timeout context manager--the deadline-scheduler approach (#45)

Note that I generated the rest of this post below with codex.

The deadline-scheduler prototype keeps the eval loop on its normal fast path until the timeout actually expires:

• with timeout(seconds) pushes a deadline block onto the current PyThreadState.
• A runtime scheduler thread tracks active deadline blocks.
• While the timeout has not expired, no timeout eval-breaker bit is set.
• When a deadline expires, the scheduler sets _PY_TIMEOUT_EXPIRED_BIT on the target thread state.
• The target thread raises TimeoutError at the next normal eval-breaker check.

So pure Python code is still interrupted cooperatively at bytecode/eval-breaker boundaries, but an active, non-expired timeout does not force pending handling on every bytecode/check point.

Benchmark environment:

• Linux 6.14, x86_64
• Intel Xeon Silver 4410Y
• GCC 13.3.0, -O3
• CPython 3.16.0a0
• pyperf
• timeout_seconds=3600.0, so the timeout never expires during the benchmark
• inner_loops=1000

Mean ± std dev:

Benchmark	Baseline	OP approach	Deadline-scheduler approach
pass_loop	22.9 us ± 0.1 us	33.3 us ± 0.2 us, 1.45x slower	22.2 us ± 0.1 us, not meaningfully different
arithmetic_loop	28.5 us ± 0.2 us	37.8 us ± 0.2 us, 1.33x slower	25.5 us ± 0.2 us, not meaningfully different
listcomp_work	35.5 us ± 0.7 us	46.5 us ± 0.4 us, 1.31x slower	35.8 us ± 0.7 us, not significant
empty timeout context enter/exit	342 ns ± 2 ns, nullcontext baseline	1.06 us ± 0.01 us	1.73 us ± 0.05 us

The small “faster than baseline” results for the deadline-scheduler approach are just build/run noise, not a claimed speedup. The same-binary no-timeout control for the deadline-scheduler build measured pass_loop at 22.1 us ± 0.1 us, arithmetic_loop at 25.4 us ± 0.1 us, and listcomp_work at 35.9 us ± 0.7 us, so the active timeout overhead is effectively lost in noise for these workloads.

The tradeoff is visible in the empty enter/exit benchmark: the scheduler-based approach pays more to register and unregister a timeout block. For very tiny timeout scopes, the OP approach is cheaper. For timeout scopes around non-trivial Python work, avoiding repeated pending handling while the timeout is active seems to dominate.

For _sre, I made the engine cooperative by reusing its existing periodic signal-check hook. _sre already has a MAYBE_CHECK_SIGNALS path that runs every 4096 iterations of the matching engine and calls PyErr_CheckSignals(). The prototype adds:

if (_PyTimeout_CheckNow(_PyThreadState_GET())) {
    RETURN_ERROR(SRE_ERROR_INTERRUPTED);
}

at the same point. _PyTimeout_CheckNow() only raises when the current thread has an expired timeout block. If no timeout is active, or if the active timeout has not expired, it returns immediately. If it has expired, it sets TimeoutError, _sre returns through its existing interrupted-error path, and the Python-level regex call propagates the timeout exception.

That means catastrophic backtracking can be interrupted without adding a public re-specific timeout parameter and without adding a separate polling mechanism to the regex engine. It also keeps the polling cadence tied to _sre’s existing signal-check cadence instead of checking on every regex VM transition.

This still has the usual async-exception caveat: a timeout in pure Python is delivered as a normal Python exception at an eval-breaker boundary, so finally blocks and context-manager exits run, but arbitrary Python code can still be interrupted between bytecodes just like with KeyboardInterrupt.

Benchmark script

#!/usr/bin/env python3
"""Compare timeout prototype overhead against a clean baseline."""

from __future__ import annotations

from contextlib import nullcontext
import pyperf


_sink = 0
_timeout_warmed = False


def _consume(value: int) -> None:
    global _sink
    _sink ^= value


def _has_timeout(implementation: str) -> bool:
    return implementation != "baseline"


def _get_timeout(seconds: float):
    if seconds <= 0:
        raise ValueError("timeout must be positive for the overhead benchmark")
    from contextlib import timeout
    return timeout(seconds)


def _warm_timeout(implementation: str, timeout_seconds: float) -> None:
    global _timeout_warmed
    if _has_timeout(implementation) and not _timeout_warmed:
        with _get_timeout(timeout_seconds):
            pass
        _timeout_warmed = True


def _pass_loop(inner_loops: int) -> int:
    total = 0
    for value in range(inner_loops):
        total ^= value
    return total


def _arithmetic_loop(inner_loops: int) -> int:
    total = 0
    for value in range(inner_loops):
        total += value
    return total


def _listcomp_work(inner_loops: int) -> int:
    values = [value * value for value in range(inner_loops)]
    return values[-1] if values else 0


def bench_pass_loop(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total ^= _pass_loop(inner_loops)
    else:
        for _ in range(loops):
            total ^= _pass_loop(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def bench_arithmetic_loop(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total += _arithmetic_loop(inner_loops)
    else:
        for _ in range(loops):
            total += _arithmetic_loop(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def bench_listcomp_work(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total ^= _listcomp_work(inner_loops)
    else:
        for _ in range(loops):
            total ^= _listcomp_work(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def bench_context_enter_exit(
    loops: int,
    implementation: str,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    if _has_timeout(implementation):
        t0 = pyperf.perf_counter()
        for _ in range(loops):
            with _get_timeout(timeout_seconds):
                pass
    else:
        t0 = pyperf.perf_counter()
        for _ in range(loops):
            with nullcontext():
                pass
    return pyperf.perf_counter() - t0


def add_cmdline_args(cmd, args) -> None:
    cmd.extend(("--implementation", args.implementation))
    cmd.extend(("--inner-loops", str(args.inner_loops)))
    cmd.extend(("--timeout-seconds", str(args.timeout_seconds)))


def main() -> None:
    runner = pyperf.Runner(add_cmdline_args=add_cmdline_args)
    runner.argparser.add_argument(
        "--implementation",
        choices=("baseline", "op", "ours"),
        required=True,
        help="'ours' is the deadline-scheduler prototype",
    )
    runner.argparser.add_argument(
        "--inner-loops",
        type=int,
        default=1000,
        help="work per calibrated pyperf loop",
    )
    runner.argparser.add_argument(
        "--timeout-seconds",
        type=float,
        default=3600.0,
        help="non-expiring timeout duration",
    )
    args = runner.parse_args()

    runner.metadata["timeout_implementation"] = args.implementation
    runner.metadata["timeout_inner_loops"] = str(args.inner_loops)
    runner.metadata["timeout_seconds"] = str(args.timeout_seconds)

    runner.bench_time_func(
        "pass_loop",
        bench_pass_loop,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )
    runner.bench_time_func(
        "arithmetic_loop",
        bench_arithmetic_loop,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )
    runner.bench_time_func(
        "listcomp_work",
        bench_listcomp_work,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )
    runner.bench_time_func(
        "context_enter_exit",
        bench_context_enter_exit,
        args.implementation,
        args.timeout_seconds,
    )


if __name__ == "__main__":
    main()

[GalaxySnail]

@blhsing

One situationally more efficient approach is to use a scheduler thread to set an eval-breaker bit only after, rather than before, a deadline expires.

The disadvantage is that it’s unsafe to fork from a multi-threaded process.

I personally prefer to introduce a general machanism for cancelling threads, and let users implement their own timeout mechanism as they want. Of course we need to find out how to make cancellations safe for context managers implemented in Python, which has always been an issue in the main thread.

(BTW, it has always confused me that Python only supports running signal handlers in the main thread, which makes pthread_kill() completely useless.)

[blhsing (Ben Hsing)]

Thanks, that is a fair concern. A scheduler-thread design would be unsafe across
fork if the child inherited the parent’s scheduler thread state, mutex, or
condition variable and then tried to keep using them.

I’ve guided codex to refactor the prototype so the state is reinitialized after fork.

I also agree with your broader point that this can be made a generic cooperative
thread-cancellation mechanism, with timeouts as one possible trigger rather than
the primitive itself. So I’ve guided codex to refactor the prototype in that direction as well.

Below is a summary of the changes produced by codex with minor edits on my part.

For thread safety, the scheduler state
is owned by _PyRuntimeState, protected by a PyMUTEX_T/PyCOND_T pair, and is
rebuilt in the child from _PyRuntimeState_ReInitThreads().

In the child:

• timeout blocks belonging to threads that no longer exist are cleared;
• the surviving current thread keeps its active timeout stack;
• the scheduler struct is zeroed and its mutex/condition variable are freshly
  initialized;
• any still-active timeout blocks for the surviving thread are registered with a
  newly started scheduler thread;
• if there is no active timeout, the scheduler remains uninitialized until the
  next timeout is entered.

So the child process does not continue with the parent’s scheduler thread or
with parent-owned synchronization objects. I added tests for both cases:
forking after the scheduler has been initialized but with no active timeout, and
forking while an active timeout is on the current thread. The branch also passes
the free-threaded build tests I added, and the full GitHub Tests workflow
passed after the refactor:

The current shape is:

• PyThreadState has an atomic cancel_flags field.
• _PyThreadState_RequestCancel(tstate, reason) atomically records a
  cancellation reason and sets a cancellation eval-breaker bit.
• _PyThreadState_RequestCancelByThreadId(interp, id, reason) is the
  cross-thread request path.
• _PyThreadState_CheckCancellation(tstate) is the cooperative delivery point.
• _PY_CANCEL_GENERIC currently raises RuntimeError("thread cancelled").
• _PY_CANCEL_TIMEOUT asks the timeout subsystem whether the current thread has
  an expired timeout block and raises TimeoutError only if it does.
• with timeout(seconds) is now just a deadline trigger that requests
  _PY_CANCEL_TIMEOUT when the deadline expires.

The _timeout.cancel([thread_id]) function in the prototype is only a private
test hook, but it demonstrates the intended factoring: a generic cancellation
request can target the current thread or another thread in the same interpreter,
and timeout is just one producer of such a request.

For thread safety, the cross-thread cancellation flag is atomic, so requesting
cancellation does not require owning the target thread’s GIL. The scheduler’s
deadline list is separately protected by its runtime-owned mutex. The condition
variable is only used to sleep until the next deadline or wake the scheduler
when a timeout is added, removed, fired explicitly, or during shutdown.

This still does not solve the long-standing Python-level async-exception
problem by itself. A cancellation delivered at a bytecode/eval-breaker boundary
has the same basic caveat as KeyboardInterrupt: finally blocks and
__exit__ methods run during unwinding, but arbitrary Python code can be
interrupted between bytecodes. I think the generic cancellation mechanism should
make that limitation explicit and then allow safer higher-level patterns to be
built on top, for example explicit safe-point checks or cancellation scopes that
can defer delivery in protected regions.

I reran the same workload benchmarks on the refactored branch. The current
baseline and deadline-triggered cancellation columns are from this rerun. The OP
column is the previously captured measurement from the eval-breaker-hot
prototype using the same script, machine, compiler family, and build settings.

Test PR for viewing of diffs:

Cooperative timeout context manager--the deadline-scheduler approach (#45)

Passing CI tests:

Fix explicit timeout check scheduler race

Updated benchmarks:

Benchmark	Baseline, current rerun	OP eval-breaker-hot approach	Deadline-triggered cancellation
pass_loop	22.6 us +/- 0.1 us	33.3 us +/- 0.2 us	22.5 us +/- 0.0 us
arithmetic_loop	27.6 us +/- 0.1 us	37.8 us +/- 0.2 us	27.5 us +/- 0.1 us
listcomp_work	34.5 us +/- 0.6 us	46.5 us +/- 0.4 us	34.8 us +/- 0.7 us

pyperf compare_to reported the three workload differences between the current
baseline and the deadline-triggered cancellation run as not significant. That is
the important part of the result: active but non-expired timeout scopes no
longer keep paying repeated pending-handler overhead in the eval loop.

Benchmark script

#!/usr/bin/env python3
"""Compare timeout prototype overhead against a clean baseline."""

from __future__ import annotations

import pyperf


_sink = 0
_timeout_warmed = False


def _consume(value: int) -> None:
    global _sink
    _sink ^= value


def _has_timeout(implementation: str) -> bool:
    return implementation != "baseline"


def _get_timeout(seconds: float):
    if seconds <= 0:
        raise ValueError("timeout must be positive for the overhead benchmark")
    from contextlib import timeout
    return timeout(seconds)


def _warm_timeout(implementation: str, timeout_seconds: float) -> None:
    global _timeout_warmed
    if _has_timeout(implementation) and not _timeout_warmed:
        with _get_timeout(timeout_seconds):
            pass
        _timeout_warmed = True


def _pass_loop(inner_loops: int) -> int:
    total = 0
    for value in range(inner_loops):
        total ^= value
    return total


def _arithmetic_loop(inner_loops: int) -> int:
    total = 0
    for value in range(inner_loops):
        total += value
    return total


def _listcomp_work(inner_loops: int) -> int:
    values = [value * value for value in range(inner_loops)]
    return values[-1] if values else 0


def bench_pass_loop(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total ^= _pass_loop(inner_loops)
    else:
        for _ in range(loops):
            total ^= _pass_loop(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def bench_arithmetic_loop(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total += _arithmetic_loop(inner_loops)
    else:
        for _ in range(loops):
            total += _arithmetic_loop(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def bench_listcomp_work(
    loops: int,
    implementation: str,
    inner_loops: int,
    timeout_seconds: float,
) -> float:
    _warm_timeout(implementation, timeout_seconds)
    total = 0
    t0 = pyperf.perf_counter()
    if _has_timeout(implementation):
        with _get_timeout(timeout_seconds):
            for _ in range(loops):
                total ^= _listcomp_work(inner_loops)
    else:
        for _ in range(loops):
            total ^= _listcomp_work(inner_loops)
    dt = pyperf.perf_counter() - t0
    _consume(total)
    return dt


def add_cmdline_args(cmd, args) -> None:
    cmd.extend(("--implementation", args.implementation))
    cmd.extend(("--inner-loops", str(args.inner_loops)))
    cmd.extend(("--timeout-seconds", str(args.timeout_seconds)))


def main() -> None:
    runner = pyperf.Runner(add_cmdline_args=add_cmdline_args)
    runner.argparser.add_argument(
        "--implementation",
        choices=("baseline", "op", "ours"),
        required=True,
        help="implementation under test",
    )
    runner.argparser.add_argument(
        "--inner-loops",
        type=int,
        default=1000,
        help="work per calibrated pyperf loop",
    )
    runner.argparser.add_argument(
        "--timeout-seconds",
        type=float,
        default=3600.0,
        help="non-expiring timeout duration",
    )
    args = runner.parse_args()

    runner.metadata["timeout_implementation"] = args.implementation
    runner.metadata["timeout_inner_loops"] = str(args.inner_loops)
    runner.metadata["timeout_seconds"] = str(args.timeout_seconds)

    runner.bench_time_func(
        "pass_loop",
        bench_pass_loop,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )
    runner.bench_time_func(
        "arithmetic_loop",
        bench_arithmetic_loop,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )
    runner.bench_time_func(
        "listcomp_work",
        bench_listcomp_work,
        args.implementation,
        args.inner_loops,
        args.timeout_seconds,
    )


if __name__ == "__main__":
    main()

[jsbueno (Joao S. O. Bueno)]

Since it would be an optional thing, if code can run with zero-overhead when not in a “timeoutabe-section” it would be a nice to have, even at a steep cost.

[dpdani (Daniele Parmeggiani)]

Hello

I think this is a desirable feature to have for the reasons you have already mentioned @zang-langyan. As I expressed in other threads, the big problem isn’t really implementing something like what you suggest, but rather doing it in a way that makes it usable, allowing code inside that with to run cleanups when the deadline is reached. This is essentially impossible with the change you proposed, and also with the way Python currently works. I think a language change is an important pre-condition.

Consider this code:

with timeout(...):
    with my_lock:
        ...  # a possibly long-running operation

Granted you should hold locks for as little time as possible, such that the example can be a little silly, what guarantees can you make here? You cannot guarantee that the data protected by the lock will always be in a consistent state: you may fire an interrupt in the middle of the critical section. Furthermore, if you have your own custom lock subclass, written in Python, you cannot even guarantee the lock will be released when the timeout fires.

I am working on this issue, albeit in the general sense of generic interruptions, be them timeouts, KeyboardInterrupt, or anything else, and I hope to have a new DPO thread to discuss a pre-PEP relatively soon.

[mikeshardmind (Michael H)]

This feels like the wrong way to appraoch the problem.

If you’ve got code that you don’t want running for a long time, design it so it has a determinstic maximum runtume or have specifically the code in question check for timeout, don’t invasively change the entire interpreter behavior and make everything have to possibly handle a synthetic timeout exception thrown into the code from outside of it.