I’ve been building an agent-style system that needs to call 3-4 different LLM/tool APIs concurrently as part of a single task step, and I’m trying to figure out the cleanest pattern for it in modern Python (3.11+).
Right now I’m using asyncio.gather() with return_exceptions=True so one failing call doesn’t kill the whole batch, then checking each result afterward for exceptions. But I’ve seen people recommend asyncio.TaskGroup (3.11+) instead for cleaner exception propagation and cancellation of sibling tasks when one fails.
For a case where:
I want partial results even if one call fails (not fail-fast)
I need to enforce a per-call timeout without cancelling the whole group
Tasks are I/O-bound (HTTP calls), not CPU-bound
Is TaskGroup actually better here, or does its fail-fast-by-default behavior make gather(return_exceptions=True) the more practical choice unless I’m okay with all-or-nothing cancellation? Curious how others handle per-task timeouts cleanly in either approach -asyncio.wait_for() wrapping each task feels a bit verbose when you have 4+ calls.
I believe gather is superior for your case. If you are put off by the verbosity of asyncio.wait_for on each argument, you can always map its partial under your desired timeout over the awaitables.
I’d use asyncio.TaskGroup in this situation. I find code that uses structured concurrency much easier to think about and review, especially when the things task groups replace are easy to misuse. Using structured concurrency means avoiding unstructured concurrency where possible, and that means not using asyncio.create_task and similarly asyncio.gather. At most, I consider asyncio.create_task a low-level primitive, but for asyncio.gather I always prefer the structured form.
In your use case, I’d handle per-call timeout (e.g., with asyncio.timeout or a library-specific parameter) and API errors within each task itself. That lets other tasks finish and gives you partial results.
If you’re attached to asyncio.gather as an abstraction, you might be interested in Quattro, which reimplements it using task groups for safer usage.