The most valuable tool for debugging Python code is the exception. Usually, if we do nothing at all, we get exceptions logged to the console, and we have to do some work to suppress them. That’s true of synchronous code, threaded code, subprocesses, but not asyncio tasks. Example code:
import threading, multiprocessing, asyncio, time
# This task takes some time, but then fails due to a bug.
# GOAL: Be able to spin this off, but still see its exception
# traceback.
def delayed_boom():
time.sleep(1)
None.sense
async def adelayed_boom():
await asyncio.sleep(1)
None.sense
# Synchronous code is easy. You call it, it crashes.
def main_sync():
print("Starting sync!")
delayed_boom()
print("Ending.") # Won't happen
# Threaded code is also easy. You spawn a thread, it crashes.
def main_thread():
print("Starting thread!")
threading.Thread(target=delayed_boom).start()
# input("Press Enter to end -> ")
time.sleep(3)
print("Ending.")
# Multiprocessing is easy, though you have more things to take
# care of, such as ensuring that stuff is importable.
def main_process():
print("Starting subprocess!")
multiprocessing.Process(target=delayed_boom).start()
# input("Press Enter to end -> ")
time.sleep(3)
print("Ending.")
async def main_async_bad():
print("Starting async!")
task = asyncio.create_task(adelayed_boom())
await asyncio.sleep(3) # it's too fiddly to wait for input
print("Ending.")
# The above code just never awaits the task, very bad. If we await it
# here, we at least see the exception *eventually*, but we won't see
# it in a timely manner.
# await task
# ------ this is a lot of boilerplate for a Python script ------
import traceback
def handle_errors(task):
try:
exc = task.exception()
if exc: traceback.print_exception(exc)
except asyncio.exceptions.CancelledError:
pass
all_tasks = []
def task_done(task):
all_tasks.remove(task)
handle_errors(task)
def spawn(awaitable):
task = asyncio.create_task(awaitable)
all_tasks.append(task)
task.add_done_callback(task_done)
return task
# ------ end boilerplate ------
async def main_async_good():
print("Starting async!")
spawn(adelayed_boom())
await asyncio.sleep(3)
print("Ending.")
if __name__ == "__main__": # guard against multiprocessing
# main_sync()
main_thread()
main_process()
# asyncio.run(main_async_bad())
asyncio.run(main_async_good())
In each case (other than main_sync()), it should spawn a task, but main runs for longer. Think like a long-running GUI app or server, and it triggers some kind of asynchronous operation. What happens when that task fails? In most cases, the exception shows up just fine. But for asyncio, that requires a blop of extra handling, for two reasons: firstly, as noted in the docs for create_task, tasks have to be retained; and without some kind of “done” trigger that checks for the exception, that still gets abandoned.
(I considered using a TaskGroup but that doesn’t help - if one of the tasks fails, it brings down the entire app.)
I propose that something akin to spawn() be added as a top-level function in asyncio, and that it become the recommended way to start tasks without waiting for them. This will bring async I/O up to parity with threads and processes in terms of “debuggable by default”. My suspicion is that issues like these may very well have been caused in part by exceptions not being seen.
function composition