I’ve just released the version 0.7 of my aiomonitor fork named aiomonitor-ng to remedy the issues I’ve reported such as tracking the task creation and cancellation chain, without modifying the stdlib asyncio but extending it using the task factory API.
It also comes with a much more user-friendly terminal interface based on Click and prompt_toolkit, which supports auto-completion of commands, options such as keyword filters, and arguments including task IDs.
The termination/cancellation tracker also has a special support for long-running tasks that should not be cancelled under normal operation: guarding them with aiomonitor.task.preserve_termination_log dectorator will keep their termination logs permanently, while the termination logs of volatile tasks are stripped by the maximum history limit to prevent a memory leak.
There are two new commands:
-
ps-terminated: List the terminated tasks with their stack traces, showing how they are cancelled or raised exceptions. -
where-terminated: Show the detailed termination/cancellation chain of a terminated task.
You can search tasks by their names or shortened coroutine repr strings using -f / --filter option of ps and ps-terminated commands.
You can also show only the tasks wrapped with preserve_termination_log (aka “persistent”) using -p or --persistent option of ps and ps-terminated commands.
Using aiomonitor-ng, my team could discover and fix several hidden bugs that happen sporadically in production setups only.
I hope this would help debugging complex asyncio applications for your cases as well!