Async generator missing unawaited coroutine warning

Async-SIG
1

demo program:

def test_async_fn():
    async def async_fn():
        pass

    async_fn()


def test_async_gen_fn():
    async def agen_fn():
        yield

    agen_fn().aclose()
    agen_fn().asend(None)

test_async_fn()
test_async_gen_fn()

output:

/home/graingert/projects/anyio/foo.py:5: RuntimeWarning: coroutine 'test_async_fn.<locals>.async_fn' was never awaited
  async_fn()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

expected:

/home/graingert/projects/anyio/foo.py:5: RuntimeWarning: coroutine 'test_async_fn.<locals>.async_fn' was never awaited
  async_fn()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
/home/graingert/projects/anyio/foo.py:12: RuntimeWarning: coroutine '<async_generator_athrow object at 0xffffffffffff>' was never awaited
  agen_fn().aclose()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
/home/graingert/projects/anyio/foo.py:13: RuntimeWarning: coroutine '<async_generator_asend object at 0xffffffffffff>' was never awaited
  agen_fn().asend(None)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
2

Originally posted as a bug async generator missing unawaited coroutine warning · Issue #89091 · python/cpython · GitHub

3

interestingly cython functions do raise a runtime warning here:

Python 3.11.0b4 (main, Jul 11 2022, 23:00:23) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import helloworld
>>> helloworld.hello_agen
<cyfunction hello_agen at 0x7feef58b4860>
>>> helloworld.hello_agen().asend
<built-in method asend of async_generator object at 0x7feef58cefc0>
sys:1: RuntimeWarning: coroutine 'hello_agen' was never awaited
>>> import inspect
>>> inspect.iscoroutinefunction(helloworld.hello_agen().asend)
/usr/lib/python3.11/inspect.py:414: RuntimeWarning: coroutine 'hello_agen' was never awaited
  return _has_code_flag(obj, CO_COROUTINE)
<stdin>:1: RuntimeWarning: coroutine 'hello_agen' was never awaited
False
>>> inspect.isasyncgenfunction(helloworld.hello_agen().asend)
/usr/lib/python3.11/inspect.py:422: RuntimeWarning: coroutine 'hello_agen' was never awaited
  return _has_code_flag(obj, CO_ASYNC_GENERATOR)
False
>>> inspect.isgeneratorfunction(helloworld.hello_agen().asend)
/usr/lib/python3.11/inspect.py:407: RuntimeWarning: coroutine 'hello_agen' was never awaited
  return _has_code_flag(obj, CO_GENERATOR)
False
4

Pretty sure these are triggered by __del__, which likely means a reference is being held around in the Python implementation that Cython is releasing more eagerly. It might even be leaking? But more likely it’s getting cleaned up with the rest of the runtime, but too late to display warnings.

5

the same behaviour happens even if the garbage collector is disabled and using a sys.unraisablehook set

generator functions finalization is handled in genobject.c

and async gen functions fail the check because they set CO_ASYNC_GENERATOR instead of CO_COROUTINE

6

Ah, well that should be an easy fix then, right?

Should the generator also set CO_COROUTINE? Seems like the check should just be updated to check for CO_ASYNC_GENERATOR as well

7

That’s not quite right either, because that’s when the async generator gets finalized and not when the async generator asend gets finalized, and so needs a different warning message - missing an async with aclosing

8

Ah I see. The automatically generated methods on the async generator don’t have the message. I bet they’re native methods and not “real” coroutines, so they likely never get registered or tracked.

I can see the value in having these warnings (and separate from the generator itself), but I have no idea the best way to implement them. Maybe additional state on the generator for each method so that the warnings can be shown when the generator is collected?

9

I think adding a finalizer to check o->ags_state == AWAITABLE_STATE_INIT or o->agt_state should do the trick?