Using async functions outside of asyncio

Async-SIG
1

I would like to clarify whether using async functions outside of asyncio is legitimate.

Context

I like using async functions and async generators for data processing, as syntactic sugar to create traditional, not-really-async coroutines that can be suspended and resumed. [1] This can be done with generators, of course, but the quality-of-life improvement using async pretty much mirrors what is described in PEP 492.

Unfortunately, for most of my applications, using asyncio is not an option, due to other parallel processing or multiprocessing constraints. [2]

Therefore, I run coroutines “by hand”, using coro.send(None) and so forth. Over time, I have made a little library of tools like run(), gather(), etc. to do just that. I have now packaged some of these tools up into a small package which I would like to use in one of my main production codes.

Since the inner workings of coroutines do not seem widely advertised, I am wondering whether the fact that coroutines can be resumed using coro.send(None) and suspended with a bare yield is an official part of the language, or merely an implementation detail which e.g. PyPy just happens to follow as well.

  1. For example, a very useful tool to efficiently process large astronomical data sets is a pub/sub model where the subscribers get their data from an async for loop. ↩︎

  2. I am mainly talking about library code here, where I don’t know what users are otherwise doing to interfere with asyncio. ↩︎

1 Like
2

There’s precedent in libraries like Trio; I figure if CPython radically changed how coroutines work it’d cause a lot of breakage downstream so it’s not likely to happen.

2 Likes
3

If you are going to look at Trio to compare your own code against, you may also find @njs bog posts about aysncio and Trio helpful. See for instance: Beautiful tracebacks in Trio v0.7.0 — njs blog (and other posts in the archive there). I learned a lot from those, when I first read them (a few years ago).

2 Likes
4

Language definiton is 3. Data model — Python 3.12.1 documentation

2 Likes
5

Thanks, while this is the page that originally pointed me to PEP 492, at the time I didn’t understand that multiple coroutine wrappers from coro.__await__() can subsequently advance the coroutine independently – that was what originally drove me towards iterating manually using coro.send(None).

I am still wondering if that definition also implies that the implementation of asyncio.sleep(0), i.e.

@types.coroutine
def sleep():
    yield

is the natural way to suspend a chain of coroutines.