`yield from` in async functions

Hi there

I’ve searched a bit I and don’t really understand why yield from isn’t supported in async functions. All arguments I’ve seen seem to be about using it for yielding from other async generators, which seems hard to do.

This to me feels odd. I would expect yield from to work on normal non async generators and iterables without any issues. And once someone figures out how to allow “piping” an async generator, to use async yield from ... just like we have async for ... in ...

3 Likes

This is discussed here in the PEP that introduced async generators: PEP 525 – Asynchronous Generators | peps.python.org. In short, it is difficult to implement (yield from isn’t just sugar for yielding several times), and because yield from was historically often used to implement a form of coroutines, it was judged to be less useful in async functions.

I’ve personally also missed yield from in async generators, though. If the implementation turns out to be feasible, I’d support adding it.

4 Likes

The way it’s worded, it feels to me that they only considered yielding from an async generator, and not regular generators. Or am I misreading it?

Sorry, I missed that in your original post. Piping from a normal generator would probably still be complicated (async generators, generators, and coroutines all are built on some fairly intricate machinery), and it doesn’t seem as useful: if I write a generator that yields data from a synchronous iterator, I don’t have much reason to make the generator async.

Well, I always prefer yield from EXPR instead of for i in EXPR: yield i. I know they aren’t equivalent, and yield from can do much more.

Currently I have a thing where I’m currently alternating a blocking call with some calculations where I use yield from ... as a shorthand for for i in ...: yield i. I wanted to make the blocking call an async one but that ended up giving a syntax error once I made the generator async. Which then led me down a rabbithole that ended with my question above :stuck_out_tongue:

As I’m typing this I do realize that the caller of this function itself is also using yield from in the same way to append a few extra things at the end of the iterator. It seems I wouldn’t just need yield from but also an async yield from.

Looking through the PEP’s, I don’t see why a straight copy of yield from formal semantics wouldn’t be correct (for non async generators).

Isn’t there a problem where, if you “yield from” within an async function, you lose the ability to communicate with coroutines of said function, since those are generators under the hood?

Upvote this. This will also allow async generators to return a value. I’m unable to find the discussion for PEP 525, but it seems the main reason it was not implemented was time limitation (reddit discussion). Is there anything that makes an async yield from infeasible?

The upvote isn’t necessary - it’s an implantation that is needed. See Jelle’s previous comment:

Are you open to implementing this yourself?

No, I’m not interested enough to work on this myself.

If you want this, you’ll probably need to start with writing a PEP.

I see value as well on having “yield from” in async generators, both for consistency and also because is a syntax that sometimes helps to write more readable and reusable generators.

I would be willing to help with a PEP this vision is shared, though given my nonexistent knowledge of CPython I’m not sure if my help would be very useful.

Just to be clear, I was asking @Linchin

Hey I’ve implemented a patch for asynchronojus generators to support returns would be willing to implement more and the discussion is here

Would love if all of y’all joined in on the discussion here :slight_smile: