Preventing `yield` inside certain context managers

As one of the asyncio folks who is moving forward with task groups (and timeouts) I agree this is somewhat urgent. We could document “don’t yield out of a task group or timeout” but without enforcement it would still be difficult in practice if we only started enforcing that in a later version.

I’ve looked over your proposal again (but not read the intervening discussion) and it looks sensible to me. Maybe you should just open a bpo issue and a PR to get this into 3.11? (IMO not every new sys function requires a PEP.)

1 Like

I’ll leave it to the current SC to decide whether this is PEP-worthy or not, but here’s another question: is this something a static analyzer could find and warn about?

AFAICS, this almost is a PEP already – use boring PEP terminology (“What’s this?” → “Abstract” and so on), use boring asyncio terminology (“task groups” rather than “nurseries”), assign a number, and it’s a PEP :‍)

I wouldn’t really mind it going in without a PEP, but there are some reasons for PEPifying:

  • The rationale is worth preserving even after/if we move off Discourse, and it’s too detailed for the documentation. A PEP is a good place to put it.
  • Request comments in the usual place people are looking for Python RFCs.

I haven’t seen any opposition (but that’s to be expected for a power plant part proposal). If asyncio experts are OK with it, I don’t see it getting blocked.

2 Likes

I might be out in left-field but it seems better to more cleanly separate generators for lazy sequences and generators for asyncio. Even though they are built on the same low-level mechanisms, they could be more cleanly separated at a high level. E.g. don’t use ‘yield’ or ‘yield from’ if doing asyncio. Does it make sense to opt-in to that new behavior using a __future__ import? It seems like a way to do it without backwards compatibility issues.

Perhaps at the same time we could implement the “eager” optimization for co-routines. I.e. they could execute up to the first suspension rather than immediately suspending. Cinder does this and they see a non-trivial performance improvement from it. To me, it seems like we might be able to kill two birds with one stone.

1 Like

Personally, I’ve never understood why or how you’d use async generators and async generator expressions, so for me, a restriction like “you can’t use yield [from] in an async function” doesn’t impose a burden. But IIUC @yselivanov (who introduced them) has use cases, and from the initial post it seems @njs has use cases too, so let’s not hurry.

We should have a separate discussion about that Cinder improvement, I agree it’s worth looking into but I worry about backwards compatibility, and I’m not sure it’s worth a new __future__ statement.

1 Like