TL;DR -
async def f() -> AsyncGeneratoris incoherent compared to other generator/function type annotations in Python.
After getting back to Python after almost fifteen years (last time it was Python 2!), I have quite happily adopted its type hints. I find the system, despite being a bolt-on feature, quite coherent and integrates well with Python’s dynamic nature and strong reflective capabilities.
Even for async functions, I believe Python’s handling of the return type is better than that of JS/TS by making async a modifier:
async def f() -> int: ...
await f()
# alternatively
def f() -> Awaitable[int]: ...
await f()
vs.
async function f(): Promise<number>;
await f();
// alternatively
function f(): Promise<number>;
await f();
In JS/TS, while the async keyword gives the function’s body extra capability (to use await inside), it does not impact how the function should be called.
However, I noticed some discrepancy with regard to how an async generator function’s signature should be interpreted.
Before diving into the details, I would like to recap what role a function’s signature plays (for me). Echoing Steve Klabnik’s view, the signature:
- For its body or implementation, dictates what parameters are available for use, and what type of value it should return.
- For its caller, dictates how the function should be used.
If we consider async to be part of a Python function’s signature, then Python meets the above criteria pretty well:
| Signature | Use case |
|---|---|
async def f() -> int |
await f() as an int |
def f() -> Awaitable[int] |
await f() as an int |
def g() -> Generator[int, None, None] |
for x in g() where x’s are int |
async def j() -> AsyncGenerator[int, None] |
? |
But what should we put at the question mark? There are actually two possible answers, depending on the body of j!
See below:
async def j() -> AsyncGenerator[int, None]:
yield 0
async for x in j():
assert x == 0
async def k() -> AsyncGenerator[int, None]:
return j()
async for x in (await k()):
assert x == 0
Here j and k seemingly have the same signature, while their use cases are quite different. An unequivocal and equivalent signature of j is actually
def j() -> AsyncGenerator[int, None]
and that of k is
def k() -> Awaitable[AsyncGenerator[int, None]]
Note that the interpretation of k aligns better with what we’ve seen with async and Awaitable in f.
In reality, however, we’ve rarely seen anyone define a function like k. The rather incoherent case of j is prevalent—incoherent because the async keyword does not modify the actual signature here, and what actually plays the crucial role is the yield inside its body.
Why?
There are several factors in play here:
- While
def j() -> AsyncGenerator[int, None]more precisely captures the use case forj, without theasynckeyword we will not be able to useawaitinside its body. - For a normal function like
f, puttingasyncon the LHS ‘cancels out’ theAwaitableon the RHS, but in the case ofj, there is nothing to cancel out. - For a normal generator function, its signature is also informed by the use of
yieldinside the body.
Let’s look at factor #3 here more closely. What it means is that
def g() -> Generator[int, None, None]:
yield 1 # This makes `g` a generator
def h() -> Generator[int, None, None]:
return g() # This make `h` a function
But a caveat for normal generator functions is that the semantic difference in their signatures does not imply a practical one:
for x in g():
assert x == 1
for x in h():
assert x == 1
Which is, unfortunately, not true for async generator functions.
On the other hand, JS/TS does better than Python on this issue, with their * marker for generators:
async function* j(): AsyncGenerator {
yield 1;
}
async function k(): Promise<AsyncGenerator> {
return j();
}
Admittedly, from PEP 362’s perspective, a function’s signature has nothing to do with the async keyword:
from inspect import signature
def a() -> int: ...
async def b() -> int: ...
assert signature(a) == signature(b)
But I believe this is because PEP 362 predates the introduction of async/await.