Async eagerness as a context variable

Ideas
1

The Problem

The “async coloring” problem is plaguing every language with an async keyword. Python is no exception here.

Psycopg offers sync and async cursors. The end result?
The sync version and async version of cursors are mostly the same code. Except with some async/await sprinkled everywhere.

Django is trying to transition to offering sync and async APIs. But because there’s a requirement to not have sync APIs pay performance costs for not using async, the end result is that Django-side work to move code into the “new hotness” of async has added friction of the inherent cost of async. And so even when Psycopg shows up with async APIs that Django theoretically can use, the async API work for Django hasn’t been done. You can’t do the prep work, because you’re left with an awkward choice: make sync code slower, or have a bunch of code duplication.

This is also also a bit of an ongoing issue in Rust, leading the the keyword generics initiative.

One idea that people have thrown around both here and in the internet at large is a new keyword. maybe_async… async?. Ignoring the awkwardness of both async and maybe_async existing in the language, this has basically been ruled out.

There are ideas of using codegen to create two variants of functions. For many cases one could statically generate variants for sync and async APIs. But now you’re introducing codegen! Either this is an outright manual build step, or some “codegen on load” magic, which is of course doable but seems pretty wasteful.

Really… I just want to turn off the await keyword sometimes. I also don’t need the generator from async. But I would like there to be a very low cost to this (preferably fixed, so that even if Django’s call stack is 10 async functions deep, I still only pay for “turning off async” once)

The Idea

Here is an experiment I have built out:

import contextvars

async def a_f(x):
    return 3

async def a_g():
    x = 1
    y = (await a_f(x)) + 3
    return y

def run_eagerly(coro, *args, **kwargs):
    ctx = contextvars.copy_context()
    ctx.set_async_eager(True)
    return ctx.run(coro, *args, **kwargs)


result = run_eagerly(a_g)
print(f"Result was {result}") # Result was 6
print(f"(If above was 6, we have done it)")

^ the above works with this commit.

The Implementation

What is happening here:

  • The RETURN_GENERATOR bytecode instruction now has an argument, set to 1 if it’s from a coroutine.
  • contextvar.Context now has an ctx_async_eager flag, set by the above function.
  • When ctx_async_eager is set on the context, the following behavioral changes are in place.
    • RETURN_GENERATOR no-ops. So calling an async function doesn’t immediately return a generator. Instead it just moves forward [1]
    • GET_AWAITABLE, instead of getting an awaitable… jumps 7 instructions. This jumps over the generator loop, instead understanding that when ctx_async_eager is set, coroutines directly return values.
      The end result of all this is calling a coroutine in this context will just act as if the await keyword (and async part of the definition for that matter) isn’t there.

This is obviously super weird, and using the Context like this is odd, to say the least. But I think it might minimize the performance impact of an async def’d function, and removes friction to offering async APIs.

The history of async with Javascript has shown one universe: everything is async in JS now. People don’t write sync APIs because people don’t write sync APIs to go along with async ones, and now everything has to be asynchronous. If Python wants to have synchronous APIs be first class, it really feels imperative to introduce a low-friction way for sync APIs to be offered without just outright code duplication.

I came up with this, prototyped it, and honestly think the basic idea (context-based disabling of async/await) could be a major unblocker for big projects to move forward with async APIs.

On a serious note, I am proposing this as food for thought, but I also want to post this to re-insist that with the state of the art complicated libraries are stuck between a rock and a hard place when it comes to actually leaning into async. Any sort of blessed solution (including, dare I say, a 2to3-style one!) feels really necessary if the end goal isn’t just “every single third party library offers async APIs and that’s it.”

8 Likes