Unified Concurrency Interface

Ideas
,
1

From my experience working on asyncio code and sync/non-asyncio code, probablythe most annoying part is actually building interfaces that support both. You get a bunch of aget or async_get verus get calls. Async adapters and everything else. Tracking what can be used inside an async context and what cannot be.

Since I have been in Django world again recent, Django recently implemented the task framework, which is just a unified interface for defining background task in Django. That really got me to thinking, how do we unify the API for asyncio/sync code? Which led to what about concurrency in general?

What if async/coroutines “just worked” in Python? Without you needing to set up an event loop or run asyncio.run?

Very similar to how the task framework for Django works, what we instead had similar concurrency backend/configuration in Python that could be configured?

So for asyncio:

# yes I know, I should be using `asyncio.run` or a runner, just an interface example

import asyncio

loop = asyncio.get_event_loop()
loop.run_until_complete(coroutine)

Might become something like:

import concurrency

concurrency.set_backend(concurrency.Asyncio)
await coroutine

But why?

This would allow automatic handling of async/coroutines. Which would unify the concurrency interface. No longer needing to make async adapter or everything else to support both sync and async. The default concurrency backend might be “immediate”, which is just “execute coroutine as a sync function”. So, this mean if you want to support asyncio, you just implement async def functions and it should “just work”.

But, it would also allow other concurrency abstracts.

So instead of:

import threading

thread = threading.Thread(function)
thread.start()
thread.join()

You could have:

import concurrency

concurrency.set_backend(concurrency.Threading)
await coroutine # automatically creates background thread and runs coroutine in thread

So, this could have the advantage of unifying the interface between asyncio/non-asyncio code and threading code. With the popularity of free-threading and how that change is really going to impact the ecosystem once it is rolled out, shifting the devex/interface could make support all of the concurrency methods a lot easier.

2

Unless you start adding locks in many places in your application, I don’t see how you can hide the differences between free threading and asyncio. One of the advantages of asyncio is that you know exactly where all of the yield points are, so you can avoid many locks that you’d otherwise need.

I think the models are just too different to be unified.

1 Like
3

Or, looking at this positively: They offer different benefits and can therefore be used very differently in order to take advantage of both.

1 Like
4

It would not replace any of the existing interfaces. So if you need something is only threading, it can still use that interface directly.

This is purely designed to be interface/devex change. It is so you can have a single outward facing interface when you need multiple interfaces. It does not even need to apply to the free-threading stuff, especially not initially. The idea is just to solve the existing burden of juggle what context you current are in and all of the interface bloat that creates.

Example:

Developer interface:

import rest_client

response = await rest_client.get_resource()

But the internal implementation might look very different:

async def get_resource():
    if backend = "asyncio":
        return await _get_resource_async()
    # if you are in a thread, you might need to lock a resource
    elif backend == "threaded":
        lock.accquire()
    # but without the thread, there is no need to lock no else case
    # but threading / no thread both use the same implementation under the hood
    return _get_resource()

Another “real world” example: Home Assistant is a project that often needs to handle both asyncio/non-blocking IO and blocking IO/sync. Supporting both interfaces are necessary because not all third party intergrations and APIs are async native. The way they handle it is that require all asyncio “safe” functions to either start with async_ or use a decorator that tells the event loop that it is. That also duplicates all of the interfaces as well. If you want to get the current state of a sensor, you have to use async_get inside of an event loop and get inside of a thread. So there is a ton of context switching that just adds a ton of complexity to the code and to the cognitive load of the developer.

I completely understand that cooperative concurrency/asyncio is very different than threading, or even no concurrency. They all have their own use cases and places you would use them. This is idea is purely for the cases where you need more than one of them. Or you need to support more than one of them (in the Web world, WSGI + ASGI).

It allows shifting more of the burden of “what interface to use where” more to the implementations of the interface instead of the developer themselves, improving devex.

We already have that divide in the sync/asyncio world for Python. If I want to interact with AWS/S3 and my code is sync, I need to use boto3, if I want to use it in asyncio, I need to know that boto3 is not asyncio safe because it has blocking IO so I either need to make my own interface or use a project like aioboto3.

Once free-threading is also default, that problem will also be multiplied. What libraries are asyncio safe? What ones are thread safe? A unified interface can make it easier/more straight forward for maintainers to include or make support obvious to end users.

5

I’m not saying they’re not both useful, nor that they can’t be used together (far from it). I’m saying I don’t see how your code can be written to swap between them without knowing which one is being used.

But hopefully I just lack the imagination to get it to work, and the OP has an implementation in mind that would solve the problem.

6

Yeah. And to truly take advantage of them, you need to write code differently. Threads means you don’t need await points, async functions let you avoid locks. I’m saying basically the same thing you are, but the other side of the coin - I see this as a huge advantage, not a flaw to be solved.

(The number of times over the years that I’ve wished for actual threading in JavaScript…)

2 Likes
7

This is a topic that’s come up multiple times. Something you’ll run into with it is that asyncio is explicitly designed and users have written code that depends on context switching being well-defined. You won’t have an implicit removal of await without breaking the ecosystem.

From the home assitant side, I would suggest that rather than try and have both interfaces, you just run each thing how it wants to be run. This is relatively trivial by scheduling all async code to a shared event loop, and having all sync code scheduled to a thread pool or executor.

I maintain a light-weight library for this and other concurrency recipes. There’s a recipe here for scheduling to an asyncio event loop in a thread from other threads (Wrapping it in a context manager that gives managed access and concurrent.futures back, allowing the use of scheduling to async from code that needs to handle a mix)

If you don’t try to unify the interfaces, and just use a high-level abstraction to schedule each to the right place, it’s relatively pleasant to mix and match as appropriate.