Asyncio without function colouring

Ideas
1

If you want to await you need to be in an async def function. I propose changing Python to allow await, async for and async with in any function.

def myfunc():
    await myasyncfunc() # same as if you're in an async function
    await myfuture # inside a Task the future is awaited; outside, an Exception is raised
    async for myasyncgenerator:
        # same as if you're in an async function
        ...
    async with with_item:
        # same as if you're in an async function
        ...

Why this is helpful: if you use async you cannot make good use of async within Python’s constructs like @property, __getattr__() etc, C-implemented functions like len() and setattr(), and operator overrides like __lshift__() as you have to wait for for the async result before returning, which negates most of the point of async, to be able to do other, useful work while waiting. Also, any library which wants to support async and non-async has to be written twice - once async and once without. These together have made async only useful in niche cases, in particular not for web sites, the highlighted use case async was added for.

(aside: I tried using async in a Django website, and concluded it was unusable in its current form, which is why I decided to try and improve Python).

I’ve taken the trouble to create a proof-of-concept cpython which does this: GitHub - JonathanRoach/cpython-await-anywhere: The Python programming language, with await allowed anywhere · GitHub (await-anywhere branch). I’ve tested this (make test without any fails) on Mac and Linux, debug, non-debug and optimised, but not Windows yet. Any feedback is very welcome (even ‘it doesn’t work’). The only new assumptions (over cpython) are that C standard setjmp() and longjmp() are available.

I am seeking a sponsor for a pep for this (@python/pep-editors please could you help me find one).

There is plenty more detail (eg ‘how?’ and ‘what are the catches?’), but lets get the community’s views first, over to you…

Add Virtual Threads to Python
2

First have a discussion. Then if a core dev is expressing support, you can ask him/her to consider sponsoring the PEP.

And to have that discussion, you have to explain what the exact semantics of await in a non-async function are. How does that work?

1 Like
3

I think the basic concept behind this has been discussed before, although I don’t remember where exactly.

I myself have often wondered about the same situation you were facing, basically mixing async and non async code in a function (or module namespace).

4

I think it probably happened in this discussion about virtual threads.

1 Like
5

It has. I contributed to that discussion, but I really I was more narrowly focussed on avoiding the function colouring problem of asyncio, so I put this narrower idea up for discussion.

6
  • await myfunction_async(). From the Python programmer’s POV this is a function call. You call myfunction_async() the value of the await is the return value from the function. This is the case in async and non-async functions. Behind the scenes, its operation is much the same: when possible, the function call is inlined, otherwise .send() is used, exactly the same as python works now. However, .send() on a coroutine (thing returned by calling an async function) now sets up a C and Python stack. The C stack management is handled by Python/coroutine.c, the Python stack is simply the Datastack from Python, but as a thing in its own right. TBH, if you don’t have to await a function, you will be better off not as await myfuntion_async()is noticeably slower and takes more typing than myfunction_sync(). In the proof-of-concept the only time you must async def is for a Coroutine to pass to asyncio.Task(), but the proof-of-concept could be extended to accept a callable to asyncio.Task.
  • await future. In a asyncio.Task the future waits, like now. Outside an asycio.Task an Exception is raised. Behind the scenes, Future.__await__() is re-engineered to behave to the caller as if it only ever completes (return value, or exception), to yield itself to the Coroutine’s .send() it calls a new function Coroutine.doyield() (a new function) which uses the Python/coroutine.c yield mechanism to preserve the C execution state, wrapped in code to preserve the Python state and yield the Future to the caller of .send(). As far as Python code is aware the behaviour is unchanged (unless you’ve rolled your own Future-like class). You can now yield in C using _PyCoro_DoYield(). Yielding and sending are much more efficient as they avoid passing the Future up the Python stack, and the .send() value down the stack - the C simply continues from where it yielded.

For almost all Python coders, everything works the same, except you don’t need async def, async for and async with (they work, you just don’t need them, IMO use the sync ones they’re less typing and faster). There are exceptions: Future awaiting is implemented differently (see above) - this will affect home-brew Future classes; (x async for x in myasyncgenerator()) is now a sync generator, not an async one; KeyboardInterrupt now behaves the same in an async.Tasks as in sync code; await and async aren’t syntax errors in sync functions.

7

See my earlier reply for the Python side - please ask if you want more detail in any area. Here, I’ll focus on the C side here. The goal was C coroutines without needing a special coding style by the user and only using standard C libraries. Only setjmp() and longjmp() are new to cpython, and these are widely available C standard libraries. There is no assembler.

The approach is to manage the C stack as a memory heap. The start of each block is the stack frame of a routine with a setjmp() recorded to re-enter that routine. These are the entry points to coroutines, called here entry_point(). To divide up the stack entry_point() calls another routine, called here use_some_stack(). use_some_stack() uses alloca() to allocate some stack before calling entry_point(). The stack now looks like this: <(A) entry_point()><(B) use_some_stack(N) with N bytes of alloca()-allocated space><(C) entry_point()>. (A) has N bytes (and a little more) of stack headroom before bumping into (C). An coroutine needing N bytes or less can longjmp() to (A) and run knowing it has the headroom - this is how coroutines are entered.

When a coroutine yields the stack looks like this: <(A) entry_point()><coroutine doing stuff……><(D) yield_here()><(C) entry_point()>. yield_here() uses setjmp() to allow a re-entry when the coroutine resumes.

For managing the heap/coroutines 'entry_point() can:

  • Chunk_Create: alloca() some space and call entry_point()
  • Chunk_Split: divide the space between this entry_point() and the next one into two pieces.
  • Chunk_Enter: start a coroutine

and yield here() can:

  • Chunk_Create: alloca() some space and call entry_point()
  • Chunk_Enter: resume the coroutine.

See Python/coroutine.c for the implementation in the proof-of-concept.

The heap is a merge-on-free, address ordered, first-fit style of heap, managed using linked lists. This kind of heap works very well up to about 50 free blocks, and should be a good fit for Python C stack fragments.

8

What do you think it means to be the “same as in an async function”?

If this is accurate, this is worse than the status quo. It means we go from clearly defined function coloring, to just hiding the function color and sometimes things will fail, or maybe even just deadlock depending on the reasonable assumptions something was written with.

9

If you decorate your function async or not, these work the same. The called async function doesn’t notice a difference, and nor does the async generator, nor does the async with item.

10

So basically, you’re eliminating function colouring by making all functions implicitly async def ...? Is that your solution?

How do these constructs work in a program that ISN’T asyncio-based? The function colouring problem is about supporting things in two different contexts, and all you’ve shown so far is that things magically work for asyncio.

11

It depends if the benefit of having function colouring outweighs having: libraries, @propertys, __len__(), __setattr__() etc, __lshift__() etc, some of the builtin function, like all() and any(), and having almost all your code base ascync. For me, I prefer coding with access to that 80% of Python’s super-powers.

On your point about unexpected deadlocks - I trust a coder who is using interlocks of some sort, and so could deadlock, is aware of the pitfalls. All Python is preemptive-thread capable, but deadlocks don’t seem to be a big issue.

12

And yet that literally can’t be true if you can’t await a future without already being in a task.

You can use all of those just fine in an async context, none of those should need to be async because none of them should ever block the event loop for a noticable amount of time, or you’re misusing the operators.

Existing async code has been able to make assumptions about the event loop that are guaranteed by it. It’s actually a benefit of function colouring detractors don’t ever properly acknowledge. Removing it changes invariants existing code has been designed around. It’s possible to intentionally not use locks and rely on structured concurrency to avoid data races. This goes away if you try to remove function coloring, and if you do it badly where some things are awaitable and others aren’t, it won’t even actually have the benefits you’re claiming.

2 Likes
13

If you don’t like asyncio, don’t use it, but proposing something that ruins the actual benefits for those who do like it isn’t helpful.

1 Like
14

Nope. The distinction between async def and def is the same, ie calling an async def returns a coroutine which can be awaited, and calling a sync function just calls the function.

To make this possible, C coroutines is the ‘magic sauce’. They allow C stacks to be set aside - the Python data stack was already in a good state to be able to set it aside. In my more detailed replies earlier I explain how this works.

15

This claim doesn’t hold up. If you have a problem with a specific framework you’ve tried, that’s not async unable to be used for websites.

Instagram is pretty much entirely async, and much of the work they’ve done to optimize their own use has led to improvements in cpython.

I’m personally skeptical of being able to await anywhere, as @Liz said, there are assumptions people are currently allowed by the language to make with regard to async code.

The inability to await a task also neuters various code. My own libraries wouldn’t work with this, as they return tasks to be awaited, as this pattern allows efficient caching of async functions.

16

being able to await anywhere doesnt remove function color if you’ve done it correctly, it just changes the scoping so that awaiting is valid everywhere.

There are several problems with this you’ve just sortof glossed over. The language allows 3rd party event loops, and functions which are allowed to both detect or depend on these. you’ve made the behavior different for 3rd party futures on even the type something results in:

This isn’t a change that can be argued lightly, and certainly not without acknowledging that it would actually be a breaking change.

I haven’t tested your changes yet, but your own comments about them lead me to conclude this is a misguided change that doesn’t fully understand the existing async landscape.

While not entirely accurate, this is the one part of the argument I have some agreement with, though I believe syntactic macros would be more promising here as a solution than trying to change behavior people already rely upon.

macros would allow easily writing a sans-io core and generating both the sync and async interfaces to it. As it stands, this is doable now with a bit of AST manipulation or code generation outside of runtime, and not actually something which requires writing the same thing twice.

17

I do like it. Just when I tried to use it, not having access to all those features made it unusable, in practice (my use case is a Django website). What I’m proposing should free up asyncio to be many times more usable than it is now. Existing code will work. New code, and old code, without await will work.

To make asyncio work as desired in its pep (pep-492), the idea was to service more requests while waiting for, say, a database request to be completed. To do this need to await / async def your entire call stack - from web request when it arrives to the innermost database call. Only when you’ve got that will await the_database_has_replied allow another Task to service another request. ORMs or template systems lean heavily on those Python features I listed earlier. Django has tried to async itself, but the implementations end up waiting for the async bit to finish - there is no benefit, only cost. I’m trying to allow ORMs like Django to practically, and fully support asyncio.

18

Apologies, I should have highlighted that this (ie await mytask) works as before - your code should still work unchanged.

19

That’s not what you said above, so how much of what you’ve said am I meant to take at your word for intent?

20

Word. Proof-of-concept developed to make test clean. There’s not much point asking for something which is this big of a stretch without showing it can be done.

See GitHub - JonathanRoach/cpython-await-anywhere: The Python programming language, with await allowed anywhere · GitHub (await-anywhere branch). Built and tested (ie make test is clean). Targets covered: MacOS; Linux (both x86 64 bits). Builds covered: debug; non-debug; optimised. Note Windows isn’t built, tested & debugged… yet.