tl;dr
Java has virtual threads. Virtual threads are a better way of doing concurrency than Python’s async and await. We should add virtual threads to Python.
Virtual Threads
Virtual threads were added to Java a few years ago.
Virtual Threads combine the best of async Tasks and normal threads.
Like normal threads, virtual threads:
- require no new syntax
- provide a more intuitive mode of execution than async/await.
Like tasks, virtual threads:
- only switch execution at well defined points in the code
- can support structured concurrency
- are lightweight
Unlike Python’s coroutines, virtual threads:
- do not divide the language in two “colors”. See What color is your function?
IMO, virtual threads offer a superior programming model to adding async and await all over your code and having to duplicate all your libraries.
But don’t just believe me, Armin Ronacher says so too.
Implementation
There is no point in suggesting a new feature if we can’t implement it.
There are two tricky parts to implementing virtual threads:
- Context switching: switching from one thread to another
- Not blocking on apparently blocking calls.
Continuations for context switching
Virtual threads in Java are implemented as pure Java objects using Continuation objects provided by the JVM to do the context switching.
We can do the same in Python. By adding Continuation objects to the CPython VM, we can implement Virtual Threads as pure Python objects.
Continuations, strictly “delimited continuations”, are a form of coroutine. Unlike Python’s coroutines which are stackless and asymmetric, continuations are stackful and symmetric.
Being stackful means that they can yield from within calls, not just at the top level.
Being symmetric means that they can yield to each other, not just their caller.
Although I expect Continuations to be mostly hidden, it might be informative to see an example using them to context switch.
This program prints “ping”, then “pong”, then “ping” and so on, until interrupted:
def bounce(other, msg):
while True:
print(other.send(msg))
pinger = Continuation(bounce)
ponger = Continuation(bounce)
pinger.start(ponger, "ping")
ponger.start(pinger, "pong")
ponger.run(None)
Non blocking calls
Suppose we want to send some data over a socket.
To do that, we would normally call socket.send(data), but socket.send blocks.
If we are using traditional threads, the operating system will run another thread for us, and it doesn’t matter that this thread is blocked.
If we are using asyncio, or equivalent, then we cannot make blocking calls, so instead of
calling socket.send(data) we must await loop.sock_sendall(socket, data) from an async function.
With virtual threads we want to be able to call socket.send(data) without all the async and await baggage and not have it block.
We can do this by using continuations. We can have one continuation running an async event loop and have that continuation manage the asynchronous operations. Something like this:
#module virtual_threads
class Socket:
def send(self, data):
return event_loop_continuation.send((SOCKET_SEND, self, data))
