A lot of my concern about this thread is the number of people who seem eager to not allow me to just use async/await.
In libraries I personally maintain, I tend to just write a sync version and an async version and make them as identical as possible API-wise. Under the hood, I use sync or async I/O as appropriate, and try to factor out common logic for re-use on both code paths.
Looking at some packages I frequently use:
httpx, which provides both sync and async implementations of its HTTP client object. From a glance at the source, it appears (though of course I could be wrong) both client classes share a common base implementation with a lot of the logic, and layer on the appropriate type of I/O.
SQLAlchemy, which provides both sync and async implementations of its SQL core and ORM functionality. I’m not an expert on its codebase, but as I understand it their async engine/connection/session/etc. classes are actually using greenlet to wrap and place the sync equivalents into an async-capable context.
advanced-alchemy, which provides both sync and async implementations of its Repository and service-layer classes. They maintain sync and async classes separately (as is effectively required because they’re tightly coupled to sync or async SQLAlchemy sessions and thus expose one or the other flavor of query methods), but appear to have at least some shared logic for non-I/O operations like wrapping error handling around their calls to SQLAlchemy.
In other words, code generation is not, in my experience, as common as you seem to think it is, nor is it anywhere near forced as the sole option for maintaining sync+async code paths in the same library or framework, which seemed to be your implication in earlier posts.
I know a thing or two about Django’s internals, and I think it is not a representative sample of what supporting sync+async in the same codebase looks like.
Appreciate the detailed reply here. I am not trying to argue with you on the existence of other libs, just wanted to expand on your examples as to highlight the issue I see with SQLAlchemy’s approach (and why I don’t consider it a good solution for the cases I’ve seen, which isn’t everything!)
httpx uses codegen through httpcore (see this script here). There is an async implementation of a low level set of libs, from which a sync version is generated. So httpx is using codegen, likely to avoid having two implementations of tricky I/O code without being forced into a command pattern style.
my understanding of SQLAlchemy’s support is at least some of its sync methods call out to an async implementation (see usages of await_ in there, where they have an event loop on a side thread and call out to that), similar to asgiref’s async_to_sync. My understanding is that this solution does not allow for calling into a sync API (that actually shims an async implementation) in an async context, however, due to the fact that asyncio’s event loop isn’t re-entrant
advanced-alchemy also has an await_ method (here) so I imagine similar constraints apply where that’s use
Especially with the list of examples, I am starting to feel like Django’s issue (not unique to Django but it’s also not as much of an issue with, say, httpx, or even just Django’s view layer) is about how using it involves Django calling into user code, that calls back into Django, that can call back into user code etc.
In particular if you want to call a user-provided validation method in, say, Model.asave, and that user-provided validation method does something like Model2.objects.get(), andget is actually a shim for aget , and you use SQLAlchemy’s await or asgiref’s sync_to_async + async_to_sync…
Model.asave is running in some event loop
Model._validate_thing is running in that event loop, calls out to get
get is actually async_to_sync(aget)… so will try to run something on the event loop manually
event loop lack of re-entrancy strikes because you can’t do loop.run_until_complete (or w/e) while inside a task being ran by loop
So you get to the color problem in reverse: within a user-provided override of asave (for example), the user cannot use a sync API anywhere. You can go sync -> async but not sync -> async -> sync.
Django is a messy example, but to my mind, I think the minimal example here is:
class ThingDoer:
def do_side_effect(self): ... # calls into ado_side_effect
async def ado_side_effect(self):
value = await self.aread_value_with_io()
self.validate(value)
await self._send_to_db(value)
... # do something with value
async def _send_to_db(self, value): ...
def read_value_with_io(self): ... # calls into aread_value_with_io
async def aread_value_with_io(self): ...
def validate(self, value): ...
In this sort of class, given what we can do with asyncio and absent virtual threads, it seems to me like one would have to establish the following sort of restrictions on user-created subclasses of ThingDoer, if you want to have only one implementation of the “do side effect” logic:
you cannot override do_side_effect, you must override the async implemetations of the API
you cannot call any of ThingDoer’s API (like read_value_with_io) in validate, as calls to the sync API might trigger sync -> async -> sync loops in ado_side_effect that cause loop re-entrancy issues
An implication of this: you are probably going to want to make every extension point in the public API of ThingDoerasync def, as otherwise your users will not be able to do much in that validate method. You probably want an avalidate.
So the easy way forward in this universe: tell users “here are these extension points, they are all async”. This doesn’t preclude sync APIs from existing! But it does preclude usage of those sync APIs within those overrides and within any code those overrides call.
So at the end of it all, you will be incentivised to turn all extension points across your codebase into async def, so that you don’t run into the dreaded await_ cannot be called from within an async task running on the same event loop. Use 'await' instead.-style tools from your async/sync helpers. And any method that does I/O will want to be async def. And any def methods that wasn’t doing any I/O but then starts using I/O… time to turn that into async def.
An aside: if you were to say “OK let’s just have two implementations, do_side_effect and ado_side_effect” (codegen), now you’ll likely need users to provide duplicate extension points on any override they do. They will need to provide validate and avalidate, for example. But if you share any of the call flow between sync and async your run the risk of sync -> async -> sync. At least on the I/O-involved paths…
I would really like to have a good answer for how to handle things like the ThingDoer example that doesn’t force usage of async/await.
Originally when posting in this thread I had a feeling that good first-class virtual thread support would give us some outs here, hopefully even an out where people could pick and choose whether to override sync or async extension points.
Thinking about it more I’m not so sure. Could just be a case of the ThingDoer API not being “good”.
Good point, they’re equivalently well-defined. As soon as you have multiple OS threads, the system will preempt and virtual threads necessarily become preemptive too.
I think this is the actual answer. Libraries shouldn’t manage event loops, applications should. Frameworks should at minimum provide an unmanaged use pattern, even if they want the basic use patterns to wrap this behavior for their users.
This means every applications naturally only has either it’s use going from sync → async or chooses to use the singular managed entrypoint of a framework.
Most application code using async libraries should do so in the simplest way that works for their needs like asyncio.run on an async def main function
Libraries that need an async event loop but want to provide a sync interface without reimplementing should start an async event loop in a thread, and schedule to it as needed, shutting it down when appropriate.
Maybe I’m not searching correctly, but I can’t find anyone proposing that they ever be removed. Just some people who aren’t a fan of dealing with them, and who would like an alternative.
I’m not proposing removing async and await. With virtual threads, I would hope they will fall out of fashion eventually, but I doubt they will ever be removed.
As proposed, virtual threads would be unable to support builtin functions in the call stack. I’m not opposed to adding support for that ability in the future, but the initial implementation would not support it. I will make sure that the design allows us to add it later if needed.
This is not an original idea and many languages have similar features, not just Java: goroutines in go and fibers in ruby were mentioned. I’m sure there are others. The reason I’m mentioning Java specifically is that virtual threads are integrated into the language and VM in such a way as to make the transition to them very smooth.
Context switching can only occur at well defined points in the program. But that is only if you are looking at the whole program. If you are looking at a single function only, then you will need to treat most calls are possible context switch points. There is no pre-emption.
Regarding true parallelism
Virtual threads are a means for single-threaded concurrency. However, for the pure Python version, they are normal Python objects and can be passed around like any other object. Combined with free-threading this would allow us to implement a NxM model where many virtual threads are run on a few operating system threads. This would, hypothetically, provide the utility of having thousands of threads with the low overhead of having only one or two OS threads per core.
If we add support for builtin functions (like greenlets) then those virtual threads would be pinned to a single OS thread.
This seems like the biggest limitation here. Most of my day to day Python code uses builtin functions extensively - range(), enumerate(), list(), sum(), and even more if we look at stdlib modules like urllib (uses socket under the hood) etc. I’m struggling to understand how code like that would work with virtual threads.
When you say “on the call stack”, do you just mean builtin functions that call back to Python code? If so, does sum(1 for x in range(10)) do that? The 1 for x in range(10) is Python code creating a generator, which sum calls (via next()) repeatedly. So my understanding is that this is a builtin calling Python code…
Sorry if I’m missing the point here, but I can’t tell how useful virtual threads would actually be because I don’t feel I understand this limitation well enough in practical terms.
There were some examples in original post that gave a flavour of what this is.
What would be very helpful is to see a bit more code - some small working prototype written in pure python or pseudocode that encapsulates the most important implementation details and user experience.
No explanation as to what constitutes an interrupt. And no explanation for
event_loop_continuation_send
This is probably intended for other core Python developers who likely do not need an explanation, and as such it is useless for the rest of us who might want to actually write code to use the proposed new facility.
I think this is a great idea and could totally see the async functions become continuation objects in the future similar to how async in JavaScript is just sugar for promises
Maybe whitelist native code calls which won’t rely on storing “pointers to the stack on the stack”?
Or maybe, more feasible, a mechanism to lazily bind the stack in place if numbers that look like pointers to the stack are found upon transitioning to a Python call?
I don’t want to derail too much, but there was already a lot of anti-async/await sentiment being expressed, along with a hope that it would become unnecessary in a “virtual threads” world.
And I guess I’m in the minority, but I likeasync/await.
async/await is one of 2 paradigms that I haven’t been able to force myself to adapt. For the time being I use threading, gevent or anything else that doesn’t require to do coloured functions.
But I will inevitably have to adapt something and I hope it is not going to be async/await.
I mainly write code in C#, and occasionally use Python, and I frequently use async/await in both languages. Although the syntax is very clear, too many awaits can be overwhelming and make the code feel a bit “heavy.”
I have a feature that connects to a device, retrieves data, and saves it in a database. Initially, I used context variables to temporarily hold the current state, but now I’ve decided to move that context into a database table so that the current state is written to the table each time — that way, even if the program crashes, the state won’t be lost.
As a result, nearly every line of my code starts with an await. There are more asynchronous operations than synchronous ones, and since you can’t have asynchronous properties, writing everything isn’t as clean and crisp as in synchronous code.
But in scenarios where synchronous operations usually outnumber asynchronous ones, async/await is still very useful. I do hope for some solution to the function-coloring issue (algebraic effects?), but implementing that would probably be quite difficult…
tldr: I’ve professionally developed in Python for over 12 years so far. I’ve had good experience with gevent, which was so fantastically integrated with the usual web stack that many juniors didn’t realise we had it (e.g. me for some time). Asyncio is the “what’s good is not new and what’s new is not good” situation.
There will be a few stand-alone sections in this comment - you don’t need to read all of them.
Insight on function-colouring
Asyncio doesn’t only suffer from sync-vs-async function colouring. When there are more than 2 loops, each loop is its own colour. I’ve recently experienced it when dealing with async tests invoking dramatiq tasks - the test worker used its own loop, distinct from the pytest-asyncio loop. The 2 loops effectively poisoned each other by using the same connection pool. Problems like this manifest when integrating 2 or more asyncio projects, so it is rather hard for a single project to foresee it.
Similarly, it’s basically impossible to embed IPython in a running async context - its internal asyncio stuff will break. Workarounds break as IPython’s core changes - ptpython is an alternative.
Someone who doesn’t know any better starts a project in asyncio
I am forced to use it to no benefit over gevent.
Usual tools (like py-spy) break. Asyncio-aware tooling underperforms, crashes, etc. Instead of doing project work, I am forced to fight asyncio demons, while no longer being able to do IO when debugging.
The genius of moving the execution model into the VM
Gevent (or eventlet) contains a genius idea: let’s move the sync/async execution model choice out of the program and into the VM! While monkey-patching of stdlib IO has been used as a scarecrow against gevent, it should rather be viewed as a feature request.
What could we get if we could choose the IO execution model when starting Python by specifying an env var?
100% code compatibility (besides blocking code in C) - async is in the VM!
no function colouring between sync/async - they become unified
no function colouring between 2 loops - the loop is started with the VM
no monkey patching
instead, “IO execution model frameworks” implement a well-defined interface
existing issues caused by gevent patching being done too late are no more
I think this episode of core.py might help answer it. I think Yury Selivanov kinda explains the origins of the idea and how it grew into maturity along with help from Guido.
You can find it on other podcast platforms like spotify, Apple podcasts or other platforms.
I created a virtual thread implementation in python already 3 years ago..
I had to write an asynchronous streaming system for thousands of ip cameras, solved this using python asyncio programming (what a crash course..!), and came up with the idea of virtual threads to organize asyncio tasks.
The idea in a nutshell: self re-scheduling tasks, neatly grouped into task groups (threads).
I wrote the idea clean into this python module:
Wow…
It seems you scored right on target for
“If the implementation is easy to explain, it may be a good idea” with this one ("self re-scheduling tasks, neatly grouped into task groups (threads) " )
I for one would like to see something like this moving forward.
In my opinion, the real cost of asyncio goes beyond function coloring. It has completely fractured the ecosystem. Most of all, I don’t see that it will ever be possible to mend that fracture. Correct me if I’m wrong, but asyncio was introduced (as Tulip) sometime in 2012. Python 3 was introduced just 4 years earlier. Unless you still have to deal with Python 2, using Python 3 today is a non-issue. That is far from the case with asyncio, and that is very likely to hold 4 years from now.
Python is supposed to be a friendly language, and not just for beginners. And, all else being equal, I do understand why explicit switching seems more friendly. But a fractured ecosystem is very far from friendly, and has much greater consequences than someone not understanding how concurrent programming fundamentally works. Perhaps worse, the messaging around asyncio draws beginners to it, and they end up using it in inappropriate situations where it only adds complexity.
In practice, and with an understanding of concurrency, I really do not think implicit switching is very difficult to reason about. Most of the time you can easily spot the switch points. Is it an IO call, or a sleep? That’s a switch point. However, it is true that you won’t necessarily know exactly when a switch will happen. But it is easy to write your software in such a way that it doesn’t really matter. Protect your critical sections and make sure that you are switching somewhere and you generally won’t run into any issues. I have used gevent (and eventlet before it) for many years, and only the initial learning curve (understanding concurrency in general) was challenging. It has been a pleasure to work with ever since. (Notice there are not so many articles expounding the horrors of gevent.)
I don’t know if adding virtual threads is the way forward (though I like the idea), but I believe that asyncio is a net negative to the ecosystem. We were much better off without it. Countless hours spent rewriting tried and true libraries only to end up with alternatives that can’t be used with anything else. They might as well be written in another language.