The problem is: I call a 3rd party library function foo.bar(). Is that function a switch point? What about in the next version of bar? What if it adds logging output? Is that now a switch point? What if I had logging turned off, but now I turn it on? In what libraries did I just add switch points? At least with function coloring it’s trivial to answer these questions.
11 Likes
I understand your concern, but my point is that it usually doesn’t matter.
def query(url: str) -> None:
res = requests.get(url)
foo.bar(res)
log.info('request complete')
Every one of those calls could be async, but so what? In the broader context of my program, query() will run in a coroutine and it will switch a couple of times. The only thing I really need to worry about are shared resources and blocking. The former is required whether you’re working with threads, asyncio, gevent, etc…. The latter is required for any non-preemptive multitasking.
Further, no I don’t know at a glance that foo.bar() is a switch point, but if it really matters, I can always find out!
edit: To extend this just slightly, since we’re talking hypotheticals… foo.bar() might also add .1s of CPU processing to every call. That would have a much more significant effect on my async program than a new switch point.
5 Likes
Agreed, but now the problem is “just add locks like you would for multi-threaded code”, which the industry does not have an awesome track record with!
3 Likes
In practice, not any more than you would have to with any async framework. The vast majority of the time it will be very clear which library calls are likely to do IO and which are not, as well as which are likely to keep state and which are not. A dict keeps state, but is unlikely to switch. log.info might switch, but it is unlikely to keep state. Libraries that keep state internally will likely protect it with locks, because threading is still a reasonable thing to do.
Regardless, the somewhat greater safety of explicit switching is just not worth the cost.
2 Likes
Everything becomes problematic when abused. Switching to an asynchronous environment is quite straightforward and doesn’t require building a complex async call tree. I’m pretty sure that if threading were used everywhere, it would quickly become annoying. I’m not entirely sure what virtual threads are, as the original post isn’t clear on that, but introducing a different approach to concurrency with the possibility of parallelism would be a significant advantage.
It would be nice if we spent more time developing abstractions like concurrent.futures and point beginners to them instead of asyncio.
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
seems like a really nice abstraction which is in my opinion for the simple use quite superior and more reasoning-able than involving the whole asyncio infrastructure.
4 Likes
This kind of example that looks like it would be entirely IO bound looks like the exact thing that asyncio does better than threading.
Less than 150 LoC gives you a well-typed wrapper around asyncio similar to thread executors. This particular wrapper runs an event loop in a single thread, and can give back awaitables or concurrent futures.
assuming a coroutine equivalent of load_url, here’s what the equivalent use to your example is (note: your example seems a little incomplete, you assign url in a loop and never use it…)
with threaded_loop() as bg_loop: # see link above
future_to_url = {bg_loop.schedule(load_url(url, 60)): url for url in URLS}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
All of the task running happens in that background thread, which means your sync code plays nicely with it (no await necessary from the caller) and the overhead of an event loop in a thread is generally smaller than that of an executor.
Maybe it’s less that we shouldn’t tell people to use asyncio, but that more utilities like what exists for threading need to be standard in asyncio as well.
5 Likes
By “function colouring between sync/async” I meant ecosystem fracture as well. So we’re complaining about the same thing.
As for what can mend this fracture, this project or a similar one could GitHub - gfmio/asyncio-gevent: asyncio & gevent in harmony
1 Like
Just to note, I don’t agree that asyncio is fracturing or harmful to the Python ecosystem in any way. Sometimes silence is mistaken for agreement, but that’s not the case here. It’s simply an effort to stay on topic. That said, if there are concerns about asyncio, let’s discuss them in a separate thread.
This also means I don’t share the rationale in the original post. A new approach to concurrency doesn’t need to solve a problem, and presenting it doesn’t require diminishing others’ work.
7 Likes
My original post may have come across as harsh, so allow me to say that I appreciate the motives of the asyncio authors, and I do think there has been a benefit in terms of learning, exploring concepts, and introducing many others to concurrency. While I do think the consequences of introducing function coloring were evident, it wasn’t obvious how significant an effect it would have. I hold that asyncio has fractured the ecosystem, and has been a net negative (so far), but I’ll allow that one possible way to mend the fracture could be a standard event loop.
That said, and to return to the topic at hand, I think this thread is suggesting that we learn from and move past asyncio and alternatives like gevent to create an even better, and more integrated option. Especially with the prospect of free-threading.
My understanding of virtual threads (as they’re implemented in Java) is that they’re based on continuations (fibers, greenlets, etc…) but provide a traditional threading API and support for structured concurrency. They have no special syntax for context switches, and therefore do not introduce function coloring. Just like gevent, switching happens implicitly on IO or yields (aka sleep). Some call this well-defined, others do not, but it is not explicit.
Sidenote: I found this quote discussing alternatives to virtual threads in JEP 444:
Add syntactic stackless coroutines (i.e., async/await) to the Java language.
…
It would split the world between APIs designed for threads and APIs designed for coroutines, and would require the new thread-like construct to be introduced into all layers of the platform and its tooling. This would take longer for the ecosystem to adopt, and would not be as elegant and harmonious with the platform as user-mode threads.
So no, virtual threads would not solve the problem of explicit switch points like asyncio does. What it does offer, however, is a lightweight and safer alternative to threads. There is still a pivot point between grokking where the switches happen (I do understand why that makes some people uncomfortable) vs. locking as you would with threads. Personally, I think the uncertainty/learning curve is a price worth paying for cleaner, more readable codebases. That, and not having to write two versions of everything.
As @encukou called out above:
We accept quite a lot of uncertainty in our programs all the time. Perhaps the reason some single out switching from the rest is because it’s hard to understand concurrency when you first encounter it. But it’s understanding concurrency that’s hard, not spotting switch points.
I think it’s worth reviewing all of the following for a deeper understanding of where we are with all of this (not allowed to link them):
- Unyielding (Glyph, 2014)
- Why we have asyncio
- What Color is Your Function? (Bob Nystrom, 2015)
- Why people don’t like asyncio
- Pull Push: Please stop polluting our imperative languages with pure concepts-Curry On (Ron Pressler, 2015)
- Fundamental concepts. Highly recommended
- Notes on structured concurrency, or: Go statement considered harmful (Nathaniel J. Smith, 2018)
- Structured concurrency makes asyncio (and other concurrency models) a bit better
- Playground Wisdom: Threads Beat Async/Await (Armin Ronacher, 2024)
- From Async/Await to Virtual Threads (Armin Ronacher, 2025)
- Extensions of the discussion in this thread
4 Likes
I’m happy to see that revived! And I hope it continues to get support. The last time I tried to use it, it wasn’t working and looked abandoned.
My 2c, the fracturing is what it is, and getting too upset at reality isn’t helpful. Different people have different needs and will make different choices.
What excites me about virtual threads is that the pathway for beginners can introduce concurrency later in the learning process. Where with async/await syntax beginners need to have some understanding of why and when we need to use those keywords. I think that’s an unfortunately complex topic for beginners to take on.
So async/await is still great for some folks, but I think there’s still independent value for others in being able to avoid async/await syntax and do great concurrency.
3 Likes
Where is that limit to at most five concurrent HTTP connections opened in one time?
Not present here, as you weren’t explicitly limiting to 5 connections, but to 5 worker threads managed by the executor, which is a concept that only makes sense in that executor due to system resource limits. It’s trivial to add such to it, but it doesn’t make sense to at that level of abstraction. (you can just set max connections on something like an htttp client, use a semaphore, or complete the example a bit more with something like a semaphore).
But we’re getting significantly further off topic. The point here is that people are making a lot of claims about async being bad to prop up the idea of virtual threads, but as someone who uses both threading and async in the same applications, in multiple languages, I just don’t see it. They serve different purposes and complement each other when used well.
Virtual threads have to stand on their own merits to be worth inclusion, and the various limitations proposed here don’t seem to be worth it to me, but the motivation expressed seems largely in frustration with asyncio, without any appetite to improve the frustrations with asyncio.
3 Likes
asyncio doesn’t really work in an M:N scheduling scenario (the asyncio invariant, where state doesn’t change between suspension points, cannot hold). M:N scheduling is a pretty big deal, I think especially in a “slow” language like Python there P99 latency is often pretty weak. This was the tipping point for me. If we had no free-threading (and so M:N was not really possible) I think asyncio would still be the best we could do.
2 Likes
M:N scheduling of independent tasks is perfectly possible with current tools. Work-stealing M:N scheduling is not. I don’t see workstealing as necessary, it’s not even always ideal in runtimes that have it. Tokio (over in rust) only manages to reasonably provide this because of rust’s borrow checker and other compiler enforced safety about move semantics, and it’s still often worse to default to tokio for everything, over a mix of tokio for IO, and rayon for non-IO parallelism.
In python, Reasonable services can accomplish this with either a work queue consumed by multiple threads for top-level entry to an event loop, with subtasks scheduled on the same event loop, or with a binning function for distributing work across threads and the same with subtasks on the same event loop for just independent tasks. For other things, it’s often good to dedicate specific threads to specific tasks, and have the work that does need the same state in the same thread.
Virtual threads as proposed here aren’t going to help with speed. I expect them to do worse. One of the limitations proposed is that you can’t call into builtins.
Yes, the door is being left open to changing that in the future, but without a commitment to it. If your criticism of asyncio is speed, then that criticism holds for virtual threads as proposed and is only a distraction from “does this proposal stand on it’s own merits?”
2 Likes
There’s no way around it.
A major problem is that the way people talk about this (numerous examples here in the thread) presents it as if there are two options: (A) async/await with a “function coloring problem”, or (B) some form of threading, with no problem.
But threading–whether “real” threads or “virtual” threads, or whatever lightweight threadlike thing people are proposing–is not magically free of problems. In fact, it’s full of problems so difficult that people go invent things like Rust to try to solve them.
Similarly, there’s never going to be an easy or gentle way to introduce this type of programming to beginners. Starting them out on threads (or lightweight threadlike thing) is just going to come with a different gigantic pile of issues and concepts you have to explain: a whole taxonomy of synchronization primitives and complex rules and techniques for using them correctly (something even seasoned experts frequently fail to do).
The more I think on this, the more I think the async/await style might be the least-worst option yet developed. If null references were a “billion dollar mistake” (in Tony Hoare’s words), then threading–or at the very least, threading in languages with shared mutable state–is probably into the many billions already in terms of dollar value of damage inflicted at this point. I doubt it can be made meaningfully simpler, and I doubt it can be made meaningfully safer without adopting the kinds of approaches that make Rust’s learning curve so steep.
3 Likes
Thank you James, your response gives me a little opportunity to clarify. You’re very correct that, with or without function coloring, concurrency is a topic that is complex and not beginner friendly. I’d hoped that async/await would reduce that, but I’ve not found that to be the case.
However, what I was referring to here wasn’t actually a comparison between different forms of concurrent programming, but lamenting the cognitive overhead required to understand async programming even when you’re not doing anything concurrent. Because the library a beginner wishes to call wants to be used by programs that might be concurrent, they have to either make their API work with the concurrent library (e.g. asyncio) and syntax primarily, or they have to implement both (or really all, for every loop implementation) colors of the functions.
Library authors don’t want to maintain parallel abstractions, and while sans io is awesome, I, at least, find it less than intuitive to write and maintain, and my feeling of the libraries I’ve seen seems to back that up to some degree. So they may, and do, choose to prefer asyncio and async/await syntax.
However, this places a new barrier for the new developer who isn’t yet ready to take on the complexity of concurrency. They need to understand why there are two function colors, and when to use them, and their rules. I don’t see this complexity as necessary. For example, Django community members often tend to discourage new developers from their inclination to asyncify everything, unless and until it actually helps them, which it often doesn’t.
Having beginners need to learn about (hopefully only the most basic parts of) concurrent programming and the rules for function coloring in order for them to do non-concurrent programming is what I lament. As long as libraries have good reason to think that asyncio coroutines are the only or preferred way to do concurrent programming, I only see beginning in programming getting more complex, when what I want to see is a more gentle ramp that starts with Python.
4 Likes
Correct, but also notable: Any library author has to take on the cognitive overhead of threads even when not doing anything concurrent. Threads offer a lot of convenience, but have a number of subtleties. The virtual threads proposed here have all the same subtleties as Python’s existing threads, with very little benefit that I’ve seen - everyone still has to be concerned with the issues of concurrency, whether they’re aware of them or not. At least async/await puts the yield points right in front of you.
There is no easy solution, so I am glad that we have many options for how to organize things, and especially glad that most of the options “play nicely” together (you could have multiple processes sharing work across multiple threads that each run an event loop).
3 Likes
As Chris pointed out, you still have to carry the cognitive load of threading even when you personally are not writing threaded code, because a library or framework somewhere might be doing threading without telling you. The async/await approach at least has the virtue of making this explicit.
And I’m not convinced that threading really leads to a meaningfully easier learning experience for beginners; if you skip over it at first, that’s a lot to suddenly spring on someone late in the process, and a lot of bad habits they’ll have to unlearn. Writing code on the assumption that it’s safe to open a file or modify the contents of a data structure, only to learn later on that it’s very very unsafe and in fact requires such care that even experts regularly mess it up, is not exactly a nice experience.
I think it’s interesting that there’s such pushback against the idea of isolating I/O operations. We’ve known, as an industry, for a long time that shared mutable state is a massive source of bugs. And when writing code with threads (or lightweight threadlike alternative), generally you still have to identify places where shared state–whether in-memory data, file contents, etc.–is potentially accessed and modified and protect those places, and one of the simplest ways to do that is to isolate those points inside functions/methods which implement the correct synchronized access and make it as difficult as possible to use them incorrectly. Several languages even have explicit syntax or annotations for marking such functions/methods (see Java’s synchronized for a prominent example). At which point you’re doing one of the things people insist is a downside of async/await, but without getting any of the benefits of async/await.
1 Like