Add Virtual Threads to Python

Ideas
184

That is also not true.

185

This thread was started by a core dev…granted, that was a long time ago

3 Likes
186

I can attest to this. I built a relatively simple app in Go: web server, open socket connection. Then I asked the LLM to convert the entire thing to Python which it did, but it took quite some time going back and forth between asyncio/trio/etc. to get it right.

When asking around then about whether there’s any “best” way to do this in Python, the answer was: not really.

For all the wailing in the Rust community around async, at least there’s a clear winner there (tokio) which, if you pick it, will make your life pretty easy.

This is a burning topic in Python and anything you can do to fix it, would be much much appreciated.

2 Likes
187

Among languages that use async/await, Rust is quite an odd one.

Rust chose postfix await.
It’s not the prettiest syntax, but it’s practical—especially for method chaining:

let result = create_async_chain(5).await
.method1().await
.method2().await
.method3().await;

This reduces the “disjointed” feeling between synchronous and asynchronous code, making them feel more consistent.

The Rust community has also been actively working to align the asynchronous development experience with synchronous code:
https://github.com/rust-lang/rust-project-goals/issues/105

Interestingly, someone once proposed postfix await for C#, but it was rejected:
https://github.com/dotnet/csharplang/issues/4076

Ultimately, I think many people (myself included) just want to write synchronous-looking code and have the system handle the asynchronous waiting automatically.

5 Likes
188

I don’t really care about coloring/syntax issues, but virtual threads are really mind-blowing.

One thing I understand is that eventloop is very lightweight, but a blocking single task will stuck everything! This will be different with virtual threads (which are responsive). Even on a single OS thread, instead of freezing instantly, other tasks can still run but just get gradually slower. - This is fairer but comes at the price of runtime scheduling. Whereas eventloop is lazy/efficient.

1 Like
189

Should we include go routines in this discussion? I really like that approach.

2 Likes
190

+1 for me for virtual threads along the line of Java’s, I really like:

  1. No colouring of functions.
  2. Errors propagate from children to their parent.
  3. Cancel exists!
  4. Cancel propagates from the parent to its children.

However as others have demonstrated with the IODict.incrementer example above there are still problems, in the case of the example the sharing of t is the bug. Other languages, recently Swift, eliminate this bug by preventing shared access in parallel operations.

So in addition to Java style virtual threads I would like ownership constraints, in particular:

  • Add the concept of a transfer. I would propose the annotation @transferred_args that is used as an annotation when declaring a function like gevent.spawn that says that any mutable arguments the function takes are transferred and use the keyword transfer to transfer a mutable object to a transferred_args function. To transfer an object it must have a single reference to it. Once transferred that reference becomes None. Immutable objects don’t need to be transferred (the compiler would have to know common immutable objects like int and str and about frozen dataclasses, etc.).

For the sake of an example, lets assume that gevent.spawn had been annotated with @transferred_args then the IODict example would be the same as before except for:

gevent.joinall(
        [
            gevent.spawn(incrementer, "a", "k", transfer t),  # Note transfer keyword on this line and next
            gevent.spawn(incrementer, "b", "k", transfer t),  # Error t already transferred
        ]

The error in the code would be caught because an attempt is made to transfer t twice. Note that functions do not exist in two colours, in particular IODict can be used in single threaded and parallel code. Also note that spawn exists in one colour only, @transferred_args.

1 Like
191

‘Normal practice’ simply means it’s common, not that it’s optimal or appropriate. What was pragmatic in one case can be sub-optimal or gate keeping of better fit pathways exist.

We shouldn’t confuse the two.

192

I think it would pose a problem if cancellation became a thing outside of async code. Where could code be cancelled then? What happens when code that wasn’t designed for cancellation starts suddenly seeing cancellations? I think this is where function coloring has an upside.

As for “transfers”, you didn’t explain the concept. What would that do really?

193

Anywhere that it can currently get a MemoryError or KeyboardInterrupt. IOW, pretty much anywhere.

194

What happens if the thread gets cancelled during a system call?

195

It notices afterwards. This is all exactly as existing Python code is; there’s a check to see if an exception needs to be raised. Cancelling is just raising an exception.

This would mean that an exception can be raised at any point in any code. That is, in fact, exactly what you get when you aren’t colouring your functions - but even when you are, since certain types of exceptions can happen literally anywhere.

196

Ok, that explanation makes perfect sense. But this makes it clear that virtual threads can’t replace async by themselves, as you can’t cancel (regular) socket operations. It would of course be possible to use the same mechanisms as async event loops, but it’s not clear how well those would work in a multi-threaded environment.