There are a handful of reasons I don’t find trio guest mode appropriate that are not related to this issue specifically[1]. There is one reason I don’t find trio’s guest mode appropriate that is relevant here, and that is that asynchronous background work should never prevent the UI from being responsive (and my threshold for local responsiveness is that I should never so much as see text input stutter). It is a pretty typical app architecture across languages to place user interaction on the main thread, and everything else elsewhere. It can at times be a little worse in python than other languages with the GIL, but there’s ongoing work to fully remedy that.
I’ve written some code that makes handling mixing asyncio and other concurrency slightly easier. Within that repo, there’s a context manager that starts an event loop in a background thread, and exposes the functionality to schedule or wait for tasks on the background event loop. It works from both synchronous and asynchronous contexts, allowing use with other concurrency paradigms included in the standard library, including running multiple asyncio event loops with free-threading[2]. Exiting the context manager stops the loop and joins the thread. While this is similar to trio’s guest mode at a surface level, at least for users, I’ve intentionally avoided trying to hide complexities that can’t be entirely smoothed over on behalf of the user, it should be obvious what will and will not work.
It’s on my todo list to better document these as well as the reasons they exist as separate utilities, along with a full prose guide about the issues that caused these to be needed, and some examples that won’t involve me sharing proprietary code intended to make doing the “most correct thing” intuitively understandable. It’s not my opinion that just adding threading is easy to get right, but that if one structures their application with the known limitations of thread safety to have logically separate units, many of the issues solve themselves.
I fundamentally disagree with trio’s philosophy on cancellation, and can’t use it anyhow since I rely on several libraries that use asyncio, and cannot be changed to use anyio, but that’s a much longer discussion, and it’s informed by wanting to make stronger guarantees about delivery in concurrent systems than many users of asyncio or trio need to make. ↩︎
While many larger applications will have a dependency that isn’t free-threading ready, including obvious ones like aiohttp (due to cython), it’s good to lay the foundation now. ↩︎
I accidentally deleted my replies because the forum rules only allow new members to reply to 3 posts, or there might be a cooldown mechanism where you can reply more after some time. I thought deleting them would let me continue replying
Let’s get back to our expectations. First, after looking at Trio’s implementation and the cool integration with various frameworks, I’m convinced that’s exactly what we want. The guest mode and approach is what I was trying to express in my PEP. I hadn’t discovered Trio before and had only seen timer-based and Tk integration solutions many years ago.
The sample I provide is a working one, with custom uvloop.
The overall working mode might be opposite to Trio’s approach. First, UI is started in the main thread, while message absorption (like file I/O/network etc.) happens in the worker thread, then returns to the UI thread for processing events. Each UI platform has its related mechanism - in the prototype, I used calllater/callafter for simple handling. Several key points:
The UI thread/Tk thread and backend thread cooperate through semaphores. Time waiting and event dispatching occur in two threads and cannot/will not happen simultaneously.
Timers are based on select timeout. If a new timer is added in the main thread, it interrupts the backend waiting early through an additional auxiliary pipe handle and requeues.
Detailed thread coordination hasn’t been considered yet.
I’m confident that with just a few small modifications, we can implement the guest mode. After seeing Trio’s guest mode, I realized my prototype was already a simplified version of the guest mode.
I was previously concerned that the standard library’s event absorption and execution couldn’t be separated, so I chose uvloop. But after reviewing asyncio’s event and other modules, I found that making adjustments is just as easy as with uvloop.
Additionally, I finally understand that Python’s coroutines are implemented directly at the virtual machine/bytecode level, unlike C#'s state machine rewriting approach. This has greatly helped me understand the entire coroutine system.
Sorry this happened to you. This server has had some recent trouble where posts would disappear by accident due moderation mistakes, and people have been deleting their own posts for various reasons, so people are quick to assume nefarious things going on when they see a deleted post.
I think you’re exactly right to propose implementing a guest mode a la Trio for asyncio. It may be a little tricky to do this right with alternate loops like uvloop (which one of asyncio’s co-designers cares deeply about) but don’t let that influence your design. We can initially advertise that guest mode is a feature of asyncio that not all loops need to implement. (It’s an application-wide choice anyway.)
This pull mode performance is not good; we hope for a high-performance push mode and to make things happen, like doing UI stuff immediately once needed but not waiting even at any small interval.
Sometimes it’s impossible to do UI work in a thread other than the main thread.
Then why do I say “they are not suitable for GUI programs”? It’s because they cannot immediately start/resume tasks. For instance, asyncio.create_task and asyncio.TaskGroup.create_task are the ones that start tasks in asyncio, but neither of them does it immediately.
Let me clarify what I mean by “immediately” just in case. It means the following test should pass:
import asyncio
flag = False
async def async_fn():
global flag; flag = True
async def main():
asyncio.create_task(async_fn())
assert flag
asyncio.run(main())
which does not. The same applies to trio.Nursery.start and trio.Nursery.start_soon (This has “soon” in its name so it’s obvious).
Yep - I’ve been running (at least simpler) TK UIs with this basic idea for sometime now, and got no issues:
TLDR: just keep an asyncio tasks which will call tkinter’s “update” (I don’t even put any time in the asyncio.sleep() call - just leave it at 0)
Maybe just digging some edge cases from this approach, if any, could be enough to have a call in stdlib’s tkinter to run its mainloop as an asyncio task.
The webview_python binding now supports async python call from the javascript side, which run webview in OS GUI message loop, and asyncio io/timer pool on a background thread, and asyncio io process/timer callback/ ready process will happen on OS GUI message loop main thread:)
