Connecting Asyncio and Tkinter event loops?

smontanaro · March 30, 2022, 5:38pm

I recently resurrected a long dormant tkinter program. I was mildly
surprised it still worked with no (or minimal) changes. It contained a
commented out chunk of code which read “uncomment for Python 1.4”.
I’m sure at some point I “ported” it to Python 3, but I don’t recall
if or when that might have happened.

I’ve never used asyncio, but I got to wondering about better tracking
of keyboard and mouse than my little program implements (it’s a
computer activity monitor to force you to rest when needed). I thought
about using pynput, which seems geared toward asyncio. That got me
wondering if providing an async event handler function for tkinter
makes sense. Given that both are in the stdlib, it seems like it might
serve as a template for how it might work for other GUI libraries.

A quick check didn’t reveal anything, but did suggest questions about
linking the tkinter and asyncio event loops do come up from
time-to-time.

guido · March 30, 2022, 6:37pm

Hi Skip! When asyncio was first introduced we did give some consideration to integrating it with UI event loops, but other than thinking “it should be possible” I am not aware of anyone actually creating such an integration.

smontanaro · March 30, 2022, 8:37pm

Thanks, Guido. Would this be the right place to hammer out the details? I can’t see this requiring a PEP unless there are more than two opinions about were such a function belongs. I’ll see if I can create a hello world to start with.

guido · March 30, 2022, 9:31pm

I would love it if you managed to get this working! Agreed that no PEP would be needed. Maybe the new APIs could be in a new submodule of the tkinter package?

blink1073 · March 30, 2022, 11:28pm

We have some (somewhat hacky) asyncio/tk integration in ipykernel: ipykernel/eventloops.py at fc51ba4cccbc3b56323f9fc75f0ba631916812d9 · ipython/ipykernel · GitHub

smontanaro · March 31, 2022, 2:39pm

I came up with the simplest thing I could get to work (warning: I have never used asyncio before), then extracted just an AsyncTk class. I’m sure it’s missing things a full-fledged asyncio-friendly Tk event loop would need.

gist.github.com

https://gist.github.com/smontanaro/5e12c557602a76c609e46ca59387ad1c

tkasyncio.py

#!/usr/bin/env python3

"Testing the idea of providing an off-the-shelf Tk event loop for asyncio"

import asyncio
import random
from tkinter import Tk, Button

class AsyncTk(Tk):
    "Basic Tk with an asyncio-compatible event loop"

This file has been truncated. show original

matmel · March 31, 2022, 2:54pm

You might be interested the Trio “guest mode” approach to the problem of interoperability of an async event loop and a GUI event loop.

I have no experience with it personally, just know that it exists. For what I understand it seems to work. And as asyncio will gain task groups in python 3.11 you may be able to reproduce that approach if you think it is worth it.

encukou · March 31, 2022, 4:16pm

FWIW, I used the Qt integration projects quamash and then asyncqt for some light GUI work in the past and they felt pretty seamless. They’re unmaintained now, the current successor seems to be qasync.
So yes, it is possible. Thank y’all for thinking about it :‍)

congzhangzh · November 30, 2024, 2:36am

maybe we can follow electron’s way to make it work, which seems fit all the conditions!

github.com

electron/electron/blob/8f00cc9c0e78b95e4de14b9b9e1a7df20935e948/shell/common/node_bindings.cc#L929


      
            DCHECK(task_runner_);
            task_runner_->PostTask(FROM_HERE, base::BindOnce(&NodeBindings::UvRunOnce,
                                                             weak_factory_.GetWeakPtr()));
          }
          
          void NodeBindings::WakeupEmbedThread() {
            uv_async_send(dummy_uv_handle_.get());
          }
          
          // static
          void NodeBindings::EmbedThreadRunner(void* arg) {
            auto* self = static_cast<NodeBindings*>(arg);
          
            while (true) {
              // Wait for the main loop to deal with events.
              uv_sem_wait(&self->embed_sem_);
              if (self->embed_closed_)
                break;
          
              // Wait for something to happen in uv loop.
              // Note that the PollEvents() is implemented by derived classes, so when

guido · November 30, 2024, 5:15am

I can’t follow the code you quoted, but this has been a long-time wish for me. Can you write code to do it, or are you just suggesting we do it? Because this needs someone dedicated to follow through, and the existing asyncio devs just don’t have the time.

congzhangzh · November 30, 2024, 6:44am

sure, I’d like to make a demo project to test this idea! @guido

ssweber · November 30, 2024, 2:59pm

This library looks promising:

async-tkinter-loop

brnsb · November 30, 2024, 7:05pm

I need to look through my old projects, I had a very specific need for asynchronous tasks in Tkinter a couple of years back. By memory, I had a very similar approach to how they’ve handled it.

congzhangzh · December 1, 2024, 11:40am

First, need some help on libuv & python micro task process

I feel a simple PEP to guide the eventloop implement and gui message loop should works, and should works with all UI framework like QT/GTK/Win32/TK/Mac, some implement like pyside is very trick and too bad!

My prototype is work in progress, I make it work in some version, which use timely poll, which is bad!

I try to use semaphore to make a better one, it’s broken, should be fixable in the next step!

github.com

congzhangzh/python_gui_with_asyncio/blob/main/tk_with_asyncio_concept.py

#PEP idea

# **Functional view**
# Loop implement side
    # loop.check_event()
    #   check if there is any event to process
    # loop.run_once()
    #   run all events in the queue
    # loop.process_ready()
    #   process ready events (micro task)
    # loop.process_timers()
    #   trigger all timers
# Gui event loop side
    # hook_micro_task_process(callback)
# New sementic of asyncio running loop
    # running=true or integration_mode=True
    # asyncio.sleep or related coroutine will not raise error when loop is not running (integration mode=True)
    # asyncio.sleep will trigger loop.trigger_timers()

# **Runtime view**

This file has been truncated. show original

Btw,

I use uvloop and Linux to test the idea
I fork uvloop and create a custom version
Integration to Ui loop should be easy, it seems all ui framework provides some utility to callback on ui thread, and support hook the message loop to do something at each loop,We need this to process microtask(loop’s ready queue)

graingert · December 3, 2024, 11:01am

I’ve had some luck using trio guest mode and trio-asyncio

github.com

richardsheridan/trio-guest/blob/master/trio_guest_tkinter.py

#
# Copyright 2020 Richard J. Sheridan
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import collections
import tkinter as tk
import traceback

import trio

This file has been truncated. show original

richardsheridan · December 4, 2024, 4:17pm

aioguest probably deserves to be mentioned in this thread as well.

guido · December 13, 2024, 12:06am

Hi Cong Zhan,

I don’t see your post in the Discourse thread, and I also don’t see it as a Direct Message. So I’m not sure what happened. Maybe this reply appears on Discourse.

In any case, I recommend that you look into the Trio Guest Mode and aioguest (the same for asyncio), linked in the Discourse thread. I’m not sure I understand your code well enough to see what it does – do you have an explanation somewhere?

(I also recognize that you may have some trouble with English – alas, I have trouble with Chinese, and I’m not sure how to bridge that except via e.g. Google Translate or ChatGPT.)

davidism · December 13, 2024, 12:28am

They deleted their reply.

mikeshardmind · December 13, 2024, 3:34pm

I’m not sure how well connecting these would go from other experiences, it’s generally been best (In my experience anyhow) to use threading to run each needed event loop in it’s own thread, and keeping the UI event loop on the main thread, and put an asyncio event loop in a background thread, then communicate between them as needed.

I’ve been putting together a list of specific concurrency functionality within the standard library that if we could get stronger guarantees about safe usage that won’t be broken in the future could make this route more obvious and something that could be documented for use, possibly even with some repository template that has the skeleton of what’s needed already provided.

So far, the things that aren’t guaranteed but that have remained safe for years that would enable this:

The safety of passing an asyncio.Queue between threads so long as methods of the queue are only used from the event loop it is bound to.
The ability to create a concurrent.futures.Future not managed by an executor (This is specifically documented as something people should not do currently, but is safer than documentation suggests so long as only user of .set_result is the creator of the Future)
The ability to use .set_result on said futures without an executor (also documented as not to be done)

The concurrent.futures.Future bit is a little awkward since while users can recreate the functionality without using a Future, it then won’t work with things like asyncio.wrap_future, or concurrent.futures.wait( ..., return_when="FIRST_COMPLETED")

guido · December 13, 2024, 4:15pm

Did you look at the docs for Trio guest mode? The do this but improve on it by executing all user code (I.e., callbacks) on the main thread. That makes a lot of code simpler to port (apparently, I haven’t tried it).