I have got a set of tasks to run while also trying to maintain a communication between them and the code that controls GUI in my program. In short my requirements are:
The tasks should be run in parallel. They are already designed in a way they don’t require any communication or shared state between each of them at all
Each of tasks should somehow tell the process that run them about progress. This is in order to update a progress bar in GUI
GUI contains also a “Cancel” button that should stop all tasks as quick as possible (they should abandon what they do)
During all the time GUI should be responsive: it should continue to read inputs and draw controls.
I don’t know how to “chomp” this puzzle. I wish to have got some idea how to structure my code to work with those requirements.
PS: I’m not using any specific GUI library, I do the drawing on my own, inside a pygame framework.
As you have stated, it is a GUI based application. So, in your task function, include code to update the progress widget in the GUI. The task can be written as a function and wrapped as a task per the tutorials listed above. This will necessitate a type of loop via a timer to periodically update the progress widget. Just make sure that you write your script such that it does not hang up the tasks.
Button widgets have associated functions (or methods) by design. This association links the function with the button. Any time that the button is pressed, the associated function is called. Therefore, in the function body of statements, include code to kill the task(s).
GUIs are event driven applications. Thus, any time that a button is pressed (i.e., ENTER, CANCEL, etc.), the application will be respond.
For progress updates or for anything else that touches the GUI, post a message from the background thread for the GUI thread’s event loop to pick up and handle.
For cancellation, set a flag (e.g. an attribute on some object) from the GUI thread and check it periodically in the background thread, so that it can abort voluntarily.
How you post a message to the event loop depends on the GUI framework. In pygame, the way to do it seems to be with pygame.event.post. There’s a fairly good explanation on TkDocs of how to do it with Tkinter, which explains the basic concepts.
If the tasks are CPU-heavy, you probably want to run them in backgrounds processes, possibly using the multiprocessing module as Paul suggested. But you still need threads, as well, to manage the communication with the GUI, at least if you want to send back progress information. I’m not sure how to handle cancellation in that case – you could just kill the process, but the multiprocessing module seems unhappy with that sort of thing. (Or if you’re using a very recent Python version and your libraries are compatible, you could try the new free-threaded mode or isolated subinterpreters.)
As for the best way to structure things, I wish I knew! Maybe someone else can elaborate on that part.
I’ve had this problem myself, and solved it a slightly different way, using async code. Trio has a ‘guest mode’, which allows it to run inside another event loop, like Pygame or Tkinter, taking care of scheduling events to run async tasks as they become available. It also has robust cancellation support, so you can have the GUI just cancel a task tree, triggering it to be torn down safely. The big downside is you have to convert all your code to be async compatible, and ensure you add lots of checkpoints/sleeps so that it’s able to switch back to handle GUI events.