I’m designing a Python desktop application that is meant to be extended by third-party developers via plugins.
A hard requirement is that plugins can be hot reloaded both during development and while the app is running, without restarting the main application.
It it aimed to be something like VSCode/Obsidian etc. where users will be able to add UI elements as well.
What are my options.
[Misdirected answer deleted by author.]
I think you have written in the wrong link.
This is a pretty broad question and it’s hard to answer. It is certainly possible to write code that would allow hot reloading, but it can lead to surprising behavior in some cases (e.g., if multiple plugins and/or some other app data get into a mutually referential configuration). Some web frameworks have an option to allow hot reloading of backend code (although sometimes only in a development/debug mode), so you could look at how they do it.
If this is a critical feature, I’d suggest you first look at just implementing hot reloading of stuff on its own without the complexity of a desktop app. As in, write some kind of simple text-mode demo that has multiple plugins and see if you can work out some of the kinks.
UI-modifying plugins is a whole other kettle of fish and a lot depends on how much freedom you want to give plugins to modify the UI. A simple approach is to compartmentalize such modifications by say, allowing each plugin to define a single button or panel which then appears in a standardized place within the overall app UI.
The basic problem is that if plug-ins are imported like any other package or module (unless you want to run them in an external subprocess?), they’ll be cached in sys.modules. In a particularly peculiar Python environment, I added a little extra quietly documented component / button (primarily for the developer’s benefit) that deleted all my modules (the plug-in knew its own name) from the cache when activated. Messing around with sys.modules and sys.path is hacky. But firstly if sys.modules is only deleted from, then Python will just revert back to the normal import procedure (as for a cache miss). Secondly, I have done it myself, used it many times (at least by definition, used exactly in and only in the ways I intended and anticipated) and shipped the result.
This is bound to create all manner of bugs and glitches, if the application still has artefacts that were rendered from previous versions of plug-ins etc., that you as a developer are signing yourself up for. Personally I would make it abundantly clear to your users and plugin devs, that this support for third party plug-ins running their own arbitrary code, is you treating them as adults, giving them enough rope, that clashes and breaking their own system is very much a possibility, and any ensuing problems are their own.