It is possible to reload modules – if a reloaded module contains enums, they will be recreated, and the new members may not compare identical/equal to the original members.
The reason for this not explicitly stated here is that each enum and its variants are supposed to be global Singletons. __eq__() then just uses the is operator (I think), i.e. it compares the object IDs as id() returns them. When using importlib.reload() –which is what is meant here with ‘reloaded` – the Enum gets recreated and thus it and its variants get new IDs. So I have a few thoughts and questions about this
When such a reload happens how do you figure out why/where it happens? I am not even sure why it happens in my application. It might be caused by airflow which has hot reloading capabilities. But I’m not sure how to find out.
How do you fix the problem if you cannot stop the reload from happening? In our codebase we typically use my_enum_var.value == MyEnum.SOME_VARIANT.value but that seems visually noisy and it defeats the purpose of Enums to have quick comparisons with is. Also it can go wrong if two different Enums happen to have the same values for a variant and I happen to accidentally compare them. Also now I have to make up my own variants. At this point I could just use a separate module with named constants.
Do you think we should expand the Docs here a bit? A reference to what module reloading is or a reference to importlib.reload() would go a long way I think. Maybe also mentioning that it is supposed to be a global singleton and that being the reason why it fails on module reload.
Why does the Enum docs not mention the definition of ___eq__()?
Thank you very much for your thoughts, inspirations, answers
Because __eq__ is not the only problem. (My initial qustion was, why are we only talking about enum here, but I see it’s from the enum docs.) In general, reload() is something to avoid. It executes the module body again, so you have to understand all the consequences in a particular case to use it successfully.
A module body is mostly definitions, but definitions in Python are executable code. They create and save (in the module state) the object they create under the given name, which finally is no more than a variable like any other. This means that all the classes, functions and methods (and enums) are new objects assigned to the old name. However, if you have captured the old meaning of that name somewhere, and that work is not re-done, there will be surprising results. You can demonstrate this at the prompt:
>>> class C: pass
...
>>> c1 = C()
>>> class C: pass # assigns a new type object to C
...
>>> c2 = C()
>>> isinstance(c1, C)
False
>>> id(c1.__class__), id(c2.__class__)
(2617365930160, 2617365920400)
This is quite different from import, where a second import of the same module will find the original and give you a new reference to that state. No such worries there.
Thanks for pointing this out - it hadn’t occurred to me before. But I think this is merely one of the myriad of possible ways, that a hot update can break a currently running process (or cause unintended behaviour). Even if the hot update is good and well tested, it might not be intended or tested to be used as a patch over a previously running live process. Weird mismatched state bugs can be particularly tricky to reproduce, let alone unpick. It reminds me of installing all your dependencies in one Python env (never using venvs), then upgrading a shared transitive dependency that, some other dependencies require the old version of.
Use reload (and mess around with sys.modules) only with great caution.
It all depends on how the app is architected, and yes, a hot update absolutely CAN break a running process if you didn’t plan for it.
I’ve done plenty of systems that hot-update, but I usually design it in from the very start (notably, separating state from code - no classes, just state objects - so the code can be replaced more safely). You MAY be able to get away with it by updating more code at once, but at that point, you’re possibly better off just restarting the app anyway.
Quite. I like to think that’s just what I did when I supported this as a feature (I just ensured I deleted absolutely all my app’s modules from sys.modules, to force a not-quite-so hot reload of all of them). However, clearly enum and other stdlib modules were not designed for hot reloads from the start.
Maybe there’s a third party alternative, but the design criteria seem to require mitigating arbitrary code execution by the user’s update.
I think if you want a persistent enum, they need to be backed to disk or some other single source of truth somehow.
I don’t think it’s so much that enum can’t handle hot reloads, as that your code can’t handle half-and-halfing of those enums. If you completely replace one with another, it’ll be fine. But that comes back to the separation of code and state [1] and the trickiness of using classes.
That said, though, Python’s import system really isn’t designed for hot reloads. If you’re really trying to get something that has dynamic changes to code, it is likely to be easier to avoid imports altogether, and directly load and exec your code. That way, you get full control of what’s copied and what’s reconstructed.
