There is an import loop issue created by “from X import Y” that is not present with “import X”. I don’t know if this has been resolved in recent Python releases (I primarily use 3.7) - I searched this forum but didn’t see any discussions.
If this issue still exists in Python 3.10/3.11, then I am proposing adding a change to Python where “from X import Y” will use the same type of forward referencing for symbol Y in X that “import X” uses to forward reference X at the module level. This will resolve many “level 1” import errors caused by import loops in cases where only references of symbols are required. For example, if Y is a class, if an object of the class is never instantiated within the code in the import loop, then the loop should be resolveable directly by Python as it successfully does with “import X” loops.
I look forward to any input. I have not started a new PEP for this, seeing if this is reasonable and not already addressed first. Thanks!!
I’m an intermediate to advanced Python programmer, leading a small team on a company project. Just recently had to really dig into import loops in Python and discovered the following:
- Using “import X” syntax will create a forward reference to the module X, and if “import X” is encountered against in the same module (i.e. same namespace) then the import statement will not be reexecuted, but will continue to the rest of the code in that module.
- However, using “from X import Y”, which is importing a particular symbol Y from module X, will NOT create a forward reference to Y in the local namespace, and if the “from X import Y” statement is encountered again in the same module (typically through an import loop), the statement WILL BE EXECUTED AGAIN, which completes the import loop and causes an ImportError to be raised.
The end result of this situation is that Python avoids import loops in any case where the references are not actually executed when using “import X”, but it causes what appears to be an unnecessary import error for the exact same case when using “from X import Y”. If Python would create a forward reference for the “Y” symbol and ignore the second request to import Y from X, then the loop would be broken, just as it is with “import X” at the module level.
Even in the case of “import X”, which creates forward references to resolve “first level” import loops, Python can only resolve the loop for the user as long as the user does not try to actually use the reference prior to it being resolved. For example, if module A imports B, and module B both imports A and creates a top-level symbol that instantiates/uses one of A’s symbols, that is an import loop that requires more exotic techniques to break than simply ignoring the second duplicate import statement when the loop occurs. However, if all of the A references to B and B references to A are simply forward references that are either type annotations (not needed at run-time) or inside of function definitions (not needed until the function is actually executed), then the forward referencing strategy works fine.
So, currently “import X” python code allows “level 1” import loops to resolve, whereas “from X import Y” does not.
One can fix the “from X import Y” import loops of “level 1” type with a couple of different types of semi-hacky solutions:
- Switch to using “import X” and “X.Y” references in all code. Essentially this means spending a lot of time to figure out when one can use “from X import Y” or just never using it on principle. Mass refactoring of code from “Y” to “X.Y” for references is not always easy or trouble-free, and why force refactoring anyway?
- Move “from X import Y” statements that are only required for forward references to the end of the code file. This means you need to constantly look for import statements both at the beginning and the end of the file. Import statements that are not part of loops and are used to access symbols directly in the code will still need to exist prior to their usage, so it causes a split in where imports are placed. Also, many tools developed for things like auto-importing symbols do not work with import at the bottom of a file
- Techniques that work with breaking more complex import loops will also work with “level 1” types of import loops, but are, by nature, more complex and require more code and a more complex architecture.
This state of different handling of forward referencing between “import X” and “from X import Y” has caused me a lot of confusion as I’ve learned Python, and I see the same confusion with almost all other beginner and intermediate Python programmers I have discussed this with. Online, when searching for how to break import loops there are a ton of questions, many of which go unresolved or get answers like “fix your dependency flow”, when just switching from a “from X import Y” to an “import X” statement and changing the reference from “Y” to “X.Y” will resolve the issue (if it is a “level 1” import loop issue).
All-in-all there is a lot of mystery from beginning programmers about how imports work. It seems (from an intermediate level user’s perspective), that the forward referencing that is applied to “import X” statements at the module level could be extended to apply also to symbols in the “from X import Y” statements and that would clear up the mysterious nature of import statements - both forms of imports would always work with “level 1” import loops that only need forward references during the loop, while the more difficult import loops would have to be resolved with other techniques.