Add forward referencing of symbol imports, similar to existing module forward references to break import recursion with "from X import Y" style references

Ideas
1

There is an import loop issue created by “from X import Y” that is not present with “import X”. I don’t know if this has been resolved in recent Python releases (I primarily use 3.7) - I searched this forum but didn’t see any discussions.

If this issue still exists in Python 3.10/3.11, then I am proposing adding a change to Python where “from X import Y” will use the same type of forward referencing for symbol Y in X that “import X” uses to forward reference X at the module level. This will resolve many “level 1” import errors caused by import loops in cases where only references of symbols are required. For example, if Y is a class, if an object of the class is never instantiated within the code in the import loop, then the loop should be resolveable directly by Python as it successfully does with “import X” loops.

I look forward to any input. I have not started a new PEP for this, seeing if this is reasonable and not already addressed first. Thanks!!

Description

I’m an intermediate to advanced Python programmer, leading a small team on a company project. Just recently had to really dig into import loops in Python and discovered the following:

  1. Using “import X” syntax will create a forward reference to the module X, and if “import X” is encountered against in the same module (i.e. same namespace) then the import statement will not be reexecuted, but will continue to the rest of the code in that module.
  2. However, using “from X import Y”, which is importing a particular symbol Y from module X, will NOT create a forward reference to Y in the local namespace, and if the “from X import Y” statement is encountered again in the same module (typically through an import loop), the statement WILL BE EXECUTED AGAIN, which completes the import loop and causes an ImportError to be raised.

The end result of this situation is that Python avoids import loops in any case where the references are not actually executed when using “import X”, but it causes what appears to be an unnecessary import error for the exact same case when using “from X import Y”. If Python would create a forward reference for the “Y” symbol and ignore the second request to import Y from X, then the loop would be broken, just as it is with “import X” at the module level.

Limits of Import Loop Resolution by Python

Even in the case of “import X”, which creates forward references to resolve “first level” import loops, Python can only resolve the loop for the user as long as the user does not try to actually use the reference prior to it being resolved. For example, if module A imports B, and module B both imports A and creates a top-level symbol that instantiates/uses one of A’s symbols, that is an import loop that requires more exotic techniques to break than simply ignoring the second duplicate import statement when the loop occurs. However, if all of the A references to B and B references to A are simply forward references that are either type annotations (not needed at run-time) or inside of function definitions (not needed until the function is actually executed), then the forward referencing strategy works fine.

So, currently “import X” python code allows “level 1” import loops to resolve, whereas “from X import Y” does not.

Hacky solutions

One can fix the “from X import Y” import loops of “level 1” type with a couple of different types of semi-hacky solutions:

  1. Switch to using “import X” and “X.Y” references in all code. Essentially this means spending a lot of time to figure out when one can use “from X import Y” or just never using it on principle. Mass refactoring of code from “Y” to “X.Y” for references is not always easy or trouble-free, and why force refactoring anyway?
  2. Move “from X import Y” statements that are only required for forward references to the end of the code file. This means you need to constantly look for import statements both at the beginning and the end of the file. Import statements that are not part of loops and are used to access symbols directly in the code will still need to exist prior to their usage, so it causes a split in where imports are placed. Also, many tools developed for things like auto-importing symbols do not work with import at the bottom of a file
  3. Techniques that work with breaking more complex import loops will also work with “level 1” types of import loops, but are, by nature, more complex and require more code and a more complex architecture.

The Need for This Change

This state of different handling of forward referencing between “import X” and “from X import Y” has caused me a lot of confusion as I’ve learned Python, and I see the same confusion with almost all other beginner and intermediate Python programmers I have discussed this with. Online, when searching for how to break import loops there are a ton of questions, many of which go unresolved or get answers like “fix your dependency flow”, when just switching from a “from X import Y” to an “import X” statement and changing the reference from “Y” to “X.Y” will resolve the issue (if it is a “level 1” import loop issue).

All-in-all there is a lot of mystery from beginning programmers about how imports work. It seems (from an intermediate level user’s perspective), that the forward referencing that is applied to “import X” statements at the module level could be extended to apply also to symbols in the “from X import Y” statements and that would clear up the mysterious nature of import statements - both forms of imports would always work with “level 1” import loops that only need forward references during the loop, while the more difficult import loops would have to be resolved with other techniques.

2

I do not understand what you think does not work.

This:

from X import Y

is the same as this

import X
Y = X.Y
del X
3

Hi Mike,

It looks like you’re still a bit confused – perhaps because you’re more familiar with a traditional language like Java.

There is no such thing as a “forward reference to a module”. There is a “partially initialized module”, which you may observe occasionally during an import cycle, e.g. when A imports B and B imports A, B can only see those things in A that precede the import B. For example,

# A.py
def before():
    print("Before")
import B
def after():
    print("After")

# B.py
import A
A.before()  # prints "Before"
A.after()  # raises NameError on 'after'

You will see the same errors if you try this:

# B.py
from A import before  # Works
from A import after  # ImportError on 'after'

If we change B.py to

# new B.py
import A
def foo():
    A.before()
    A.after()

and we call B.foo() from some other module that imports B, you get no error.

It is also quite possible that what’s biting you is the effect of “script” vs. “module”. If you have the first versions of A.py and B.py and you run python3 A.py from your shell, you will not get an error. But if you run python3 B.py, you will. This is because the script passed on the command line is not considered to be the same as the corresponding module, and the import will run a copy of that module.

If you disagree with this analysis, please post some short modules that demonstrate the behavior you are seeing – it’s much easier to understand a problem when there’s concrete code.

3 Likes
4

@LightCC, I don’t understand what you mean by “forward reference” or why you think it applies to Python imports.

There are two common meanings for “forward reference”, but neither seems to apply to imports.

I fear that your entire post, from the diagnosis of the problem you are experiencing to your suggested solution, may be based on an entirely false model of Python’s execution model in general and imports specifically.

It is difficult for me to be sure, because you seem to be using language in a way I don’t fully understand:

  • import X doesn’t do any sort of forward reference at all, at least not according to any definition of “forward reference” that I understand.

  • What do you mean by a “level 1” import error? What are these levels?

  • There are known issues leading from circular imports but your post seems to imply that you are having trouble with straightforward, simple, non-circular imports as well.

  • When you talk about import loops, do you mean circular imports where module A imports B which in turn imports A?

It is not correct that import statements are skipped if they occur twice in the same namespace. Run this to demonstrate that the second import does run:

import math
print(math)  # Show that it is a module.
math = None  # Rebind the name to something else.
print(math)  # Prove that the name still exists.
import math  # Run the import again.
print(math)  # The second import is *not* skipped.

What does happen is that the import statement doesn’t re-execute the module code on subsequent imports (regardless of which namespace they occur in), but reloads the module object from a cache.

I don’t understand your Point 2 here:

Sorry, I cannot reconcile your description there with the way Python behaves. Maybe I don’t understand the scenario you are describing. Can you provide a minimal reproducible example please?

I think that right now we are probably talking past each other. Even if your understanding of Python imports is correct, the language you are using to describe it seems to me to be non-standard, and I’m having trouble interpreting what you mean.

As far as beginners go, in my experience, beginners initially have trouble with the concept of namespaces, and so have trouble with understanding why you can refer to a bare sqrt after from math import sqrt but not after import math, where you have to use math.sqrt.

Once they understand that, the next stumbling block is module caching and reloading: when you change the source code on disk, re-importing the module doesn’t see those changes.

After that, the next problem is complex circular imports, but let’s be frank here, even experts have trouble with complex circular imports :slight_smile:

5

Hi Barry,

Thanks for your response. I agree with you from a syntax perspective within a given file when you are simply attempting to call an imported reference. i.e. the end result is the same in the immediate file of your example.

However, the reference within a Namespace is different, as shown by the syntax required to access the reference. When importing X you get a reference to the entire module, that you can dig into as deeply as you want. It also appears to be “pre-initialized” with a reference which doesn’t have to be immediately satisfied with all the remaining details (i.e. the definition of Y) until some later point. In other words, X.Y does not appear to have to be defined until you attempt to actually execute X.Y (i.e. y = X.Y() or something similar).

On the other hand, When importing with “from X import Y”, the import appears that is MUST IMMEDIATELY return the full definition of Y in the case of an import loop. There doesn’t appear to be a reference placed into the namespace which can be filled in later (in the case I’m discussing, through completing an import loop). And, of course, one can reference Y directly. It also doesn’t give access to any other symbol in X, except for any direct children of Y (e.g. if it is a module and has it’s own namespace, there could be a Y.Z, etc.).

However, your simple example breaks if you do what I’ve been calling a “level 1” circular import loop - one that only initially need references to the items, and the loop resolves before any of the items are executed.

Code to show the difference

Let me illustrate with some code. Create A.py and B.py within the same folder

A.py:

from B import B

class A:
  def __init__(self) -> None:
    self.b = B(self)

B.py:

from A import A

class B:
  def __init__(self, a: A):
    self.a = a

from X import Y style fails with an ImportError

Now run A.py:

> py A.py
Traceback (most recent call last):
  File "A.py", line 1, in <module>
    from B import B
  File "C:\code\temp\dependency_test\B.py", line 1, in <module>
    from A import A
  File "C:\code\temp\dependency_test\A.py", line 1, in <module>
    from B import B
ImportError: cannot import name 'B' from 'B' (C:\code\temp\dependency_test\B.py)

Now - take note that neither file actually executes any of the symbols. Class A and B are defined but have no class variables with values set, and no other symbols or variables are created - only function definitions within the classes are made. The only actual reference from one file to another other than the import statements is inside the class A __init__ function, when self.b is set to a new instance of B, but no code is executed that actually creates an instance of A or B. Yet, an ImportError is still created due to the import loop.

But import X style with X.Y reference succeeds without any error

However, if we use exactly the same code and just change from the “from X import Y” syntax to the “import X” and “X.Y” syntax, then this same code will run just fine (though it still does nothing).

A.py:

import B

class A:
  def __init__(self) -> None:
    self.b = B.B(self)

B.py:

import A

class B:
  def __init__(self, a: A.A):
    self.a = a

If I run A.py, it just ends with no output (makes sense). But I can even add some code that only runs if A.py is called directly. Append the following to A.py:

if __name__ == "__main__":
  a = A()
  b = a.b
  print(f'b = {b}')

And execute it:

> py A.py
b = <B.B object at 0x000001D1A938E748>

The sequence of events for each case (using debugging)

For the “import X” style syntax

If you step through the execution of both import loops, you will find the following steps:

  1. A.py executes with __name__ set to __main__ and at the first line, the “import B” statement, causes execution to jump to the B.py file.
  2. in B.py, the “import A” statement is executed and immediately jumps execution to a new run of A.py.
  3. A.py executes with __name__ is set to A instead of __main__. However, upon encountering the “import B” statement again, it DOES NOT return to B.py, but appears to accept a reference to B that was put into the A namespace when the first “import B” was run at step 1, which satisfies it - even though nothing is yet defined inside of B. It doesn’t actually jump to B.py, but just skips to the next line.
  4. It completes the definition of class A in A.py, then returns to the B.py file.
  5. Python and continues from the import A statement in B.py, goes through the definition of class B
  6. With B.py file complete, python returns back to the original A.py execution and continues through the file. At this point “B” is not just a reference, but also has B.B defined as a class.
  7. If any code is entered in the if __name__ == "__main__": block, that is executed with all definitions in both files complete - i.e. the import loop is complete with all A and B references complete in their respective namespaces for each module.

The sequence of events with “From X import Y” syntax

However, if you convert back to the “from A import A” version of the imports, you will find that the sequence is this:

  1. A.py executes as __main__. Upon executing from B import B it jumps to executing B.py
  2. In B.py, upon executing from A import A it jumps back to A.py
  3. Executing A.py with __name__ == A, it encounters the from B import B again, but rather than being satisfied that we already have a reference to B that is in process of being executed (in step #1), like it does with the import A case above, it generates an ImportError at this point.

Execution fails with the ImportError.

Conclusion

This is my long winded way of saying that, in addition to the pure syntactical differences between from X import Y and import X... X.Y, the import X option will resolve import loops that only require references (where no definition that is a part of the loop must be executed), while the from X import Y statements will NOT resolve the import loop with the otherwise exact same code.

2 Likes
6

Hi Guido,

Thanks for the response.

Yes, I was concerned that some of the language I was using would not be correct, and it probably is not, as there is a lot of confusion on the blog posts and Q&A discussing import issues.

I believe you are correct in saying that I appear to be seeing a “partially initialized module”. It appears that when the import X format is used that the X module is placed into the local module’s namespace and if you create an import loop that gets back to the import X statement again, then python accepts the partially initialized X module (what I have been calling a “forward reference”) - apparently because it finds X in the cache (of that namespace, I think). In other words, this module notices that X has already starting being initialized, and so it moves forward based on that partial initialization and therefore does not feel it needs to try to run module X again, but just moves on with the rest of the code in the same module, rather than creating an ImportError. This will allow a “simple” import loop to successfully complete - one where the loop completes all symbol definitions before they need to be used, called, or executed.

I also got some insight from the new type annotation functionality that I believe started in 3.7 where you can put a class name in quotes like it is a string for a type annotation, and a reference to that class is put into the namespace, and as long as that class/type is defined at some point in the namespace then IDEs are able to resolve the type and show errors and use “Intellisense” (i.e. VS Code) to show references to definitions, etc.

However, if you change the “import X … X.Y” syntax into the “from X import Y” type of syntax, the exact same import loop will fail with an ImportError.

Please see the simple code with A.py and B.py code files, with an import loop between them in my response to Barry Scott. The file with the import A and import B style imports resolves the references fine, despite the import loop. The files with from A import A style, with otherwise exactly the same code, generates an ImportError immediately, despite none of the code needing to execute any reference within the loop.

That is the main inconsistency I believe can be resolved so that any import loop that does not actually execute any of the symbol references within the import loop can resolve regardless of whether one uses “import X” and “X.Y” referencing, or chooses to use “from X import Y” style referencing.

7

Hi Steven,

I was concerned about terminology (i.e. my terminology being wrong) and perhaps should have added some definitions to a few terms I used, such as “forward reference” and “level 1 import loop”. I will do my best to clarify here.

Clarifying “Forward Reference”

With respect to “forward reference”, to me this means the code puts a “placeholder” for a definition into the current namespace, and moves forward with more code despite the reference not being complete yet. It does this with the hope that the symbols that have these placeholders will be fully defined before they are actually needed (i.e. before a class of that type or reference to a function is actually instantiated or called).

I heard the term “forward referencing” in discussions of type annotations, where using quotes around a type would create such a “forward reference” to that class or type, so the code could move on with the assumption that the type would be defined later. Since type annotations are typically not executed, but only used by an IDE for finding type errors and showing other references to an object by knowing a defined type for it, it doesn’t actually cause the code to fail even if the reference is never defined (in most cases).

I think Guido gave the correct term to this “forward referencing” with respect to imports by calling it “partial initialization” of a module in his response. It does appear that when you first use an “import X” statement, that a reference to X is put into the local namespace, then the X module is executed to fill in the details. If an import loop is created in X by another import statement back to the original module, it appears that Python notices that X has already been “partially initialized” and avoids calling the X module again, but just goes on to the rest of the code rather than executing X.py again, and also doesn’t create an ImportError due to the import loop. Instead, Python appears to take it on faith that the rest of module X will eventually be defined since it has been “partially initialized” already, and just goes on with the rest of the code. This assumption works out fine as long as you don’t try to execute any of the symbols in X before the import loop completes.

You can see the “import X” style code of A.py and B.py in my response to Barry Scott to see an import loop that resolves fine if you use “import X” style imports, but fails with an ImportError if you change them to “from X import Y” style imports.

What is a “level 1” import error/loop

I definitely should have defined this better. By “level 1” I mean an import loop that doesn’t call (or access, I think) any of symbols that are being defined within the loop. If you actually need to execute/access any symbol before it is defined, that would be more complex (i.e. Level 2 or maybe more?) and cannot be resolved by “forward referencing” or “partial initialization” or whatever you want to call it. i.e. you would have to reorganize your code in some fashion.

I think perhaps we could call the “level 1” type of import loop a “simple” import loop, or maybe a “reference only” import loop. This is where symbols are only defined as a part of the loop, but nothing within the loop has to be executed before it is defined. A “definition only” import loop could always be resolveable by Python (currently is for “import X”, but not for “from X import Y”).

I suppose the next most complex loop might have executions in the loop, but if the loop can be resolved by moving the definitions around so the code is in a different order, such that the symbols that need to be executed are always defined before they are needed, these “2nd level complexity” loops can still sometimes be resolved by ordering things correctly.

The most complex import loop would be where the nature of the loop itself dictates that the loop cannot be resolved by simply reordering the code - i.e. the execution of something in the loop occurs which requires executing something in the other part(s) of the loop such that both definitions are needed at the same time before the definitions can be completed.

Troubles with non-circular imports?

No, I’m not having troubles with non-circular imports. It’s only when an import loop is created - since “import X” type of loops resolve on their own in Python, but “from X import Y” type of loops do not, I see an inconsistency and if the loop is a “level 1” type that is resolvable only with “partial initialization” (or placeholder references/forward references, or whatever you want to call them), then I’d like to see Python also resolve such loops when using the “from X import Y” type of syntax.

When you talk about import loops, do you mean circular imports where module A import B which in turn imports A?

Yes, exactly.

Multiple import statements are reexecuted

Yes, thank you for the example. My wording was incorrect. When the import statement is reencountered you are correct that it is still executed, and, in fact, if the definition of that reference was set directly, per your code, then the next execution will reset it based on the import.

What I meant was that the second encounter of the import X statement will not attempt to call the X.py module again and reexecute the code there. Even in the case of an import loop where the X module has only been partially initialized, that partially initialized reference to X will be accepted (even if no other symbols in X have been defined yet) and rather than attempting to call X.py to import it, the reference in memory (in cache, per your comment) will be reused.

The important distinction here is that while import X creates a partially initialized reference to X that will be accepted from cache, when using from X import Y syntax instead, this same behavior DOES NOT occur. At least this is true in the case of an import loop where instead of accepting a partially initialized reference to the symbol Y, python will generate an ImportError.

Do I have a minimal reproducible example?

Yes, please see my response to Barry Scott. It has two versions of A.py and B.py code that creates a circular import loop. The version that uses “import A” and “A.A” references works fine, and completes the loop fine (it does not attempt to reimport B.py upon looping back to the first import B statement). But the version that uses the from A import A style fails with an ImportError as soon as the loop is created when from B import B is reencountered in A.py.

The progression of understanding python imports

I agree with your progression, except I would add another item before complex circular imports:

  1. Understanding namespaces, and how an import generates a reference to an external symbol in the local namespace.
  2. Caching and Reloading - though I’ve found this a bigger problem personally when using Jupyter Notebooks (because you don’t always restart the kernel) than with using straight scripts executed from the commandline (with some exceptions).
  3. Simple Circular Imports - this is the kind I am discussing here - what I called “level 1” imports - these are the kind where the loop only requires definitions of symbols to be resolved, and nothing inside the loop actually needs to access or execute the involved references.
  4. Complex circular imports - these are the kind where an import loop is created and one needs to actually execute the references, not just define them, while still in the loop.
    • As far as I’m aware, there is no method of successfully resolving complex import loop other than refactoring the code to literally remove the import loop.

The complex imports are quite a vexing problem. I’ve not yet found a way to create a standardized method of swimming against the stream - i.e. creating what I term (probably incorrectly) a reverse dependency - if you have a strict parent/child relationship to your modules/classes, then this “reverse dependency” is created when a child needs to reference a parent, it can also be created pretty easy when cross-cutting concerns - things that need to be globally accessed - attempt to access other classes rather than always being a child module/class compared to everything else.

However, that said, I believe that #3 - the simple circular imports - can be resolved by Python, and already are when using “import X” style syntax. However, Python does not resolve them when using “from X import Y” syntax, and that is what I would like to change, so we can focus on #4 rather than hacky ways to resolve #3 or completely avoiding the “from X import Y” syntax.

2 Likes
8

I need to come back and read you replies in details,
From a quick read I think you are asking for lazy importing?

Only resolve the Y in X.Y when you access it?

9

I know that people are happy what python allows circular imports.

Personally I avoid circular imports and suggest you do as well.

The large code base I work on as my dayjob used to have circular imports and that lead to a number of subtle and difficult to debug issues at runtime. We refactored to avoid the circular imports and the maintenance burden that it caused vanished.

Are you looking for changes to support code to work where circular imports would otherwise break that code?

10

Hi Barry,

I believe Python already delays the need to resolve Y in X.Y when it is accessed, if you just “import X”. Simple circular imports (ones where the references are only needed after they are fully defined, typically after the loop resolves) are already resolved by Python with this format.

However, if the “from X import Y” format is used, then Y must be immediately resolved, rather than having an ability to wait or delay until it is accessed. This causes an ImportError as soon as a circular import occurs, unlike what happens with “import X”.

So I’m simply asking for “from X import Y” to work the same way as “import X” already does with respect to circular imports, with the ability to set a placeholder for Y which will allow simple circular imports to resolve (if they can).

11

It is only a problem if you insist on have circular imports right?

Just refactor to remove the problem of circular import or put up with the maintenance burden that it inflicts on your code.

12

Correct. In part it is to ensure consistent behavior by Python for all import types. Having inconsistent behavior makes it very difficult to diagnose problems, and as a beginner/intermediate at Python to understand how the import system works.

13

I do not think that this is a problem that needs a fix.

1 Like
14

Think about this example:
a.py

import b
c = 5

b.py

import a
print(a.c)

This triggers the same error as the circular from a import c, for the same reason. If we had “forward references” like you’re imagining, what would this do?

This is almost completely right, except you’re missing one key point: this has nothing to do with the import system. Its just how Python attribute lookup works.

Whenever you have x.y, regardless of what x is, the attribute lookup is done when that line of code is run and not before.

Think about it this way:

class A:
    pass

a = A()

def b():
    print(a.c)

a.c = "c"
b()

a.c doesn’t exist when the function is defined, but the code works anyway - because the attribute lookup doesn’t happen until the function is called. If you comment out the assignment to a.c, or move it below the call to b(), the lookup will fail and you get an error.

And this is exactly what’s going on with the imports as well - like has been mentioned earlier in the thread from X import Y is just shorthand for import X; Y = X.Y; del X. It needs Y to exist because its doing the attribute lookup then and there. As you’ve noticed, when you do import X followed later by X.Y, its only doing the attribute lookup later - but because its an attribute lookup, not because X is imported.

2 Likes
15

You might be interested in PEP 690; discussion thread at PEP 690: Lazy Imports Again

The primary motivation is reducing startup cost for large applications by avoiding unnecessary imports, but it implements exactly the kind of “lazy placeholder” you are asking for here, so as a side effect it does eliminate most import cycle problems too.

3 Likes
16

Ivc,

Thanks for the explanation. Yes, I agree with your example - it’s showing that you can declare access to variables within a function definition and as long as they are defined by the time the function is run, you are good. This is equivalent to what happens in an import loop - as long as you don’t execute a symbol (it’s not accessed outside of a function definition) it is fine with a namespace import (import X), but not when directly imported with “from X import Y”.

The discussion at the end clarifies why it works like it does - thanks for pointing that out. i.e. from X import Y must, based on the current mechanism, create the object immediately (due to import X, Y = X.Y, del X behavior). It’s setting the definition of Y in the local namespace, and since Y is a symbol, rather than a namespace, there is no deferral of definition possible with how Python currently works.

Therefore, if a loop is involved, the symbol I’m asking for must be defined so that it is created in the local namespace, before the loop occurs. This is why the trick of moving the “from X import Y” to the end of the file will work in some cases - the loop doesn’t happen until after Y is defined.

So - what can be done? If anything?

So, Python would need a new mechanism to handle this to make the two types of imports equivalent. Essentially, you are declaring that the imported item is directly a part of your local namespace, which forces it to be “immediately instantiated” as an object or reference to an object (as a local variable with a value), so if the import loops without that item defined, looping would just create an infinite loop. i.e. there is no “partial initialization” possible where you can wait for the definition later.

Python would need to treat the current local namespace the same way it treats an imported namespace, which would essentially allow to declare a variable without setting it’s value to make the import statements equivalent, which would have other implications.

I can’t think of any benefit to generalizing this in a simple script (i.e. declaring that a symbol exists, but defining it later), but allowing it at least for the purpose of simple import loops would allow auto handling of unrolling such loops without doing hacky things like moving the “from X import Y” to the end of the file.

17

Hi Carl,

I will have to dig into that, but agree you may be right. Enabling lazy loading likely provides the mechanism I’m looking for by deferring most imports. I’m not clear if it addresses the “from X import Y” issue with Y being declared directly as a symbol immediately, on first glance.

It does appear it may provide new mechanisms for handling import loops architecturally though.

Thanks for the link!

18

I respect your opinion, but question whether you have considered the impact on new Python programmers.

If I take your side for the moment - I still think there is something missing here. Perhaps a documentation page on how importing works within the core Python documentation that specifically addresses the mechanisms under discussion here, including the practical differences between “import X” and “from X import Y”?

One can find some blogs and such addressing this, but there is a lot of confusion out there if you try to debug importing - just search some of the related Q&A’s on StackOverflow with people trying to solve import issues. I have lived through that the last few years, having great difficulty understanding that specific point until creating this discussion, with the help of yourself and the others responding here.

19

You may well be right that improved documentation could help here. But I’d note that as far as I am aware, you are the only person to have ever had this particular misunderstanding of what’s going on here, and it seems to be a pretty fundamental misunderstanding of Python’s execution model.

That’s fine, of course - lots of people pick up Python just by trying things out. And that can result in some quite - let’s say “unusual” - understandings of what’s going on, depending on the individual’s experience with other languages. But we can’t hope to address all of those potential misunderstandings. What we can do is document the actual execution model as clearly as possible, and point users at that when they have misunderstandings.

With that in mind, have you read (and understood) the language reference, specifically the execution model chapter and the section on naming and bindings? Also, the following chapter on the import system (although this is a bit “dense”, so I’d recommend skimming this to get an overview). If so, did it clarify things at all, and if not, what didn’t it explain for you? I’d caution that you seem to use a very different set of terminology than is normal in Python, so you should probably make a particular effort to ensure that you don’t get confused by terms you don’t understand.

But otherwise, I agree with @barry-scott - there’s no actual problem here to fix, just behaviours that differ from other languages that you’re used to.

2 Likes
20

Yes, I mentor a lot of new python programmers. If a new to python engineer got into the circular import hell I would mentor fixing the design to not need the circular importing.