Question: Understanding imports a bit better - how are cycles avoided?

Python Help
2

The sequence of steps you’re looking for are:

  • import test
    • creates module object test
    • sets test.__path__ (among other things)
    • sets sys.modules['test'] to the new module
    • executes test/__init__.py, which encounters …
  • from test.foo import Foo (swapping to fully quilified import here)
    • finds sys.modules['test']
    • searches test.__path__ for search paths
    • finds test/foo.py
    • creates module object foo
    • sets sys.modules['test.foo'] to the new module
    • executes test/foo.py, which encounters …
  • class Foo
    • executes the body pass
    • creates the class object type("Foo", (), {})
    • assigns the class object to Foo
    • returns back to executing test/__init__.py
  • ... import Foo
    • gets Foo attribute of test.foo (or would try another import if it isn’t there)
    • assigns it to Foo in the current module (test)
  • __all__ = ["Foo"]
    • just an assignment of a literal
  • returns back to whoever imported test in the first place

So the cycles are mainly avoided by looking in sys.modules first when importing a module, and storing new modules in sys.modules before executing their code. So there’s absolutely a partial module in place when you are importing, but as long as anything you need has already been assigned (and __path__ is most common) then you can use it just fine.

I believe the error message you get warning about cyclic imports is actually a heuristic. The actual error will be AttributeError (or potentially NameError? I might be wrong on that one), but the importlib module will see the exception first. It knows that it’s in a recursive import situation, so it replaces the error with one saying that it’s probably due to a cycle, but sometimes it isn’t due to the cycle.

(e.g. pyzmq has/had a situation where a dynamic import would fail, and because it fails at a point where a submodule is importing a different submodule via the top-level module, the importer assumes it’s because the partial module doesn’t have the name yet. But it’s actually totally unrelated!)

1 Like