Inconsistent behavior in "normal" vs dynamic import via importlib?

Python Help
1

I am seeing a weird behavior with importlib on Python 3.12.12 and 3.14.3 that seems inconsistent between “normal” and dynamic imports. (just tried 3.14.3, got same behavior)

I have the following filesystem structure within a top-level directory (c:/tmp/bugs/python/importhell, not that this matters)

- hmm.py
- test_in
  - phobia
    - agoraphobia.py

and agoraphobia.py contains the single line QQ = 12345.

If I run Python from within the test_in directory, everything works just fine and dandy:

C:\tmp\bugs\python\importhell\test_in> python
Python 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:13:34) [MSC v.1944 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import phobia.agoraphobia
>>> phobia.agoraphobia.QQ
12345
>>> import importlib
>>> importlib.reload(phobia.agoraphobia)
<module 'phobia.agoraphobia' from 'C:\\tmp\\bugs\\python\\importhell\\test_in\\phobia\\agoraphobia.py'>
>>>

But if I do it dynamically, I run into a problem on reload where the parent module is missing, and it expects to find phobia/__init__.py . Here is the contents of hmm.py:

import argparse
import importlib.util
import os
import sys

def get_module(qualname, rootdir):
    module = sys.modules.get(qualname)
    if module is None:
        modulepath = os.path.join(rootdir, '/'.join(qualname.split('.'))+'.py')
        print("Importing %s from %s" % (qualname, modulepath))
        spec = importlib.util.spec_from_file_location(qualname, modulepath)
        module = importlib.util.module_from_spec(spec)
        sys.modules[qualname] = module
        spec.loader.exec_module(module)
    else:
        importlib.reload(module)
    return module

def doit(args):
    for run in range(2):
        print("Run %d" % run)
        module = get_module(args.module, args.rootdir)
        value = getattr(module, args.name)
        print('%s from %s = %s' % (args.name, args.module, value))

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('rootdir')
    parser.add_argument('module')
    parser.add_argument('name')
    args = parser.parse_args()
    doit(args)

and when I run it, I get this:

C:\tmp\bugs\python\importhell> python hmm.py test_in phobia.agoraphobia QQ
Run 0
Importing phobia.agoraphobia from test_in\phobia/agoraphobia.py
QQ from phobia.agoraphobia = 12345
Run 1
Traceback (most recent call last):
  File "C:\tmp\bugs\python\importhell\hmm.py", line 32, in <module>
    doit(args)
  File "C:\tmp\bugs\python\importhell\hmm.py", line 22, in doit
    module = get_module(args.module, args.rootdir)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\tmp\bugs\python\importhell\hmm.py", line 16, in get_module
    importlib.reload(module)
  File "C:\Users\{redacted}\.conda\envs\py3datalab\Lib\importlib\__init__.py", line 121, in reload
    raise ImportError(f"parent {parent_name!r} not in sys.modules",
ImportError: parent 'phobia' not in sys.modules

Why is importlib.reload creating additional requirements to be successful, as compared to a normal static import, and an initial dynamic import?

Just for some context: what I am trying to do is to load modules dynamically from files in a path “elsewhere” and process them; I don’t want that path to be placed in sys.path and would like to load those modules with the least impact possible on the Python runtime. These files are from authors I trust; I’m not trying to guard against unsafe code from an adversary, so I’m ok with the modules being executed as a module.

1 Like
2

General question: have you looked into whether runpy.run_path might be a better fit for your problem than dynamic imports? (since you’ve indicated you just want to run the code, rather than specifically caring about importing it).

As far as the original question goes, the dynamic import emulation isn’t populating the implicit namespace packages needed to properly locate the subpackage in the module hierarchy. The emulation itself is bypassing the check for those packages (since it is only modifying the modules cache, not the attribute in the parent module), but the reload is using the full import system and hence encountering its internal consistency checks (which fail, because the parent module is missing)

1 Like
3

Well, I don’t actually want to run the code, I want to import it *as a module* that I can analyze with ast and other techniques.

1 Like
4

Importing a module also runs it in order to populate the module namespace.

If you just want to perform static analysis on the source code without executing it, you’ll typically want ast.parse or the compile built-in (and occasionally the tokenize module for some checks that care more about syntactic details)

2 Likes
5

OK, I should have said, I don’t actually want to run the code to execute it to accomplish any side effects, I want to run the code to define functions and classes for subsequent analysis; I have reasons to augment static analysis with being able to run decorators and examine class object attributes.

6

runpy.run_path will do all those things without touching any global state (assuming you’re executing regular scripts, and not a directory or zip archive). What it can’t do is run code that is actively expecting to be imported as a module (uses relative imports, looks for itself in sys.modules, etc).

For the latter case, importlib.import_module does everything a regular import statement would do (including importing parent packages as necessary), but making it look for modules in locations that aren’t on sys.path is tricky (assuming making temporary sys.path modifications isn’t an option).

Otherwise, you do need a custom import function like the one you have shown, it just needs to be enhanced to include the missing parent module imports and modifications shown in the documentation’s importlib.import_module emulation example. Exactly how that would need to work in your case will depend on whether or not you want to execute the __init__.py files in parent folders or not (keeping in mind that some submodules assume their parent module has been fully imported before they execute).

1 Like