How about importing modules as strings

For example:

import "a.py"
import "E:/a/b.py"
import ".a/b.py"

This can import some modules that can’t be imported in the current method.

What would be the practical benefit of being able to import those modules, rather than simply renaming them to use valid Python names so they can be imported normally? These are really quite special cases and relatively easy to fix, and in general, you really shouldn’t be doing this in the first place unless you really need it, because it makes your code more complex, harder to understand and non-portable.

The first case can, of course, be imported via import a, or 62.5% fewer characters and greater clarity.

In the second and third case, it isn’t obvious what name the module should be mapped to. In the first, an absolute path is given, which begs the question of whether you just want to use the final path component as the module name, or if not, where along the path do you want to consider the top-level package directory. In the second, the import doesn’t work currently because the package .a is not a valid Python package name; thus, it is not obvious what import "a/b.py" should map to in the namespace.

However, you can already handle either case via importlib:

spec = importlib.util.spec_from_file_location("b", "./a/b.py")
module = importlib.util.module_from_spec(spec)
sys.modules["b"] = module
spec.loader.exec_module(module)

That’s rather more verbose, but its a special case, and there’s an issue to make it a simpler, and you can easily make it a utility function, i…e.

def import_path(path, name=None):
    if name is None:
        name = Path(path).stem
    spec = importlib.util.spec_from_file_location(name, path)
    module = importlib.util.module_from_spec(spec)
    sys.modules[name] = module
    spec.loader.exec_module(module)
    return module

which you could call as

import_path("E:/a/b.py")
import_path(".a/b.py")
import_path(".b.py", "b")

But again, you should understand and justify why specifically you need this instead of relying on standard import semantics.

1 Like

One of the most useful cases of “importing as the string” is the plugin-style GUI application (“plugin” is python script in this case). The features of load, reload, and unload can be useful because you can re-layout components, redefine functions, and debug without restarting the application.

Thanks for the nice code, @CAM-Gerlach.
I have two questions. (Sorry for being a little off topic)

  1. The document (31.5. importlib — The implementation of import — Python 3.6.15 documentation) says:

When this method exists, create_module() must be defined.

Though your code doesn’t use create_module, it works fine. Is it no problem? (Seems no problem.)

  1. The import_path seems to work as import or reload. But the module created by the import_path cannot be reloaded using importlib.reload. Is there a way to improve this?
>>> module = import_path(r"pylab\test1.py")
>>> reload(module)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Python39\lib\importlib\__init__.py", line 168, in reload
    raise ModuleNotFoundError(f"spec not found for the module {name!r}", name=name)
ModuleNotFoundError: spec not found for the module 'test1'

I’m still using the PY2-style like this, and I’d like to upgrade this to a PY3-style that exactly works in the same way.

def load_module(rootpath):
    name = os.path.basename(rootpath)
    if name.endswith(".py"):
        name,_ = os.path.splitext(name)
    
    ## Update the include path to load the module correctly.
    dirname = os.path.dirname(rootpath)
    if os.path.isdir(dirname):
        if dirname in sys.path:
            sys.path.remove(dirname)
        sys.path.insert(0, dirname)
    elif dirname:
        print("- No such directory {!r}".format(dirname))
        return False
    if name in sys.modules:
        module = reload(sys.modules[name])
    else:
        module = __import__(name, fromlist=[''])
    return module

If you look carefully at the importlib docs you linked, you can see it is a note in the exec_module method of the importlib.abc.Loader abstract base class, talking about the create_module() method of the same ABC, stating that custom Loader implementations must also implement the latter if they provide the former. This is entirely orthogonal to the code above, which is merely a wrapper function around the module-level importlib.create_module() function, and is certainly not a custom Loader implementation.

Maybe try passing reload sys.modules[module.__name__], to ensure it is reloading the actual object in sys.modules? Otherwise, I’m not sure what’s going on, unless the spec isn’t set up correctly somehow or it doesn’t store or use the actual __file__ location to reload it, and rather import_path needs to be run again instead (which should mostly have the desired effect of reload()). But this seems rather above my pay grade, as I"m not intimately familiar with the import system internals.

The importlib docs do warn:

It is generally not very useful to reload […]dynamically loaded modules.

I’m not sure if that’s referring to only extension modules, though.

Many thanks! @CAM-Gerlach
I understood that the document is about class inheritance.

I tried some, but no luck.

Yes, mostly, but my program depends on the reload feature:

importlib — The implementation of import — Python 3.10.4 documentation
When a module is reloaded, its dictionary (containing the module’s global variables) is retained. Redefinitions of names will override the old definitions, so this is generally not a problem. If the new version of a module does not define a name that was defined by the old version, the old definition remains. This feature can be used to the module’s advantage if it maintains a global table or cache of objects …

still no luck…

Don’t they love reload anymore? :slightly_smiling_face: