Complexity (especially when debugging failed imports) was certainly a key theme in the PEP 690 rejection.
One of the things I’m aiming for in this thread is to avoid changing the consumption side complexity: deferred imports are resolved the same way they are now, which is via function level import statements. Even the initial version of the await import ... idea just combined a regular import with asyncio.to_thread.
I still think there’s a potentially useful meaning that can be given to async import ..., which would be to allow the interpreter to compile an import time map of runtime import dependencies, without having to fully resolve those dependencies at import time. There wouldn’t be any effect on either sys.modules or the importing namespace at the time the deferred import notification is submitted.
This is different from the way existing meta path based lazy importers work, as those need to come up with something to stick in sys.modules and the importing namespace, which is where all the lazy attribute resolution magic that got PEP 690 rejected comes into play.
The potential pieces for a useful enhancement that I’m currently seeing would be to add the machinery described below, so an application could resolve deferred imports at the time of its choosing by calling importlib.util.resolve_deferred_imports(). Alternatively, it could resolve deferred imports for a subset of eagerly imported modules by giving the name of the starting point, or do its own thing (such as resolving deferred imports in a thread pool) by iterating over importlib.util.iter_deferred_imports() directly.
I’m still not sure this would be worth it, but it does offer a potential approach that avoids the problems that got PEP 690 rejected, while still allowing modules to declare the module dependencies that they don’t need at import time, but will need at runtime.
importlib.util.import_module_from_spec(spec)
This would be similar to importlib.import_module, but accept a full ModuleSpec instance instead of just the module name. Both checks and updates sys.modules (unlike importlib.util.module_from_spec), while fully respecting the module import locking thread synchronisation machinery (unlike the example code in the importlib docs). Raises a new ModuleSpecConflictError subclass of ImportError if the module is already present in sys.modules with conflicting __spec__ details.
importlib.util.defer_runtime_import(name, package=None, importing_name=None)
# Mapping from deferred runtime imports to their specs
_deferred_imports: dict[str, ModuleSpec] = {}
# Mapping from deferred runtime imports to the modules requesting them
_deferred_by_target: dict[str, set[str]] = {}
# Mapping from module names to their deferred runtime imports
_deferred_by_importer: dict[str, set[str]] = {}
def defer_runtime_import(name, package=None, importing_name=None):
"""Register an expected future runtime import"""
module = sys.modules.get(name)
if module is not None:
# Already imported (or is being imported)
return module.__spec__
mod_spec = _deferred_imports.get(name)
if mod_spec is None:
mod_spec = importlib.util.find_spec(name, package)
if mod_spec is None:
# Eagerly report missing dependencies
raise ImportError(...)
_deferred_imports[name] = mod_spec
if importing_name is not None:
_deferred_by_target.setdefault(name, set()).add(importing_name)
_deferred_by_importer.setdefault(importing_name, set()).add(name)
return mod_spec
importlib.util.iter_deferred_imports(importing_name=None)
def iter_deferred_imports(importing_name=None):
"""Iterate over the details of registered deferred runtime imports"""
if importing_name is None:
mod_names = tuple(_deferred_imports)
else:
mod_names = tuple(_deferred_by_importer.get(importing_name, ())
for name in mod_names:
mod_spec = _deferred_imports[mod_name]
importing_names = _deferred_by_target.get(mod_name, ())
yield mod_name, mod_spec, tuple(importing_names)
importlib.util.resolve_deferred_import(name)
def resolve_deferred_import(name):
"""Resolve a registered deferred import"""
mod_spec = _deferred_imports.get(name, None)
if mod_spec is None:
# Let the caller decide if this is an error or not
return None
module = import_module_from_spec(mod_spec)
del _deferred_imports[name]
importing_names = _deferred_by_target.pop(name, ())
for importing_name in importing_names:
targets = _deferred_by_importer.get(importing_name, None)
if targets is None:
continue
targets.remove(name)
return module
importlib.util.resolve_deferred_imports(importing_name=None)
def resolve_deferred_imports(importing_name=None):
deferred_imports = iter_deferred_imports(importing_name)
for mod_name, __, __ in deferred_imports:
resolved_deferred_import(mod_name)