PEP 810: Explicit lazy imports

[pablogsal (Pablo Galindo Salgado)]

@hprodh

Probably a lazy imported object can have a __lazy__ attribute, explicitly centralizing an interface with the proxy. I can foresee a bunch of uses now :

lazy import mymodule

mymodule.__lazy__.check()  # perform any kind of quick tests to check whether the import system will succeed to import (module init errors apart) and return boolean if yes
mymodule.__lazy__.get_path()  # seek and return path to module root file or folder, like importlib.util.find_spec
mymodule.__lazy__.raw_dict()  # seek the __raw_dict__ by parsing
mymodule.__lazy__.reify()

hasattr(mymodule, '__lazy__')  # check if mymodule was reified

Notice the hypothetical __raw_dict__ only makes sense across module boundaries when inspecting module objects:

# x.py

lazy import a
lazy import b

def foo(): ...

# y.py

import x

# If we do here x.__dict__ then both x.a and x.b will be reified so we
# need some way to know that they are there but without triggering the
# reification

print(f"names in x are: {x.__raw_dict__.keys()}")

@encukou

I appreciate that you’ve discussed this a lot, and studied the effects on large codebases, but… could __dict__ itself stay raw?

We are not closed to this if people agree that this is the best option. It means we are technically exposing lazy objects a bit more, also outside the module. Apart from the options highlighted in the PEP we didn’t went with this option initially so is also a bit easier to activate global mode, but I am happy to reevaluate it as its another option to the __raw_dict__ conundrum. I think @brittanyrey may have more info on how much this matters in practice in the wild.

@encukou

Imdividual objects? Also yes. If there’s no free function for get(), IMO the method should be made into a dunder that’s a no-op on real module objects (__reified__?). Otherwise you always need an isinstance check before calling it:

# action at a distance -- perhaps in another module, maybe in usercustomize for REPLs
import json

...  # ... some time later ...

lazy import json
json.get()  # AttributeError!

Notice the get() method only happens if you somehow stumped into a proxy object via globals() the new hypothetical __raw_dict__ or __dict__ if we use your proposal.

>>> lazy import exampl
>>> print(globals()['example'])
<lazy_import 'example'>
>>> print(globals()['example'].get())
<module 'example' from '/Users/pgalindo3/github/python/main/example.py'>

Also note that any attempt to use will cause it to reify so its really hard to ever see a lazy object:

# lel.py
print("FORTRAN 77 ROCKS")

def foo(): ...

then:

>>> lazy import lel
>>> x = globals()['lel']
>>> print(x)
FORTRAN 77 ROCKS
<module 'lel' from '/Users/pgalindo3/github/python/main/lel.py'>

or

>>> lazy from lel import foo
>>>
>>> x = globals()['foo']
>>> def print_type(x):
...     print(type(x))
...
>>> print_type(x)
FORTRAN 77 ROCKS
<class 'function'>

Perhaps this can motivate why trying to reify on module.__dict__ is worth doing and exposing __raw_dict__ or some free function that does that: it “hides” the lazy objects more. Not that they are a problem if they appear (that’s fine!) but is nice that is rare for them to show up.