PEP 810: Explicit lazy imports

I’d like to thank Pablo and the other PEP authors for writing and shepherding through this proposal. As mentioned in the PEP, the scientific Python ecosystem is a prominent example of lazy loading for libraries, as seen with our lazy_loader library which is fairly widely used (hat-tip to @effigies and others who helped maintain it).

Our goal was to enable lazy loading without downstream users being any the wiser. Mostly, that worked fine; we supported the standard import syntax and type checking (by parsing imports from .pyi files), and exposed lazy modules by using module-level __getattr__ in combination with LazyLoader (a strategy outlined in a 2018 blog post by Brett). However, I was always nervous about the fragility of the approach.

For example, under very specific circumstances doctests fail with lazy loading enabled. And at some point we ran into a race condition, which had to be addressed with a lock.

Communicating with Brett in 2021 (thank you for your helpful advice, Brett!), the conclusion at the time was that this was unlikely to ever become a standard language feature; and yet, here we are :rocket: Now, we can figure out how to retire lazy_loader and rely on a standard mechanism, while still having our libraries load fast. I am very appreciative of everyone who made this possible!

21 Likes