PEP 810: Explicit lazy imports

[thomas (Thomas Wouters)]

I’ve put up a PR to address some of the comments.

@cdce8p

I do wonder if it would make sense to add an eager soft keyword as well. It would certainly be an advanced feature but give library authors the ability to overwrite the global import global flag if they know an imports needs to happen eagerly. (Similarly __eager_modules__ could also be added.)

I’ve added a mention of this to the rejected ideas section. Adding eager would be confusing unless we were actively working to changing the default behaviour. We aren’t, and certainly as part of this PEP we won’t be.

@brettcannon

No ongoing performance penalty unlike importlib.util.LazyLoader.

This assertion is made twice in the PEP and it’s actually incorrect. LazyLoader replaces __class__ for the module so that __getattribute__ gets changed to ModuleType (or whatever loader.create_module() returns so there is no perpetual performance penalty:

I’ve removed the assertions about performance.

@nilslindemann

Therefore, better would have been:

• A transition period, from __future__ import lazy_import.
• After that, import X and from X import Y becomes lazy by default.
• For the cases where an immediate import is needed, the more verbose syntax: import now X and from X import now Y.

The default should have the shortest syntax.

Yes, it should. The default doesn’t change. This PEP does not anticipate the default will change. PEP 690 was the closest thing to a proposal that might lead to changing the default, and it was already rejected.

Making all imports lazy is a big change. There’s a reason it’s a knob the PEP provides but discourages. It’s a huge breaking change. There are too many modules that just won’t work right, or will work right only in specific circumstances (like something already imported one of its dependencies.) It may be doable in ten or fifteen years, after we’ve shown that lazy modules work great, and we’ve developed new patterns of module behaviour to replace the existing ones that rely on eager imports. Me, I’m not holding my breath.

@smontanaro

I’m guessing simply making all imports lazy (no new soft keyword) leads to breakage, or at least unexpected behavior? I only skimmed the rejected ideas, but didn’t see it as something you tried. I do see you reasoned about it.

The closest proposal to that would be PEP 690. It was rejected in part because it made all imports lazy. (It’s mentioned in this PEP.)

@DavidCEllis

Was it ever considered to use a context manager instead of new syntax?

• The previous PEP proposed adding an eager_imports manager to importlib as an opt-out so I was wondering if a lazy_imports manager was ever considered as the mechanism for an opt-in instead of new syntax.

That wouldn’t fix the issues with the existing context-manager based solutions. A hybrid where the syntax this PEP used is based on with would open a whole new world, since with has never been semantically important to the compiled code, which is quite a big lift. I’ve added it to rejected ideas, but if you want to propose a competing PEP with that syntax, feel free

@DavidCEllis

You briefly mention considering using a subclass of dict to enable lazy imports, but with the mention of calling __getitem__ directly I’m a little confused as to how it was consiidered.

• My expectation would have been that a __missing__ method could be used to check if an absent name was a lazy import, which would be supported by __getitem__, is there a reason this is not the case?

The names are there, we just need to something extra if they’re of a specific type. Or we could store them somewhere else, but then we would need to do multiple lookups and keep the dicts in sync. And either way, it still requires a dict subclass, which is problematic for all the reasons mentioned in the PEP.

@DavidCEllis

It’s possible to globally opt-out with -X lazy_imports="disabled", presumably this will cause a module import to fail if a lazy import has been used to hide a circular import?

• Might it be necessary to add a filter to opt-in for specific modules in the same way modules can opt-out of being lazily imported under -X lazy_imports="enabled"?

No, the filter intentionally is only called for potentially lazy imports. It’s meant as a safety measure, not an inversion of responsibility. And yes, it is intentional that disabling lazy imports globally would expose import cycles that only work with lazy imports. The advice for import cycles remains pretty much the same, even with lazy imports: refactor the code so you don’t have an unresolvable cycle.

@ncoghlan

@brettcannon

It makes finding the module lazy (LazyLoader is eager in finding the module but lazy for loading).

I think the PEP needs to justify this decision, as it’s what rules out using lazy imports for cheap existence checking.

I’ve added it to rejected ideas.