PEP 810: Explicit lazy imports

PEPs
21

Adding lazy as a soft keyword does not necessarily preclude it from being used in other places afaiu.

2 Likes
22

Could async work as the keyword instead of lazy?

The module is conceptually being loaded in the background eventually. And you could then await it if you want the import to have happened now. That then allows something like:

async import numpy
def meth():
try:
await numpy
except ImportError:
# ... whatever

23

That would make it way, way way more confusing, not only for those new to Python, but for everyone.

Yes, the task of importing the module is not being performed right away, but that’s not the same as awaiting an async object.

A possible way to instead reduce the number of keywords we’d need to add for this, is a built-in lazy_import function, so one could do:

np = lazy_import("numpy")
24

Is there any way to check if an import will succeed but still keep the import mostly lazy?

What I want is something like:

try:
    lazy import numpy
    using_numpy = True
except ImportError:
    using_numpy = False

# Here numpy is still only lazily imported

Obviously this requires some part of the import machinery to at least access the filesystem so it could not be a completely lazy import.

25

One question I would like to ask, is why the returned object from __getattr__ always invokes loading the module. If I were to have a layout like this:

pkg
|- sub_a
|  |- __init__.py
|  |- something.py <-- Quite small
|- sub_b
|  |- __init__.py
|  |- something_else.py <-- Very big

If I do lazy import pkg, and then I do some_attr = pkg.sub_a.something.attr, if I understand the PEP correctly (perhaps I missed something?), all submodules are now loaded. That means that I’ll load sub_a/__init__.py and sub_a/something.py but I also load sub_b/__init__.py and sub_b/something_else.py.

So if I want to use some feature, I instantly load all submodules? Or is that for the pkg/__init__.py’s __getattr__ (if implemented) to decide?

26

This would work, I think:

import importlib.util

if importlib.util.find_spec("numpy") is not None:
    lazy import numpy
    using_numpy = True
else:
    using_numpy = False

It’s not perfect, but I personally find it difficult to imagine this lazy import syntax intentionally doing any filesystem access during the import statement (at least, I’d prefer it doesn’t).

27

Unfortunately this won’t work. This is a syntax error that’s covered in the PEP:

1 Like
28

“eager” and “explicit” do not mean the same thing to me. “eager” would read to me as the same as “disabled”, not “default”:

  • "default" (or unset): Only explicitly marked lazy imports are lazy
  • "enabled": All module-level imports (except in try or with blocks and import *) become potentially lazy
  • "disabled": No imports are lazy, even those explicitly marked with lazy keyword
2 Likes
29

This would be up to pkg to decide…if importing pkg always imports sub_b then you can’t really get away from doing it. But pkg could make its own internal imports lazy. Or it might not import sub_b in the top namespace at all–it depends on how that package is designed.

Put another way–right now, import pkg.sub_a doesn’t import sub_b unless that’s explicitly done in pkg.__init__. Lazily importing sub_a would be the same situation.

30

The dedicated syntax has a few advantages. For starters it works better with from imports - you could indeed make this work but it’s getting pretty clunky and there’s a lot of unnecessary redudancy (e.g. foo, bar, baz = lazy_import(“numpy”, (“foo”, “bar”, baz”))

It’s also just a lot more obvious what’s going on - if it’s a function you might be wondering what lazy_import is, is it some new built-in? It it something that’s been imported? And how do type checkers treat this?

And finally it just aligns better with a mix of other things which are being imported normally.

It’s also probably worth noting that lazy is just a soft keyword, so it doesn’t impact the ability to use it elsewhere.

The motivation section also calls out that there are some non-syntax related proposals that have downsides as well (https://pep-previews–4622.org.readthedocs.build/pep-0810/#motivation)

31

Agreed, for a proper lazy import you would certainly want to avoid all filesystem access.

I was just building on the example above that if you are checking try/except ImportError then in my experience you want to know if the import would succeed but often do not want to incur the full cost of the import so it is a different kind of lazy import.

Your find_spec suggestion seems to work though without any significant overhead over just running an empty file. Using find_spec doesn’t actually need the lazy import mechanism here (which only makes sense for deferring imports that are assumed to succeed).

32

Would lazy from __future__ import feature mean anything special? (If not, imho it should be a SyntaxError.)

1 Like
33

I really like the idea, but also have a suggestion.

The expected performance improvement should be more explored in more detail in the PEP write up.

Going through it, the expected performance improvement is too abstract, which should not be the case when it is the MAIN selling point. Some explicit results for performance benchmarks/tests or standard libraries/packages (with names) would be a good step in showcasing the upside of this recommendation.

34

It should raise SyntaxError indeed. We will add it to the PEP, thanks!

1 Like
35

Going through it, the expected performance improvement is too abstract, which should not be the case when it is the MAIN selling point. Some explicit results for performance benchmarks/tests or standard libraries/packages (with names) would be a good step in showcasing the upside of this recommendation.

The PEP already covers this and has what you are asking. It’s in the FAQ section. Here is the pyperformance run: free-threading-benchmarking/results/bm-20250922-3.15.0a0-27836e5/bm-20250922-vultr-x86_64-DinoV-lazy_imports-3.15.0a0-27836e5-vs-base.svg at main · facebookexperimental/free-threading-benchmarking · GitHub

1 Like
36

Thanks to the (many!) authors for the PEP!

I do have a slight concern with the ‘global lazy imports flag’ (-X lazy_imports=...), as it is the part of the PEP which seems to me to stand in opposition to the explicit aspect of the new lazy import ... syntax. I worry that this will become a hidden secret hack ™ that will proliferate as a way to easily increase performance with zero other changes, etc. This means that I as a libary author would potentially be liable for an influx of bug reports that my library doesn’t work with lazy imports, even though I haven’t tested for or intend to use them.

Would it be going to far to make this an opt-in build time (configure) option, or add strong language in the docs warning against using the flag for ordinary users?

A

37

I do have a slight concern with the ‘global lazy imports flag’ (-X lazy_imports=...)

It’s explicitly an advanced/opt-in mode (as well as the opt out). Default behavior stays fully explicit; nothing becomes lazy unless you ask for it. We will surely brand it as such: an advanced feature that has a lot of potential for a subset of users but in the PEP it’s important but secondary.

… which seems to me to stand in opposition to the explicit aspect of the new lazy import ... syntax…

We’ll brand and document it as an advanced switch, not a routine performance knob. It’s advertised, not hidden, and the docs will stress what is the usage mode and when not to use it.

… I worry that this will become a hidden secret hack ™ that will proliferate as a….

Docs will make the trade-offs clear: if you enable the global mode, you’re expected to use the filter and own exclusions. It’s fine for maintainers to close reports that only fail under -X lazy_imports=enabled without a minimal repro using explicit lazy. This is not really a problem: maintainers can simply decide what they support. If users open issues saying “your library could support this better,” that’s just information: you can close it if you don’t want to invest, or act on it if you do. As a maintainer myself, I don’t see harm in users surfacing that feedback, and we’ll make sure the docs hammer home that enabling the flag means you accept those trade-offs.

… Would it be going to far to make this an opt-in build time (configure) option…

We’ll include strong warnings and guidance. A build-time gate it’s possible but honestly feels unnecessary (we don’t prevent users by default to have access to ctypes just because we think is advanced and potentially dangerous). Not having it would hinder the real target users with large deployments (Meta, HRT, Google, and many others) that want to enable it broadly and tame it via the filter to realize startup/memory wins.

“What about support and tooling?” (bonus) :slight_smile:

We’ll ship examples and rollout recipes, and there’s incoming open-source tooling that Meta is using to help generate/manage filters so users can safely exclude modules with import-time side effects. This will build on the PEP but is out of scope of the discussion.

2 Likes
38

Thanks Pablo, you even answered a question I didn’t think to ask!

A

39

You gave the the perfect segway into talking about that :slight_smile:

2 Likes
40

I think another important aspect here is that there is a real desire for the global enabling (and disabling, for different reasons), and if we don’t build it in users will likely build hacky solutions on top of whatever part of the PEP we do accept (for instance, replacing __import__ with something that sometimes calls __lazy_import__) – but those would then be easy to get wrong, and we wouldn’t be in a good position to document trade-offs and provide guidance or tooling.

1 Like