Lazy imports: how to use them in a CLI that supports <=3.15?

Python Help
1

I maintain a CLI tool that supports Python 3.11 through 3.15 and I’m interested in implementing the new lazy import feature.

How do I go about updating the workaround pattern, which uses a local import in the function call?

import click


@click.command()
@click.argument("size", type=int, default=5)
def cli(size):
    # workaround local import
    import numpy as np

    data = np.arange(size)
    click.echo(f"Generated array: {data}")


if __name__ == "__main__":
    cli()

Per the docs, for backwards compatibility I should use __lazy_modules__ like the following. But won’t that cause unwanted eager imports in <3.15?

__lazy_modules__ = ["numpy"]

import click  # eagerly imported is OK
import numpy as np  # lazily imported by 3.15, BUT eagerly imported by <3.15


def cli(size):
    # import numpy as np  # remove the local import?
    ...

Is it recommended to use something like if sys.version_info.minor < 15: ... ?

Any suggestions would be much appreciated!

2

You can’t really make use of the new lazy imports in this context without making the import eager under older Python as you expect. The __lazy_modules__ attribute is more useful for the cases where imports weren’t previously deferred and can be made so under Python 3.15, making 3.15 even faster (one way to encourage people to upgrade :slight_smile: ).

There are other patterns for handling lazy imports under older Python you could use under versioned conditional statements like objects that replace themselves with the module on first import and return attributes, but these may be more awkward than what you’re doing now.

3

Click already tries to avoid unneeded imports. We’ll add lazy imports to a release soon. That will benefit anybody using 315 and will look the same to previous versions.

4

@davidism good to know, thanks for the heads up!

My question wasn’t meant to be about click specifically, so it still stands for a pure standard library CLI. For example:

import argparse


def cli(size: int):
    import numpy as np  # workaround local import

    data = np.arange(size)
    print(f"Generated array: {data}")


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "size",
        nargs="?",
        type=int,
        default=5,
        help="Array size (default: %(default)s)",
    )

    args = parser.parse_args()
    cli(args.size)

@DavidCEllis The way to implement the alternative you linked to isn’t immediately apparent to me. Is there a way to do this that’s performant across 3.11-15, even if the code is a little bit ugly?

5

@p-vdp Essentially for a one-off import you’re making a single use class. Converting the example from using annotationlib to using numpy it would look something like this:

__lazy_modules__ = ["numpy"]

import sys

if sys.version_info >= (3, 15):
    import numpy as np
else:
    class _LazyNumpy:
        def __getattr__(self, attr):
            global np
            import numpy as np
            return getattr(np, attr)
    np = _LazyNumpy()

Under 3.14 and earlier this gives you a _LazyNumpy object, when you do np.arange it will then trigger this __getattr__ which will import numpy, replace itself in the module so np now actually refers to the module, and return the arange attribute from numpy. In most cases the first thing you’ll do with a module is access a regular attribute, so this works.

Under 3.15 this is a normal lazy import.

6 
import functools

@functools.cache
def import_numpy():
    import numpy
    return numpy


def cli(size: int):
    np = import_numpy()
    data = np.arange(size)
    print(f"Generated array: {data}")

This makes sense if you are using the dependency in multiple places.
If it is just one, I’d go with the inline import.

I’ve left out the backflips you’d need to do if you want type checking, since (as is) this requires the type checker to do return type inference

7

Thanks for the ideas.

I hoped I didn’t have to do something hacky given the hype around lazy imports in the CLI use case plus the promise of backwards compatibility. Maybe this is a corner case, but I expect others will run into it as lazy imports become common. Some suggestions in the docs might be nice at some point.

8

You don’t have to do anything hacky. It is backwards compatible, with the __lazy_modules__ list instead of the lazy import keyword. Compatible doesn’t mean you get the benefits on older versions, just that it remains parseable/runnable. If you want lazy imports on older versions, it’s no more hacky than what you already had to do. So you can choose to use the existing hacks, or use the new feature and only get the benefits on newer versions, or do both.

9

Of course not, but happy to hack in this case :rofl:

As an aside, I couldn’t find any open issues or PRs about the upcoming click functionality you mentioned. Will that allow me to achieve this in a cleaner way?