PEP 810: Explicit lazy imports

PEPs
436

This is my main and only point. The rest is to indicate why the benefits are not big enough to not take precautionary measures / extra effort to minimise / eliminate this risk.

So in more general space of deferred evaluation there are roughly 4 levels of explicitness (from most to least):

  1. Explicit proxy object - everything is explicit here (think dask evaluation graph)
  2. Mutation - still no implicit substitutions, but class changes shape (think importlib.LazyLoader)
  3. Object is replaced implicitly on a chosen set of operations - still not completely implicit. (This was thoroughly explored and implemented in Backquotes for deferred expression)
  4. Completely implicit namespace variable substitution on any reference. This was one of the initial starting points for “Deferred evaluation”. (https://github.com/DavidMertz/peps/blob/master/pep-9999.rst hinted at this to high degree.)

Approach (4) is the most implicit black magic there is in this area and there was non-trivial amount of effort to explore more explicit alternatives, because for general “deferred evaluation” concept it is simply too implicit given Python’s philosophy and the way things are.

And this, although named “Explicitly lazy imports”, is using the most implicit method there is in this space.

So what I am saying is at he very least, this should be given more time for things to come together.

So my only problem is with the “implicit design of variable substitution”.
Other parts of this I have nothing against:

  1. Syntax - lazy is a good keyword - it is my favourite - defer and others I don’t fancy that much.
  2. Import specific things seem well thought out. I am quite confident that it does deliver “explicit-granular-…” promise. There were comments that it could be made simpler, and maybe it could be subject to couple more step backs, but if it goes as it is - I don’t mind - not my area here, so I don’t even have any significant opinion.

I never followed “lazy imports” because I never thought that it will drop the bomb of such implementation. I just assumed that it will do something along the lines of “explicit proxy / mutation-LazyLoader-like” and it will be nicely orthogonal and non overlapping.

But it turned out differently and the pace of this doesn’t make it easy to jump in and raise high level considerations.

So to get back to implicitness.

This will not be equivalent:

try:
    ref = lazy_import('non_existent_package')
except:
    sys.exit(1)

Currently (with LazyLoader, it will terminate early, with new lazy import numpy, it always runs without an error.

And this is another point, what is the point of “import specific” implementation if it doesn’t make any importing assurances? The current behaviour is much more explicit and useful.

lazy import does_not_exist

success…

I agree that implicit handling will not choke on any cases. But that is what “completely implicit” is - it passes silently on anything. But I would argue that it should better choke a bit and let packages and ecosystem adapt to “non choking” way instead, while slowly widening “non choking” space to its maximum possible.

There is very little information on drawbacks of LazyImports in PEP 810 – Explicit lazy imports | peps.python.org. Would you mind giving more information?

  1. What cases does it not work on?
  2. Could you give some examples where LazyImports fail to deliver what this proposal manages well?

This is my challenge:

List the number of cases, where LazyLoader fails, and if I found a way to make them work, you would seriously reconsider adjusting the approach.

1 Like