PEP 810: Explicit lazy imports

PEPs
112

Kudos to the PEP 810 team for writing this PEP. As the Sponsor and an advocate for PEP 690, I think you did a really good job at addressing the major concerns leading to the rejection of 690. Let’s hope you’ve struck a winning balance this time.

Here’s my personal feedback on the PEP after having read it and at least trying to do a good faith review of this thread. I’ll just assume that anything you ignore has already been covered.

About the filter function

The following questions come to mind regarding the filter function.

  • Is it guaranteed to be called in a thread-safe manner? Being a process-global function (since it’s currently specified to be set/get by a sys function, but more on that below), are there any special considerations that are needed, either from the interpreter or from the author of the filter function? The section on thead-safety doesn’t address this directly, but I assume at least the intent is that yes, it is thread safe. That section could add an explicit guarantee.
  • The FAQ about performance does not address any overhead of calling the filter function, whether the performance overhead is different if a filter function is defined or not, and whether there are any optimizations when no filter function is defined, which I expect to be the norm. The section in the PEP should also be clearer when the filter function is called. I read it as saying it’s called at the point of the lazy import, not at reification time. ISTM calling a filter function has to impose some performance hit, let alone any logic inside the (user defined) filter function itself, so I’d like some discussion about that in the PEP either way.

Alternatives

  • I’m personally fine with proxy approach instead of the subclass-of-dict. I thought 690’s use of a modified dict was clever, but the proxy approach proposed in 810 is a better, more explicit solution, without the potentially global unintended side-effects. I don’t think a subclass gives any benefit.
  • I’m fine with lazy as the keyword, with defer being a decent second choice.
  • I’m also fine with the lazy from form. I actually think it’s a good thing that lazy imports of either stripe always start with the word lazy. I think that’s a better choice than mixing lazy import and from lazy import. (I also don’t think it’s worth any syntax deprecation to make from . lazy import work as you’d want it to.)

Other thoughts before the bikeshedding begins

  • In the migration FAQ, I think you should mention -X importtime as a way to help “identify slow-loading modules”, and perhaps give references to other profiling tools. Eventually, this could use a devguide or howto treatment.
  • In the rejection of from lazy section, you say “This is because from . lazy import bar is legal syntax (because whitespace does not matter)”. I suggest you make this more explicit: "this is because from . lazy import bar is legal syntax, and is equivalent to from .lazy import bar.
  • “Why you choose lazy as the keyword name” isn’t grammatically correct. Perhaps it’s missing a “did”?
  • I get that module.__dict__ will reify any lazy imports, but I’m somewhat concerned about the readability of seeing bare module.__dict__ in code. Would it not be better to add a .reify() method to module objects to make that explicit? It could no-op if it’s already reified.
  • The FAQ talks about the ability to remove if TYPE_CHECKING guards with lazy imports, which is great, but it does make it a little more difficult to see at a glance which names are imported just for the non-runtime type checker. Still, as I’ve been a fan of a TYPE_CHECKING built-in (to avoid the need to import it from typing), I think this is probably an overall win. What I am missing is a discussion about this PEP’s effect on static type checkers. Will they have to worry about every import rather than just the ones in a if TYPE_CHECKING block? It’s a naive question because I don’t actually know whether type checkers look for those blocks.
  • Along those lines, would it make sense to always treat imports in if TYPE_CHECKING blocks as lazy? This, along with TYPE_CHECKING as a built-in could make for a very easy migration to lazily importing all names that are imported just to satisfy static type checkers. I kind of also wish there were just lazy blocks, e.g.:
 as lazy:
    import foo
    from bar import baz

Bikeshedding FTW!

  • __lazy_modules__ → __lazy_imports__ to more closely mirror the lazy import syntax and -X flag.
  • Shouldn’t set_lazy_imports_filter() and get_lazy_imports_filter() live in importlib instead of sys?
  • -X lazy_imports=<modes> mode names: default → normal, enabled → all, disabled → none.

I think that’s it for now!

24 Likes