PEP 810: Explicit lazy imports

[thomas (Thomas Wouters)]

@barry

The following questions come to mind regarding the filter function.

• Is it guaranteed to be called in a thread-safe manner? Being a process-global function (since it’s currently specified to be set/get by a sys function, but more on that below), are there any special considerations that are needed, either from the interpreter or from the author of the filter function? The section on thead-safety doesn’t address this directly, but I assume at least the intent is that yes, it is thread safe. That section could add an explicit guarantee.

What does “calling in a thread-safe manner” mean? Do you just mean “it’s guaranteed not to be called from multiple threads”, or “concurrently from multiple threads”, or “concurrently from multiple threads for the same module”? It’s not, for any of these. There’s no locking. It’s just a function that’s called as part of the import mechanism, just like many other functions. An implementation that does thread-unsafe things has to perform the relevant synchronisation itself. I’m not sure it’s ever going to matter, really. I mean, the simple case is a dict or set lookup, which is not a problem. If you need to do a lookup in a container that doesn’t do thread-safe lookups, or if you need to do complex shared state manipulations in the filter function, you’ll have to lock accordingly. None of this is specific to this proposal, and I think trying to spell it out every time sets a bad precedent and a bad example.

@barry

“Why you choose lazy as the keyword name” isn’t grammatically correct. Perhaps it’s missing a “did”?

… but you’re okay with the answer?! (FWIW, it’s a meme Pablo likes.)

@barry

I get that module.__dict__ will reify any lazy imports, but I’m somewhat concerned about the readability of seeing bare module.__dict__ in code. Would it not be better to add a .reify() method to module objects to make that explicit? It could no-op if it’s already reified.

The mechanism isn’t being advertised as the way to reify all values in a module, it’s just something that’s done to keep existing code behaving as expected. Perhaps we should have an explicit way to reify everything in a module, it hasn’t really come up yet.

@barry

__lazy_modules__ → __lazy_imports__ to more closely mirror the lazy import syntax and -X flag.

I think __lazy_imports__ is too close to __lazy_import__, and also doesn’t convey that it lists modules.

@oscarbenjamin

Is it reasonable for a user of a library to expect that the library functions under -X lazy_imports="disabled"?

(Replying to this one as a stand-in about the suggestion to allow the lazy imports filter to bypass the global disable.)

I think no, the library not working should be fine. The global disable isn’t meant to be a general purpose tool with common use. It’s there for the convenience of people who want to test their own modules with lazy imports disabled. I also think it’s fine for that disable to not be an option for libraries that don’t want it to be an option, and I don’t think it’s a good idea to have a complex hierarchy of enables/disables like a game of top trumps. We may end up needing this but the exact shape is unclear until we have real-world experience with this. I’ve added it to rejected ideas for now (with a slightly expanded rationale) in PEP 810: Update rejected ideas. by Yhg1s · Pull Request #4634 · python/peps · GitHub.