I tried to get at this point once above already, but I’m not sure it came through clearly, so let me try again.
What I think the discussions of “library opt-out” are missing is that “safe for lazy imports” is fundamentally not even a meaningful or coherent property of a single module or library in isolation. It is only meaningful in the context of an actual application codebase. This is because no single module or library can ever control the ordering of imports or how the import-time code path flows: it is an emergent property of the interaction of all modules in the codebase and their imports.
Every single Python module that subclasses a class from another module has visible import side effects: it changes the value of __subclasses__() on the parent class. So is it lazy import safe? Well, if you are being 100% conservative, no! We can try to be 100% conservative, and the result is that you just can’t use lazy imports at all (which won’t happen: whether PEP 690 is accepted or not, people will and already do use globally-enabled lazy imports via demandimport, LazyLoader, etc., and Python already supports them.) But in reality, it depends: is anyone looking at the value of __subclasses__() on that base class at a time when it isn’t populated yet? 99+% of the time, the answer is no, and the module is perfectly safe to use with lazy imports. But it’s a question that has no useful answer for the module in isolation: there is only a meaningful answer for an entire application in context.
Same is true (as I already outlined above) for libraries like click or pydantic. Their own modules are perfectly fine for lazy import. But they provide decorators / base classes that could be used in an application module in a way that makes an application module potentially need eager importing. So what does it even mean for a library like that to define itself or its modules as “lazy import safe” or “not lazy import safe?” How do the proposed APIs help this case?
I think the nature of the opt-out in PEP 690 is not well understood. It is not an exercise in categorizing modules into neatly-defined objective categories of “safe for lazy import” and “not safe for lazy import.” (If it were, the only possible answer would be that no module is ever fully lazy import safe.) Rather, it is a way for an application developer to say “in the context of my specific entire application and how it actually works, I need to force these particular imports to be eager in order for the effects I actually need to happen in time.”
This is why I don’t see much value in providing APIs to allow people at different points in the chain to nest both deep and shallow opt-outs and opt-ins. These APIs are mostly harmful, because they serve to take away control from the only person who can provide meaningful answers to questions of what actually must be imported eagerly, and that is the application developer who can test how their application actually behaves.
Take even the most obvious case you could imagine of “not lazy import safe” in a library: imagine some library that has some module that is imported only for its side effects, and must be imported before other things in the system happen (that don’t involve referencing names in that module.) (Set aside for the moment the fact that such a library design is already highly fragile even with eager imports: anyone can import your library lazily today by inlining the import and libraries already get no say over this.) You would think that this “definitely not lazy import safe” library is the perfect case for an opt-out. But the opt-out doesn’t even help this “obvious” case! A module earlier in the import chain leading to this module could still be lazily imported despite the library’s opt-out, and (in the context of the whole system) the side effects will still be lazy. (Exactly as can happen today with manually inlined imports.) Again this is an illustration of the same basic point: the overall application is the only context in which questions of import ordering and import side effects can be usefully resolved.
The one case in which it makes sense for a library module to explicitly opt out of a lazy import is if the library author knows that their own module B imports module A and the rest of the code of B directly depends on side effects from A (without referencing any imported names from A). Again, this is an already-fragile and I think rare case. But PEP 690 already provides all the tools a library author needs in order to handle that case (if they want to): put the import of module A, in module B, inside any try/except or a with statement.
@zrothberg I think the only way we can reasonably evaluate the need for the full matrix of context manager APIs you’ve suggested is to look at specific real world examples. Can you propose a real-world library that you think would benefit from these capabilities? Then we can look concretely at how that library could handle its needs with the existing APIs proposed in the PEP or with the full matrix of context managers.