Many comments in a short time on the first public draft does not mean that it has had reasonable review. The many comments are a reason why many people would wait before commenting themselves.
After this:
I was waiting to see an announcement that the PEP was updated. Instead the announcement says it was sent to SC.
I understand and I respect your view. We still think that, given the current state of the discussion and the overwhelmingly positive feeling, we believe it’s ready for review. If the SC believes some aspect must be discussed more or they don’t have enough signal they will surely tell us, as many times the SC has done. I respect if you would have acted differently. The Steering Council recommendation in the SC tracker repo is waiting around a week, which we have done (:
Generally you should wait about a week for people to react there before filing this issue.
On top of that, I can tell you that this PEP won’t be reviewed at least in the next two weeks (as you know the SC is quite slow and has a considerable queue including at least two other PEPs), so please feel free to continue commenting in this thread as much as you think it’s necessary and the SC will surely read your comments when the PEP is reviewed. I will continue to read all the comments and update the PEP as needed.
The PEP was updated with the things I mentioned in the post. If you think you don’t like the text or you don’t agree with it, please mention it here or open a PR to the PEP with the proposed fix, and we will surely consider it!
Update: I have requested in the submission post to not review this PEP before at least one or two weeks just to be sure everyone is happy.
It is good to see that this is in the update:
We think it’s reasonable for package maintainers, as they update packages to adopt lazy imports, to decide to not support running with lazy imports globally disabled.
I think it is inevitable that the global flag to force eager imports would immediately become unusable as soon as any libraries changed to use lazy imports.
I’m late to the party but this would be a life saver for projects like Kedro, which has a complex CLI interface that requires importing lots of code that most of the time is not used. The syntax looks good to me.
The global lazy imports control looks like a smart way of giving users early access to the feature. Its existence will mitigate the wave of “please make everything lazy" GitHub issues, so maintainers will have less pressure to adapt, I think. Plus it will work with older releases, which is also great.
This makes me wonder if there should be a way for each module/package to state whether it’s safe for lazy importing.
(how to expose this would be an open question, because you don’t want to eager-import numpy just to query the numpy.__safe_for_lazy_importing__ attribute 
)
This is indeed an interesting point although we left it as a deferred idea due to the scope:
Meanwhile the filter will be the main mechanism. We also will be releasing to PyPi a tool that is currently used at Meta that statically analyzes the project and the dependencies and generates output that can be directly used in the filter. @brittanyrey will be able to provide more details.
I have updated the demo to the latest version fixing a bunch of small bugs and making it closer to the PEP document. There are still some differences with the PEP so please refer to the PEP as the canonical description.
Correct, we’re currently developing a static analyzer that detects lazy imports incompatible code (i.e. side effect detection). We currently have a version of this analyzer that we’ve been proving out internally on a few dozen Python binaries over the last year–but have been actively developing a much more performant version of this tooling that we hope to open source and release next year.
I agree that this seems quite fast for a “big” PEP like this (that is, proposing a fairly major feature).
That is true but I think just as relevant is the stuff in PEP 1 about “building consensus”. Although there is overall positive sentiment about the PEP it’s not clear to me that consensus was reached on some issues (e.g., the try/except stuff). It’s interesting that the SC tracker repo says that because it made me realize that based on past PEP discussions I tend to expect more like “wait a week after discussion has stabilized”, but I guess it doesn’t actually say that anywhere.
To clarify the PEP process here, it’s up to the PEP authors to decide when to send a PEP to the SC as it’s their PEP. It’s then up to the SC to decide whether they think enough time and consideration was given for the PEP. So the PEP authors don’t have a hard, clear signal/requirement on when to send a PEP on to the SC beyond a week’s time. Ultimately it’s up to the PEP authors to balance getting their PEP accepted (or rejected).
There’s also a balance with PEPs like this which are hotly contested or trigger a huge volume of posts (especially when people start popping in with comments without reading the PEP and the comments here). Having gone through this myself, at some point you have to cut off discussion as it starts to go in circles and exhausts the whole process. Plus some things will never garner consensus, so as a PEP author you make a call and hope it’s the one the SC likes and won’t reject your PEP over.
Add in the inevitable, nasty emails being sent to the authors over this – which happens with every syntax PEP – and it makes keeping the conversation going until everyone burns themselves out not a viable option.
Hopefully my comment is not what you mean by angry emails. I’m not angry at anyone and I apologise if it came across that way.
Fine, but I give my opinion that it looks rushed and that there has not been time for people who were not involved in prior discussions and who saw the PEP when announced to consider the options and details here.
I think that it is also important to recognise that the process situation looks quite different from the outside when the set of PEP authors overlaps with the membership of the SC. In this situation in particular it doesn’t look good if there is the appearance of rushing through public discussion to get to an SC pronouncement.
Please no one interpret this as me saying that I disagree with the PEP or with the intentions of the authors or anything like that. I just think that this is a big language change that is worthy of thought and consideration.
No, not at all! I’m talking about actual emails where the PEP authors have been cussed out and yelled at for this PEP. The comment about this is for people to understand they have not seen all of the feedback that the PEP authors have received (good or bad).
Having been in this position myself, I can tell you the SC won’t be swayed by Pablo being currently on the SC or Thomas having been on it previously (if it actually mattered then Pablo and I would have submitted PEP 760 – No More Bare Excepts | peps.python.org to the SC regardless of the community feedback).
I personally don’t call 1.5 weeks and now 312 posts “rushed” (especially when it’s in response to a previous PEP which itself had 216 posts). Plus, you just have to call it “done” at some point. It’s a balancing act about trying to reach consensus and giving people time to provide feedback as well as not letting it drag on. I’m the 5th most prolific PEP author and I can tell you that this is impossible to balance in a way that makes everyone happy. Do it “too soon” in some people’s view and you get accused of rushing it (which, once again, is the SC’s job to decide if that’s true). But leave it and you either burn out or run out of time to work on the PEP.
Personally, I’m not seeing new issues being brought up, so that’s the first sign that the PEP could be considered ready for submission. After that it’s a gut feeling as to when the discussion is stuck in a loop because consensus just isn’t possible on some topic. Once again, the things people are still discussing are subjective and thus up to the SC to decide if they agree with the PEP or not.
But in the end the PEP authors decided to give everyone a bit more time, but if it were me I would take the lower bound one week opening as a deadline (and I personally would have made it a hard deadline of either 1 or 2 weeks so people don’t procrastinate and to know there is an end date). Having been a core developer for over 22 years, I can tell you there will always be someone who feels their voice wasn’t heard because the timing to comment didn’t align with their availability no matter how long you wait (and I’ve waited over a month and still been told I didn’t wait long enough by someone).
Okay, so here is one. This is not a new idea since it was mentioned in different ways above although it is hard to find the references now.
I envisage that if this PEP is accepted as-is then I will change thousands of library code modules to look like:
__lazy_imports__ == ['x', 'y', 'z']
import x
import y
import z
It would be awkward to keep __lazy_imports__ in sync with the imports below. It would be better to be able to do something like:
__lazy_imports__ == ['*']
The same consideration applies to a future time when __lazy_modules__ is no longer necessary. Probably I will want almost all imports to be lazy by default and so the end result will be something like
lazy import x
lazy import y
lazy import z
# This will be the mechanism to force eager imports when needed:
z.initialize()
I think I would prefer to have some more global (in the code e.g. module level) way of specifying that the imports should be lazy though. This longer term part needs the mode consideration: I expect almost all library imports to become lazy so how do you really want that code to look in the end state?
I think the answer to all these I’m going to have to mass refactor all my imports comments is just that you don’t have to to get the benefits. It only makes sense for module A to lazy import module B if A can provide a significant functionality without using B. Otherwise, it makes more sense to lazy import A (assuming that’s also meaningful).
I’ve been playing around with the reference branch locally these last couple of weeks – my experience has been that once I’ve lazified the obvious places (imports that aren’t needed to run --help, whatever -X importtime says is slow, maybe the imports in the __init__.py), the timing difference between that and -X lazy_imports=on [1] quickly drops beneath the noise floor.
There’s an on vs all asymmetry between the reference and the PEP ↩︎
Put it the other way round: when would you need A to eager import B if lazy imports were default?
Doesn’t the reasons for PEP 690’s rejection answer that?
It’s a change that shouldn’t be capable of breaking well behaved existing code, so I somewhat agree here, but I do think that the general idea that a week is a reasonable amount of time to solicit feedback on major changes to the language may not hold up to what people want in terms of stable expectations of how a mature language behaves[1], and expecting everyone potentially impacted to be online weekly, with time to consider the impact of changes to their use and reason about it may actually contribute to why some people are so impassioned when they argue against changes.
I’m not speaking to stability of the language here, but to the expectations people have for a language. I’m not discounting stability of the language, but it’s barely relevant to this topic, even while being relevant elsewhere. ↩︎
You start by replying to @brettcannon saying there are no new ideas being mentioned, and then immediately follow with “here is something new that’s not new” 
I have to assume that was meant as a joke.
Also, the point you raise has already been covered several times in the discussion and in the rejected ideas and design rationale sections. It was also part of the reasoning behind the rejection of PEP 690. In my view, the authors have already evaluated it and made a deliberate design choice. You can’t simply claim that this topic “hasn’t been discussed enough” just because the authors don’t agree with your preferred outcome. Disagreement doesn’t mean lack of discussion: it just means a different conclusion was reached. At some point, decisions have to be made, and in this case they’ve been made transparently and with a lot of input.
Although I agree that maybe a bit more time may be good in general, just to be fair, the authors immediately reacted to feedback about timing by giving two more weeks for discussion before review. In my book, that shows a lot of goodwill and that even if they don’t agree on every point, they genuinely want people to feel heard and happy with the process.
Yet here we are, with everyone talking more about the discussion process and the timing than about the actual PEP itself.
You raise an interesting point but allow me to say that this doesn’t need to be necessarily a matter of time if people think that the design space has already been explored and there are no blind spots left. Everyone has a version of the PEP they would like, but the authors have to make decisions. More time for the sake of time will not help much if those decisions have already been made after careful consideration. People can still comment here and the Steering Council will read (I hope), so at this point we are just boiling the ocean and making this discussion harder to follow.
Also, this kind of meta discussion probably should not be happening here, especially since it is old news at this point. The PEP won’t be reviewed right now by the SC, and the authors already paused to give everyone more time for feedback.