I acknowledge that, and I disagree that “many” is a significant number at this point. I made the decision that the original warning I added would do more harm than good for now based on user experience combined with past Python warnings shipping in releases and showing up for the wrong users experience. Without evidence that the majority of Linux/BSD users are going to be broken as opposed to not care, I’m unlikely to change my mind.
Why did I decide that? Warnings that show up to the wrong people at the wrong point in time are much more harmful. When there are more wrong people and wrong times than the right people and right times, it shouldn’t be a warning. We’ve learned this lesson the hard way in the past. I really wanted that not to be the case when coding up this warning, but it already proved itself otherwise from projects testing against nightly builds.
The default start method was flipped on macOS in 3.8 without advanced notice and without users showing up with pitchforks. (we didn’t want to do it that way but had no choice).
I predict the amount of unpleasant surprise as likely to be far lower than the amount of non-broken user frustration that telling literally everybody on Linux to add a few lines of questionable logic to some already working code (not necessarily owned by them) that declares that they’d like it to keep working without seeing a warning will generate.
The lines of code that we could ask them to add in a 3.7-3.14 compatible way (as the short lived warning did) do not clearly match the intent we’d like to see expressed. There is no clean understandable way to express “I don’t care, the default is fine, give me my processes” as a multiprocessing start method context. Telling people to explicitly pick a specific start method would move the worlds code backwards: Most users do not care but cannot express that without forcing a potentially inferior over-specific choice. What we want is only for the few that do care to be explicit about their actual requirement.
The fix for anyone surprised when the default changes away from "fork" is to add 1 line of code to explicitly request it. I see this as the least disruptive way forward. It is normal to need to make some changes on occasion when adopting a new Python release, this will be one of them for that presumed minority of code.
There are still user helpful improvements that can be made to ease the default transition. We’ve still got time before 3.12 to explore these - in TODO list form:
- 3.12 raises a DeprecationWarning from
os.fork()on Linux & macOS when it is able to detect that the process is multithreaded. It is not guaranteed to be able to do so, but in common configurations it works. multiprocessing can trigger that warning internally. As is, this serves as a heads up to people already in sketchy situations - One reason why we’re changing the default in the first place. - As noted in the default change issue, error messages in common failure modes when switching some code from
"fork"to"spawn"or"forkserver"could probably be made more informative to the user about potential reasons and actions they need to take. - It seems reasonable to finish up exposing an API to try to find out how many threads the process has and use that to add a more directly informative DeprecationWarning when the presence of threads in the multiprocessing-forking process is detected. Because those already get the new
os.fork()warning. This way it’d be specific text tailored to what actions can be taken in the context ofmultiprocessingorconcurrent.futuresAPI use rather than blaming their internals without directed advice. - Consider adding the abstract ability to express what semantics are required or preferred from the start method rather than having code specify a specific start method.
I’ll probably be pasting that list into the issue or opening individual issues for some of that.
If we have a huge numbers of things breaking without the ability to adapt to this change during 3.14 alpha and beta period we can reconsider the default flip. I don’t expect that to happen. (Spanish inquisition meme goes here)
-gps