As a “consumer” of open source, I’m grateful to Sam Gross for forcing this discussion. There’s been a lot of very technical discussion, but as an outsider, I can only say that a good deal of what has happened in this thread looks more like turf wars and motivated reasoning than a community trying to solve the problems of its “consumers.”
Python has succeeded beyond almost anyone’s wildest expectations, and outside the genius of the initial design, it’s not hard to see that a a great deal of its success has come from those who built up the ecosystem around it. This ecosystem now functions as a moat which other languages with the advantage of 2-3 decades of hindsight have nevertheless been unable to surmount.
Nevertheless, Python will someday fade in significance just as other languages have faded. For Python, the reason for its fade is unknowable, but I think I can say confidently that the average senior developer knows two things about Python: it is “slow”, and it cannot do parallel processing with shared memory. There have been many efforts to improve Python along the first axis, but the common knowledge is that Python is exceptionally dynamic and this flexibility will never allow it to be as performant as an AOT compiled language in this regard. The second axis, however has seen only two notable efforts, and the consensus seems to be that this is the first one that comes in at single digit single-threaded performance reduction.
The apparent focus of core developers on single-threaded performance is perhaps the strangest thing about this discussion. When (also open-source) competitors like Java are forging ahead with efforts like Project Loom, acknowledging that the greatest challenge to software engineers of the present (and future) is making efficient and comprehensible use of multiple cores as Moore’s Law approaches its end, CPython’s fixation on trying to squeeze the best possible performance out of a single core truly boggles my mind.
Core Python developers did not (and could not have) predicted the rise of NumPy, Pandas, and the various other libraries that have catapulted Python to the top of the heap for any number of uses for which it would have been an unfathomable choice even 15 years ago. It seems rather odd to assume that the Python ecosystem cannot evolve to meet the challenges of “nogil”, and not just evolve, but indeed open up dramatically new horizons for system-building. And claiming that the steering council somehow needs to know the answers for what higher-level abstractions could or would be built on top of free threading is a dramatic display of narrow thinking.
Ultimately, my thesis, which is borne out by the history of Python itself, is that core developers should be far less concerned about trying to predict the future, and far more concerned about opening up possibilities that were previously impossible, and letting the incredibly powerful ecosystem push things forward as it has done for a long time. Many of us in that ecosystem have been hanging on for dear life as we wait for Python to prove that it still has a future. But languages content to rest on their laurels (like all software) eventually get eclipsed by those that are willing to take risks and trust to the broader community to explore the space opened up by foundational improvements.
Perhaps my overlong post here is just so much rhetoric. But there are many of us (here I speak mainly for the scientific and data processing community) who so far have remained on the sidelines, commenting to each other on how disappointed we are in how long it has taken to move this forward. And seeing the opposition of a few long-time core developers in public has confirmed what we already feared.
To be brutally honest, a Python that leaves these types of dramatic evolutions sitting on the workbench gathering dust is a Python that, as far as many of us are concerned, is already on its way out. And we all know that when something is on its way out, its consumers quite often find themselves looking for the next big thing. 