I will admit some trepidation wading into a discussion of Python heavyweights 
, but I wanted to reinforce the comment from @thinkwell (and the others he links to) that, from the outside, this seems like a transformative opportunity for CPython and it would be really disappointing to walk away from the huge amount of work this represents.
I also wanted to show a very small example here. I cloned Sam’s repo and built it on a GCP VM (n2d-standard-2). Then I wrote this:
from concurrent.futures import ThreadPoolExecutor
def parmap(fn, *args, n_threads=4):
with ThreadPoolExecutor(n_threads) as exec:
yield from exec.map(fn, *args)
And this just works, out of the box, to parallelize map. I can pass a lambda function to it (ProcessPoolExecutor will error out, to say nothing of the overhead). [1]
I would humbly say that I’m pretty good at writing Python, but nowhere near the people in this thread. I wrote this in a minute or two. edit to walk back my “pretty good” comment, I didn’t run this test properly the first time, because it was something that python is already fine with. The current nogil repo isn’t obviously better depending on the function, but this is more about a demonstration of usability and possiblities.
It feels to me like an important aspect of the performance discussion is to consider how much a nogil Python enables, both for extension and package maintainers but also for users. I’d happily take a 10% hit on single-thread performance for trivial parallelization like this.
This is just as true for nogil as well, right? There would be an additional release of CPython, but no one has to support it.
It seems like from a practical point of view, both options require effort from package maintainers [2]
to make sure they work in the new mode, and they may want some way to advertise that fact in their releases.
In that respect, ABI incompatibility seems very similar to the new “fails to import” behavior described PEP 684, except it could happen even earlier, when installing the package.