PEP 703: Making the Global Interpreter Lock Optional

Distribution

This PEP poses new challenges for distributing Python. At least for some time, there will be two versions of Python requiring separately compiled C-API extensions. It may take some time for C-API extension authors to build --without-gil compatible packages and upload them to PyPI. Additionally, some authors may be hesitant to support the --without-gil mode until it has wide adoption, but adoption will likely depend on the availability of Python’s rich set of extensions.

To mitigate this, the author will work with Anaconda to distribute a --without-gil version of Python together with compatible packages from conda channels. This centralizes the challenges of building extensions, and the author believes this will enable more people to use Python without the GIL sooner than they would otherwise be able to.

What is the recommendation for distributors other than Anaconda? Are we encouraged to build and offer a separate nogil build of Python (similarly to how we currently build the debug build)?

4 Likes

After reading through the PEP, I stand by my initial reaction that this is exciting and impressive. However, I agree with those saying that the compiler option is problematic. My blunt summary of this PEP is that it proposes to introduce a second “officially blessed” Python version, like PyPy or micropython. While the PEP makes the desire for multithreading clear, does it need to be part of CPython?

What if it instead was introduced as a another distribution, potentially called PyNG (for NoGil or “Next Generation”)? I think, likely naively, that it would make it easier for Python distributors to separate the projects and it would help package maintainers advertise the scope of their package.

Furthermore, PEP 690 (Lazy Imports) was introduced for similar reasons (improving performance for a subset of Python applications), with a similar opt-in mechanism (run-time flag -P), and it was rejected due to concerns that it would put significant burden on package maintainers. I think this PEP would do well to address PEP 690 and it’s rejection in the Related Work section.

4 Likes

Also, what will be the process for the builds distributed by python.org? Specifically, will people using the standard installers on Windows (or “casual user” distributions like the Windows Store distribution) have access to nogil builds outside of Anaconda?

To put this another way, will nogil be targeted as a “specialist” option, which you should only be using if you have a specific need for it (e.g., scientific/ML use, which is what Anaconda targets)?

4 Likes

I understand that the compiler option is to maintain a single cpython branch , but otherwise as in doubts of indirect effects, explicity would be nice:

Python-3.12…3.99 for gil , and Python-4.12…4.99 for nogil are released in parrallel

My_wheel-1.2.3-py3py4.whl

… universal wheels or not , it’s made explicit, no guess work, no need of dedicated site, … and has long that nogil is not estimated mature, you release binaries with the permanent rc tag.

It would handle the situation where nogil for linux is ready, and makes an environmental impact, but not nogil for windows or wasm

1 Like

It would also break Python-only codebases that (directly or indirectly) depend on GIL-only libraries.

2 Likes

I would forgo 10-20% slowdown from 3.11 (and perhaps even 3.10) speeds for single-threaded code.

I became a programmer at the tail end of the Python 2 line, so with the following two caveats that: A. memory is fading, and B. I was not responsible for huge amounts of legacy Python 2 code, the following things seem worth recalling about the 2 vs 3 split.

  • For quite a number of years, it wasn’t possible to run the same codebase in v2 and v3; six and later Python 3 versions eventually changed that, but still it did require modifying the Python parts of the codebase.
  • For quite a few years, many programmers didn’t realize a clear advantage to moving to Python 3, especially for those who didn’t have unicode problems. Rather, one had annoyances, such as print becoming a function, and your code getting slower than 2.7.
  • The ergonomics of Python3.5+ really made the upgrade unquestionably worthwhile (at which point Python 3 was ~7 years old.)

The incentive dynamics are different now, as I see it, at least.

  • Running multiple interpreters for some part of the workload is not as difficult if both enable-gil and disable-gil interpreters can run the same Python codebase without six-type modifications.
  • The pressure to utilize the entire CPU is intense.
    • Any single-threaded slowdown is accompanied by easier multi-core programming, and memory savings.
    • Thus the slowdown shouldn’t appear as apparently arbitrary to python programmers, at least not as did the slowdown from Python 2.7 to Python 3000.
  • The core devs understand the “existing code” problems vastly better than they did before.
    • Breaking changes won’t be made as lightly as before.
    • Many migration difficulties are likely to have much more documented solutions and even PRs 3rd party packages.

I don’t want any of the above to be taken as making light of the risk of splitting the community in 2 vs 3 fashion, or as provoking the core devs to be frivolous.

But I do think there’s some room to consider that the upside potential for a GIL-less python is immense, and to presume that the community will be more motivated to transition more quickly than the in 2 vs 3 era.

Hats off to the core devs for their rigorous blending of innovation and caution! Thank you for Python and increasingly, thank you for Faster Python!!

6 Likes

As maintainer of PyO3 (Rust bindings to create native extension modules as well as Rust binaries embedding Python), I’m hugely excited to see this! Safe concurrency is a common selling-point for Rust, and many times I’ve had users asking about how to do multithreaded Rust and Python.

It would take some work for PyO3 to support the new ABI proposed here. It should be possible to get to a point where all Rust extensions could be built for both ABI flags and PyO3 would (mostly) hide the details away from extension authors.

I think the resulting API which PyO3 would be able to offer would be simpler if the GIL were removed. (For any familiar to Rust, we model the binding of the GIL to a thread as a Rust “lifetime”, which is a somewhat advanced Rust topic, so we necessarily have to throw Python users in the deep end if they choose to use PyO3 to make their first foray into Rust.)

On the distribution side, I think that rather than terming it as “no GIL” I suggest the new variant should be termed something more like “multithreading optimized”. I’d speculate that would be easier to communicate to users less familiar with Python internals, which would help drive adoption.

I’m sure that PyO3 users would be keen to start experimenting with this functionality as soon as it were available, and I’m sure it would help with adoption if there were official distributions of this rather than leaving it to third-party distributions like Anaconda.

If we did not make this a 4.0, potentially python.org downloads could come in the existing default variant as well as the “multithreading optimized” variant, while extensions move towards supporting the new ABI. After a few Python releases, maybe the default can then be switched and the official downloads instead offer default or a “single-thread backwards-compatible” build.

Regardless of whether this becomes 3.13n or 4.0, I think it’s inevitable that PyO3 would have a protracted migration period (potentially even 5 years) while existing with-GIL Pythons reach end-of-life. Taking this pain seems worth it to me and we’d do what we can to make it easy for extensions built in Rust to straddle the two variants over that period.

21 Likes

I don’t see it as “huge”. To be more specific:

  • unlike the py2/py3 transition, this would mostly concern C API code, not pure Python code (I have no doubt that some Python code out there may rely on the GIL being present, but it’s certainly a small minority of all Python code written)
  • unlike the py2/py3 transition, the incompatibility is only in one direction (code that works without the GIL should work with the GIL as well), which largely eases the migration
  • the changes required are much more limited than for the py2/py3 transition; in particular, you don’t have to think about redefining visible API semantics to accomodate the bytes/str separation

However, I do think this would be important enough to warrant a major version number bump to Python 4.

17 Likes

Critical sections don’t compose in the sense that starting a new critical section may suspend a previous one. If you need to lock two structures at once, you must use the Py_BEGIN_CRITICAL_SECTION2 function, which locks two mutexes “simultaneously” in one critical section. The APIs do not handle locking more than two mutexes at once, but I haven’t seen a need for that within CPython or in C API extensions.

Like Antoine, I don’t think the change is huge. I agree with his points and I’ll add a few extra bits of context:

  • “pure” Python code doesn’t require any changes to continue working. The exception is that there are some bugs that are triggered rarely with the GIL (maybe 1 in a million runs) but are triggered much more often without the GIL because thread “interleavings” happen much more often without the GIL. I’ve found and fixed a few of these bugs in CPython (and occasionally written a few of these sorts of bugs in packages I’ve worked on.)
  • Most C API extensions don’t require any changes, and for those that do require changes, the changes are small. For example, for “nogil” Python I’m providing binary wheels for ~35 extensions that are slow or difficult to build from source and only about seven projects required code changes (PyTorch, pybind11, Cython, numpy, scikit-learn, viztracer, pyo3). For four of those projects, the code changes have already been contributed and adopted upstream. For comparison, many of those same projects also frequently require changes to support minor releases of CPython.

I don’t have a strong opinion about version numbering, but if the PEP were adopted and labeled as Python 4, I would like to avoid using the version bump as an opportunity to introduce other backwards incompatible changes.

EDIT: six → seven projects

12 Likes

This PEP doesn’t introduced any large breaking API changes, but making it Python 4 would: a large number of scripts and instructions for Linux run Python as python3, which I assume wouldn’t work with Python 4

6 Likes

I discussed this with Peter Wang at PyCon US back in May, but have not worked out all the details like which channel it would be available in. The likely contenders would be “main”, “conda-forge”, or a channel I maintain specifically for this purpose. Either way, the recipes will likely be based conda-forge recipes, so having support from the conda-forge team would be great. I’ll reach out to them and try to sync up with Peter.

I think the PEP should address this, but I’d like to gather more feedback before proposing something. What do you think the recommendation should be for other distributors? I don’t have a sense if distributing multiple versions of Python on Fedora, for example, is too much of a burden or provides sufficient value.

At the PyCon 2022 language summit, I suggested not distributing the --without-gil build on python.org initially, but after reconsidering this, I don’t see a good reason not to. Again, I’d appreciate suggestions and feedback on this.

You might be able to do a per-thread freelist, but that’s basically what mimalloc provides. Mimalloc’s freelists are based on block size instead of concrete type, but otherwise pretty similar. There might still be a small performance advantage of essentially re-creating the freelists outside of mimalloc to avoid some of the indirections involved in calling the allocator, but it might be a better use of effort to focus on optimizing those code paths.

1 Like

Yeah, I think this should addressed in the PEP, but I don’t know what the behavior should be. I might lean towards issuing a warning. Do people have suggestions?

I think the --without-gil build should be available from python.org, and I think the PEP should cover how packages suitable for use with the nogil build would be distributed on PyPI (from what you’ve said, I’d imagine that many packages could just work without change, but some would need different builds - I’m not clear how you’d do that without a new packaging tag and requiring existing projects to rebuild or at least retag their wheels…)

After all, unless nogil is intended to forever remain a specialist option, you’ll need to address those matters at some point, and so why not in the PEP?

7 Likes

Some reasons:

  • It will double the amount of 3.x builds (& testing!) required by any medium-and-up-sized Python project, which is a substantial burden.
  • It will also plague infrastructure that has been built with the assumption that there’s only one ABI per python (minor) version, which has been true ~forever.
  • I don’t know if the pip resolver would be capable of distinguishing the ABIs when trying to resolve the installation of a package, but I suspect not. This would be a huge usability issue (installing a package compiled for the wrong ABI, or falling back to source installs of packages not yet nogil-ready, etc.)

Not counting the debug ABI (which isn’t generally something people distribute), there hasn’t been a python version with two productive ABIs, so the above list is just the start of a likely large number of unintended side effects that someone needs to fix. IMO the onus is on the PEP to argue why the benefits of having parallel ABIs outweigh all that work.

I understand that it sounds like an appealing option to have nogil be available more quickly, and you strengthen that point with…

… but IMO it’s going to take a while to digest this either way, and the acceleration provided by parallel ABIs risks ending up being a mirage that will cause very high costs for maintainers.

On that topic: By my reading of the stable ABI promises, if you intend to break that, a major version bump is unavoidable. And certainly, other pent up changes will attach themselves to such an occasion – e.g. the packaging ecosystem is considering various flavours of large overhauls, c.f.

But that’s to be expected IMO – nogil is an amazing piece of work, but it’s still not realistic to ask that all other brewing / pending changes in the rest of the Python ecosystem give nogil “exclusivity” on a major version bump.

4 Likes

Great! I have patches to PyO3 that I use to build a compatible version for the “nogil” fork. I’ll open a PR with those changes. I think the PR will be useful to discuss the implications for PyO3 in the context of the actual code.

6 Likes

Yes, I think it needs to be part of CPython to be useful. CPython is much more widely used than any other Python implementation. CPython solves a coordination problem: that’s where most of the work on improving Python will be done because that’s where the most users will get value. As a practical matter, I don’t think it will be possible to maintain this long term as a separate fork of CPython.

9 Likes

I’ll describe the process and changes to build wheels for --without-gil. I’m not sure which of these details should be in the PEP, because a lot of these tools are not usually covered by PEPs.

  1. The manylinux images should include a --without-gil build. For example, based on the PEP, there would be a Python installation in /opt/python/cp312-cp312n/ as well as /opt/python/cp312-cp312/ and the nine existing Python installations.
  2. The cibuildwheel GitHub action should support cp312n (the --without-gil build).
  3. The setup-python GitHub action should similarly support 312n.
  4. For a project that uses cibuildwheel, like SciPy, you might change this line to include the item [cp312n, 3.12].

For (2) it would be helpful to have installers available on https://www.python.org/ftp/python/.

If you were building a wheel by hand, you would just need to run pip wheel . with a --without-gil install, just like you would for other Python versions. Pip already handles the appropriate abi tags.

There aren’t any changes necessary to pip, wheel, or pypi. For context, I’ve made most of these changes in forks of the mentioned projects (i.e., cibuildwheel, setup-python) to build C API extensions for the “nogil” proof of concept.