jrincayc
(Joshua Cogliati)
August 4, 2023, 2:27am
122
FYI: The steering committee decided that “We intend to accept PEP 703, although we’re still working on the acceptance details.”
4 Likes
hugovk
(Hugo van Kemenade)
October 25, 2023, 6:22am
123
The steering council has now formally accepted PEP 703:
(Posted for the whole Steering Council.)
As we’ve announced before , the Steering Council has decided to accept PEP 703 (Making the Global Interpreter Lock Optional in CPython) . We want to make it clear why, and under what expectations we’re doing so.
It is clear to the Steering Council that theoretically, a no-GIL (or free-threaded) Python would be of great benefit, and the majority of the community seems in agreement. Threads have significant downsides and caveats, but they are widely adopte…
12 Likes
jrincayc
(Joshua Cogliati)
October 30, 2023, 2:00am
124
4 Likes
smontanaro
(Skip Montanaro)
November 22, 2023, 12:25pm
125
Has the no-gil code been fully merged to main at this point or is there more to do? I’ve tried configuring with and without --disable-gil
and was mildly surprised not to see a change in the number of test cases run. I would have expected to see some new test cases.
ambv
(Łukasz Langa)
November 22, 2023, 12:27pm
126
It’s work in progress.
Follow this issue for the current state:
opened 03:59PM - 21 Aug 23 UTC
type-feature
interpreter-core
3.13
topic-free-threaded
# Feature or enhancement
The steering council has [accepted](https://discuss.… python.org/t/pep-703-making-the-global-interpreter-lock-optional-in-cpython-acceptance/37075) [PEP 703](https://peps.python.org/pep-0703/). This is intended as a top-level issue to keep track of integration status.
The "up for grabs" list contains issues that no one is currently working on and are ready to be implemented. If you are interested in working on one of them, please comment on the specific issue and CC me (@colesbury) on related PRs.
```[tasklist]
### Up For Grabs (comment on specific issue to take)
- [ ] https://github.com/python/cpython/issues/112205
```
```[tasklist]
### Linked issues
- [ ] https://github.com/python/cpython/issues/108223
- [ ] https://github.com/python/cpython/issues/108374
- [ ] https://github.com/python/cpython/issues/108337
- [ ] https://github.com/python/cpython/issues/108724
- [ ] https://github.com/python/cpython/issues/109549
- [ ] https://github.com/python/cpython/issues/109740
- [ ] https://github.com/python/cpython/issues/110119
- [ ] https://github.com/pypa/packaging/issues/727
- [ ] https://github.com/python/cpython/issues/111062
- [ ] https://github.com/python/cpython/issues/111569
- [ ] https://github.com/python/cpython/issues/112062
- [ ] https://github.com/python/cpython/issues/111903
- [ ] https://github.com/python/cpython/issues/111916
- [ ] https://github.com/python/cpython/issues/111928
- [ ] https://github.com/python/cpython/issues/112070
- [ ] https://github.com/python/cpython/issues/111956
- [ ] https://github.com/python/cpython/issues/110481
- [ ] https://github.com/python/cpython/issues/111506
- [ ] https://github.com/python/cpython/issues/111650
- [ ] https://github.com/python/cpython/issues/111870
- [ ] https://github.com/python/cpython/issues/111924
- [ ] https://github.com/python/cpython/issues/111926
- [ ] https://github.com/python/cpython/issues/111962
- [ ] https://github.com/python/cpython/issues/111964
- [ ] https://github.com/python/cpython/issues/111968
- [ ] https://github.com/python/cpython/issues/111971
- [ ] https://github.com/python/cpython/issues/111972
- [ ] https://github.com/python/cpython/issues/112050
- [ ] https://github.com/python/cpython/issues/112066
- [ ] https://github.com/python/cpython/issues/112075
- [ ] https://github.com/python/cpython/issues/112087
- [ ] https://github.com/python/cpython/issues/112071
- [ ] https://github.com/python/cpython/issues/112069
- [ ] https://github.com/python/cpython/issues/111965
- [ ] https://github.com/python/cpython/issues/112213
```
### Linked PRs
```[tasklist]
### Deferred tasks
- [ ] Revisit and update [`--disable-gil`](https://docs.python.org/dev/using/configure.html#cmdoption-disable-gil) configure documentation
- [ ] Reenable test_cppext on `--disable-gil` builds
```
### Upstream functionality from nogil-3.12
This is a list of commits from the nogil-3.12 PR plan. The crossed-out entries are commits that do not need to be upstreamed, usually because the functionality is already in the main branch.
- [ ] [`cefe5dfee9`](https://github.com/colesbury/nogil-3.12/commit/cefe5dfee9) configure: disallow "--with-trace-refs" for "--disable-gil" builds
- [ ] [`dcddbe2ddb`](https://github.com/colesbury/nogil-3.12/commit/dcddbe2ddb) configure: add support for --with-thread-sanitizer
- [ ] [`f546dbf16a`](https://github.com/colesbury/nogil-3.12/commit/f546dbf16a) Enable/disable the GIL at runtime
- [x] [`f30d8d8f50`](https://github.com/colesbury/nogil-3.12/commit/f30d8d8f50) Add pyatomic.h
- [x] ~~[`385eb1d99c`](https://github.com/colesbury/nogil-3.12/commit/385eb1d99c) pyport: add new macros~~
- [x] ~~[`de2be447b3`](https://github.com/colesbury/nogil-3.12/commit/de2be447b3) Make PyThreadState_GET thread-local~~
- [x] [`a24dc2ecc3`](https://github.com/colesbury/nogil-3.12/commit/a24dc2ecc3) pystate: keep track of attached vs. detached state
- [x] [`4584be5950`](https://github.com/colesbury/nogil-3.12/commit/4584be5950) parking_lot: add mutexes and one-time notifications
- [x] [`6845b133cc`](https://github.com/colesbury/nogil-3.12/commit/6845b133cc) critical_section: helpers for fine-grained locking (gh-111569)
- [ ] [`8ed62cab6a`](https://github.com/colesbury/nogil-3.12/commit/8ed62cab6a) pystate: use _PyRawMutex for internal mutexes (https://github.com/python/cpython/issues/111924)
- [ ] [`e15443b1f2`](https://github.com/colesbury/nogil-3.12/commit/e15443b1f2) ceval: move eval_breaker to per-thread state (#112175)
- [x] [`b6b12a9a94`](https://github.com/colesbury/nogil-3.12/commit/b6b12a9a94) Implement biased reference counting
- [ ] [`b6b12a9a94`](https://github.com/colesbury/nogil-3.12/commit/b6b12a9a94) Implement BRC inter-thread queue (#110481)
- [x] ~~[`7b6b6f1a01`](https://github.com/colesbury/nogil-3.12/commit/7b6b6f1a01) unicode: immortalize interned strings~~
- [x] ~~[`fc173e3711`](https://github.com/colesbury/nogil-3.12/commit/fc173e3711) unicode: always immortalize interned strings~~
- [ ] [`dd9b78460c`](https://github.com/colesbury/nogil-3.12/commit/dd9b78460c) Add safe memory reclamation scheme based on FreeBSD's GUS
- [x] [`901e134921`](https://github.com/colesbury/nogil-3.12/commit/901e134921) Add mimalloc v2.0.9 (DinoV)
- [ ] [`d13c63dee9`](https://github.com/colesbury/nogil-3.12/commit/d13c63dee9) pymem: remove uses of _PyMem_SetDefaultAllocator during finalization (https://github.com/python/cpython/issues/111924)
- [ ] [`654be8ffd6`](https://github.com/colesbury/nogil-3.12/commit/654be8ffd6) gc: make the garbage collector non-generational
- [ ] [`967fe31473`](https://github.com/colesbury/nogil-3.12/commit/967fe31473) gc: Traverese mimalloc heaps to find all objects.
- [ ] [`2864b6b36e`](https://github.com/colesbury/nogil-3.12/commit/2864b6b36e) Implement stop-the-world pauses (#111964)
- [ ] [`2864b6b36e`](https://github.com/colesbury/nogil-3.12/commit/2864b6b36e) gc: implement stop-the-world GC
- [ ] [`c1befd7689`](https://github.com/colesbury/nogil-3.12/commit/c1befd7689) Stop the world before fork() and Python shutdown
- [ ] [`82800d8ec8`](https://github.com/colesbury/nogil-3.12/commit/82800d8ec8) ceval: stop the world when enabling profiling/tracing for all threads
- [ ] [`7423dff344`](https://github.com/colesbury/nogil-3.12/commit/7423dff344) pystate: use stop-the-world in a few places
- [ ] [`86efa7dfe3`](https://github.com/colesbury/nogil-3.12/commit/86efa7dfe3) pystate: implement _PyRuntime.multithreaded
- [ ] [`d896dfc8db`](https://github.com/colesbury/nogil-3.12/commit/d896dfc8db) dict: make dict thread-safe (#112075)
- [ ] [`df4c51f82b`](https://github.com/colesbury/nogil-3.12/commit/df4c51f82b) list: make list thread-safe (#112087)
- [ ] [`9c1f7ba1b4`](https://github.com/colesbury/nogil-3.12/commit/9c1f7ba1b4) mro: thread-safe MRO cache
- [x] [`7a7aca096b`](https://github.com/colesbury/nogil-3.12/commit/7a7aca096b) getargs.c: make parser_init thread-safe (#111956)
- [ ] [`0dddcb6f9d`](https://github.com/colesbury/nogil-3.12/commit/0dddcb6f9d) weakref: make weakrefs thread-safe without the GIL (#111926)
- [ ] [`410ba1036a`](https://github.com/colesbury/nogil-3.12/commit/410ba1036a) dtoa: make dtoa thread-safe (#111962)
- [ ] [`6540bf3e6a`](https://github.com/colesbury/nogil-3.12/commit/6540bf3e6a) unicode: make unicodeobject.c thread-safe (#111971)
- [ ] [`5d006db9fa`](https://github.com/colesbury/nogil-3.12/commit/5d006db9fa) codecs.c: fix race condition (#111972)
- [ ] [`d1b5ed128e`](https://github.com/colesbury/nogil-3.12/commit/d1b5ed128e) _threadmodule: make _thread.lock thread-safe
- [ ] [`cfc11bcb1a`](https://github.com/colesbury/nogil-3.12/commit/cfc11bcb1a) typeobject: thread safety
- [ ] [`cfecf6f4eb`](https://github.com/colesbury/nogil-3.12/commit/cfecf6f4eb) threading: remove _tstate_lock from threading.Thread
- [ ] [`74df7785f5`](https://github.com/colesbury/nogil-3.12/commit/74df7785f5) pyqueue: add internal queue data structure
- [ ] [`4450445f51`](https://github.com/colesbury/nogil-3.12/commit/4450445f51) pymem: add _PyMem_FreeQsbr
- [ ] [`7e60a01aee`](https://github.com/colesbury/nogil-3.12/commit/7e60a01aee) queue: make SimpleQueue thread-safe
- [ ] [`4ca2924f0d`](https://github.com/colesbury/nogil-3.12/commit/4ca2924f0d) set: make set thread-safe (#112069)
- [x] ~~[`3cfbc49229`](https://github.com/colesbury/nogil-3.12/commit/3cfbc49229) moduleobject: fix data races~~
- [ ] [`5722416ef5`](https://github.com/colesbury/nogil-3.12/commit/5722416ef5) _threadmodule: thread-safety fixes
- [ ] [`31ec6f0290`](https://github.com/colesbury/nogil-3.12/commit/31ec6f0290) pystate: refcount threads to handle race between interpreter shutdown and thread exit
- [ ] [`45bdd27ee5`](https://github.com/colesbury/nogil-3.12/commit/45bdd27ee5) threading: make _thread.lock thread-safe
- [ ] [`07f5f8c318`](https://github.com/colesbury/nogil-3.12/commit/07f5f8c318) slice: move slice_cache to per-thread state (#111968)
- [ ] [`ea1160c6d7`](https://github.com/colesbury/nogil-3.12/commit/ea1160c6d7) asyncio: fix race conditions in enter_task and leave_task
- [x] [`212fef480e`](https://github.com/colesbury/nogil-3.12/commit/212fef480e) object.c: fix race when accessing attributes and methods (#111789)
- [x] [`70856f126d`](https://github.com/colesbury/nogil-3.12/commit/70856f126d) asdl: use _PyOnceFlag in Python-ast.c (#111956)
- [ ] [`360a79cb88`](https://github.com/colesbury/nogil-3.12/commit/360a79cb88) socketmodule.c: use relaxed atomics for global 'defaulttimeout'
- [x] [`041a08e339`](https://github.com/colesbury/nogil-3.12/commit/041a08e339) functools: make lru_cache thread-safe (#112070)
- [ ] [`9bf62ffc4b`](https://github.com/colesbury/nogil-3.12/commit/9bf62ffc4b) random: add a mutex to guard random.Random (#112071)
- [x] [`a8251a8d25`](https://github.com/colesbury/nogil-3.12/commit/a8251a8d25) clinic: support '@' syntax for recursive mutexes (#111903)
- [ ] [`ffade9d6f6`](https://github.com/colesbury/nogil-3.12/commit/ffade9d6f6) bufferedio: add locks to make print() thread-safe (#111965)
- [x] [`5b83c16dcd`](https://github.com/colesbury/nogil-3.12/commit/5b83c16dcd) textio: add locks to make textio thread-safe (#111965)
- [x] [`6323ca60f9`](https://github.com/colesbury/nogil-3.12/commit/6323ca60f9) stringio: make stringio thread-safe (#111965)
- [ ] [`f1e4742eaa`](https://github.com/colesbury/nogil-3.12/commit/f1e4742eaa) deque: make most functions thread-safe (#112050)
- [ ] [`78825e0508`](https://github.com/colesbury/nogil-3.12/commit/78825e0508) importlib: fix data race in imports (PyImport_ImportModuleLevelObject)
- [ ] [`2f5c90a284`](https://github.com/colesbury/nogil-3.12/commit/2f5c90a284) semaphore.c: decrease count before release sem_lock
- [x] [`22eca6e215`](https://github.com/colesbury/nogil-3.12/commit/22eca6e215) sha1: make sha1module thread-safe (https://github.com/python/cpython/issues/111916)
- [x] [`ada9b73feb`](https://github.com/colesbury/nogil-3.12/commit/ada9b73feb) _struct: fix race condition in cache_struct_converter (#112062)
- [ ] [`0e0b3899d1`](https://github.com/colesbury/nogil-3.12/commit/0e0b3899d1) signalmodule: fix thread-safety issue on macOS (Unclear if this is still an issue on macOS)
- [x] [`964bb33962`](https://github.com/colesbury/nogil-3.12/commit/964bb33962) json: make JSON scanner thread safe (https://github.com/python/cpython/issues/111928)
- [ ] [`86e7772c64`](https://github.com/colesbury/nogil-3.12/commit/86e7772c64) http: fix dependency on finalization order
- [ ] [`9ab96964e7`](https://github.com/colesbury/nogil-3.12/commit/9ab96964e7) faulthandler: don't dump all threads when running without the GIL
- [ ] [`cff32694a4`](https://github.com/colesbury/nogil-3.12/commit/cff32694a4) test_gdb: skip test_threads when running without GIL
- [ ] [`2ae5ee5ed4`](https://github.com/colesbury/nogil-3.12/commit/2ae5ee5ed4) tests: fix and work around some race conditions in tests
- [ ] [`9f9b3d085f`](https://github.com/colesbury/nogil-3.12/commit/9f9b3d085f) ceval: fix some thread-safety issues
- [ ] [`2a4c17e896`](https://github.com/colesbury/nogil-3.12/commit/2a4c17e896) pystate: move freelists to per-thread state (#111968)
- [ ] [`149ea9dc43`](https://github.com/colesbury/nogil-3.12/commit/149ea9dc43) Deferred reference counting
- [ ] [`7e7568672d`](https://github.com/colesbury/nogil-3.12/commit/7e7568672d) specialize: make specialization thread-safe
- [ ] [`90d34f0d18`](https://github.com/colesbury/nogil-3.12/commit/90d34f0d18) specialize: optimize for single-threaded programs
- [ ] [`42d3e11d8c`](https://github.com/colesbury/nogil-3.12/commit/42d3e11d8c) code: make code object use deferred reference counting
- [ ] [`c9fc49666c`](https://github.com/colesbury/nogil-3.12/commit/c9fc49666c) test: add support for checking for TSAN
- [ ] [`7507a77a98`](https://github.com/colesbury/nogil-3.12/commit/7507a77a98) thread: don't use sem_clockwait with TSAN
- [ ] [`a62d37674c`](https://github.com/colesbury/nogil-3.12/commit/a62d37674c) _posixsubprocess: disable vfork when running with ASAN
- [ ] [`398204d57b`](https://github.com/colesbury/nogil-3.12/commit/398204d57b) object: fix reported TSAN races
- [ ] [`4526c07cae`](https://github.com/colesbury/nogil-3.12/commit/4526c07cae) Disable the GIL by default in `--disable-gil` builds
3 Likes
smontanaro
(Skip Montanaro)
November 22, 2023, 12:43pm
127
Thanks. I now recall that, but must have lost track of it with all the other chit-chat (a couple threads with ~250 posts).
henryiii
(Henry Schreiner)
May 13, 2024, 8:40pm
128
The PEP also lists a HOWTO, which doesn’t seem to be present still: PEP 703 – Making the Global Interpreter Lock Optional in CPython | peps.python.org and don’t see it in the tracking issue. Is there a good place for docs on it, especially how to start supporting it from extensions?
4 Likes
colesbury
(Sam Gross)
May 18, 2024, 3:52pm
129
@ngoldbaum is working on a guide for porting extensions and I am also working on the HOWTO.
11 Likes
@colesbury Has there been any progress on the HOWTO? It would be great to read this, and to share with the community to bring more people in for testing free threaded builds. Thanks for your work!
1 Like
colesbury
(Sam Gross)
October 10, 2024, 3:22pm
131
Yes, there are two HOWTO guides:
4 Likes