PEP 703: Making the Global Interpreter Lock Optional

[h-vetinari]

Thanks for the response!

@colesbury

PEP 683 tries to be compatible with the stable ABI (abi3) and makes some trade-offs to support that, such as putting the immortal bit as the 2nd highest bit. […] I don’t think PEP 683 should be changed – I think it’s a good solution given the constraints of ABI stability.

So if I’m reading this (and the PEP) correctly:

• nogil needs both biased & deferred reference counting (for thread safety resp. performance)
• nogil needs a break of the stable ABI^[1]
• it’s thus independent of PEP 683, but would replace(?) PEP 683’s changes in this space with its own

@colesbury

Regarding the build modes: I wrote this in the PEP, but my hope is for the two build modes to be temporary. I think having two build modes – for a short period – is worthwhile because it reduces the risks during integration into CPython and gives a bit more time for extensions to adapt.

Not to nitpick, but the linked section says “These may be worthwhile if a longer term goal is to have a single build mode”, which is… at odds… with “for a short period”. BTW, this choice is a really hard problem, and it’s fine that the PEP doesn’t really commit to a timeline, but IMO those trade-offs (additional build mode – what would that imply over short/long term? – or new default) could use some more discussion.

@colesbury

I don’t really see the build modes as a means for extension authors to “opt-out” of supporting it, although it’s possible it’s used that way. Like you, I think there will be pressure from users on extension authors to support it, and I think that pressure will be because the users see value in running without the GIL.

I absolutely see libraries opting out of this, at least for a while (e.g. “making our library threadsafe needs a large overhaul of core component X, and we’ll do it in the next Y months”), which will create a situation where people want to install a given library on the nogil ABI, only to find out that it doesn’t exist (or worse: since there are no nogil-wheels, it falls back to trying to install from source) – would pip even be able to take the python ABI flavour into account for the resolver, resp. give a reasonable error?

It’s also going to create confusion for users who have no idea what an ABI is, when a library advertises itself as “we support Python 3.12 (but not nogil)”, and then users need to understand that distinction.

The least painful scenario I could imagine (very subjective, obviously) is to redeclare Python 3.13 as 4.0, no additional build mode but with changed ABI, merge nogil immediately after branching 3.12 (and publish an alpha right after), then work with the extension ecosystem over the following year to get as far as possible in getting the most used libraries “4.0/nogil-ready”. I know you have some experience with that already with the libraries you patched for the existing nogil fork, but it’s gonna take a while for the ecosystem to digest such a change IMO (whether with a separate build mode or not^[2]).

---

1. that in and of itself probably makes it “Python 4.0” material? ↩︎

2. I just think a changed default will cause less friction overall, rather than subjecting the library maintainers (as well as tooling & infrastructure) to having to deal with both modes + users who can’t wait for nogil either way. ↩︎