Want to understand the significance of GIL and does it replace Mutex Locks?

Python Help
1

is there a need to use mutexs in python ? because of the architecture of CPython (namely the Global Interpreter Lock) you’ll effectively only have one thread running at a time anyway

2

The GIL protects the Python interpreter from messing itself up and absolutely nothing else. You still need mutex locks for most real multithreaded code because it can switch which thread holds the GIL at almost any point. Including half way through something like a += 1.

There is also the (slightly experimental) version of Python without the GIL, where again you’ll need mutexes.

3

In the default (GIL-enabled) build, code running inside the Python interpreter is mostly serialized. I.e. you can’t have concurrent execution since the GIL is held by one thread and all other threads have to wait for it to be released. The GIL is for protecting the Python interpreter itself and Python extensions.

It does not eliminate risk of data races in regular Python code. For example, the following program will crash both in the GIL-enabled and the free-threaded build. It is much more likely to crash in the free-threaded build since the opcodes are actually executing in parallel, rather than the opcode execution being merely interleaved. The time.sleep() is there to make the race more likely but the code can race without it, even with the GIL.

With the GIL, execution in different threads can be interleaved (run one for a while, then switch to another) but you mostly don’t get overlapped (concurrent) execution. Note that the GIL can be dropped at certain times, e.g. when doing IO or when calling certain extensions, and so there is sometimes concurrent execution, even with the GIL.

If you write a C extension, the GIL does protect you from races. It ensures your C functions do not get called concurrently. This youtube talk has some explanations about how the GIL works and what needs to be done to make code compatible with free-threaded Python.

import threading
import time

def worker1(state):
    while True:
        for i in range(5):
            if i in state:
                time.sleep(0.0)
                del state[i]

def worker2(state):
    while True:
        for i in range(5):
            state[i] = True
        state.clear()

def main():
    state = {}
    t1 = threading.Thread(target=worker1, args=(state,))
    t2 = threading.Thread(target=worker2, args=(state,))
    t1.start()
    t2.start()


if __name__ == '__main__':
    main()
3 Likes