Define a concurrency model (aka memory model) for Python

In more detail, the following holds true for lists in the current implementation (iterators are omitted):

Summary
function behavior/order requires notes
list_insert() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#143019
list_clear() synchronized & release (on list->ob_item) & relaxed (on list->ob_size)
list_copy() synchronized & regular (on list->ob_item[i])
list_append() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size)
list_extend() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957
list_pop() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957
list_sort() synchronized & release (on list->ob_item) & relaxed (on list->ob_size) python/cpython#126559
list_reverse() synchronized & release (on list->ob_item) python/cpython#143019
list_index() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i]) [1]
list_count() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i]) [1:1]
list_remove() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957
function behavior/order requires notes
PyList_Size() relaxed (on list->ob_size)
PyList_GET_SIZE() relaxed (on list->ob_size)
PyList_GetItemRef() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i])
PyList_GetItem() regular (on list->ob_item[i])
PyList_GET_ITEM() regular (on list->ob_item[i])
PyList_SetItem() synchronized & release (on list->ob_item[i])
PyList_SET_ITEM() regular (on list->ob_item[i])
PyList_Insert() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#143019
PyList_Append() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size)
PyList_GetSlice() synchronized & regular (on list->ob_item[i])
PyList_SetSlice() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957
PyList_Extend() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957
PyList_Clear() synchronized & release (on list->ob_item) & relaxed (on list->ob_size)
PyList_Sort() synchronized & release (on list->ob_item) & relaxed (on list->ob_size) python/cpython#126559
PyList_Reverse() synchronized & release (on list->ob_item) python/cpython#143019
PyList_AsTuple() synchronized & regular (on list->ob_item[i])
function behavior/order requires notes
PySequence_Size() relaxed (on list->ob_size)
PySequence_Concat() synchronized & regular (on list->ob_item[i])
PySequence_Repeat() synchronized & regular (on list->ob_item[i])
PySequence_InPlaceConcat() synchronized & release (on list->ob_item & list->ob_item[i]) python/cpython#142957
PySequence_InPlaceRepeat() synchronized & release (on list->ob_item & list->ob_item[i]) python/cpython#142957
PySequence_GetItem() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i])
PySequence_SetItem() synchronized & release (on list->ob_item[i])
PySequence_DelItem() synchronized & release (on list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#143019 [2]
PySequence_Contains() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i])
function behavior/order requires
PyObject_Size() relaxed (on list->ob_size)
PyObject_GetItem() seq-cst (on list->ob_item & list->ob_item[i]) / synchronized & regular (on list->ob_item[i])
PyObject_SetItem() synchronized & release (on list->ob_item & list->ob_item[i]) & relaxed (on list->ob_size) python/cpython#142957, python/cpython#143019

The “requires” column lists PRs without which the description is false for ≥3.14.5 in general (but true for 3.15). One PR has already been backported (in 3.14.7), but the other has not yet been backported (see python/cpython#158488).

Whenever list->ob_size is not explicitly referenced, a load operation on it is implied: regular if synchronized, relaxed otherwise. When explicitly referenced for non-size-like functions, a relaxed store operation on it is implied. However, there are some exceptions:

  • Some functions with critical sections, such as PyObject_SetItem() (list_ass_subscript_lock_held()), sometimes use relaxed load operations (PyList_GET_SIZE()) instead of regular ones (Py_SIZE()). Unless intentional, this has only an insignificant impact on performance.
  • Some functions without critical sections, such as list_index()/list_count(), sometimes use regular load operations (Py_SIZE()) instead of relaxed ones (PyList_GET_SIZE()). Unless intentional, this may be a bug (non-atomic parallel load).

  1. calls list_get_item_ref() in a loop over i, retrieving list->ob_item each time, so it races with parallel resizes (not a bug; just possible accesses to different arrays, with the loop breaking when the index goes out of bounds) ↩︎ ↩︎

  2. never actually resizes the list (see python/cpython#158592) ↩︎