Now, I had a few points I wanted to make after reading the PEP a few times.
Optimistic lock avoidance for lists and dicts
Under this proposal, lists and dicts can be immutable (freezed), local, stop-the-world mutable, and protected, but not synchronized, right? If that is so, does it mean that the optimistic reads detailed in PEP 703 would no longer be necessary? I reckon that would simplify a lot of code inside CPython.
SuperThreads
The name SuperThread doesn’t make me think about GIL-like serialization of threads. Maybe we can bikeshed on a better name? I found SerializedThreadGroup in §With the GIL enabled, that sounds like a better name to me.
About __protect__()
I tried coming up with a simple pure-Python mutex implementation based on this PEP, that would also resemble a Rust mutex. (Granted, it’s very easy to implement a mutex when you already have *.__mutex__.)
I stumbled on this problem: is a previously-protected object allowed to no longer be protected? I think the code below illustrates why it might be useful.
class Mutex:
def __init__(self, obj):
self.protected = self.__protect__(obj)
class UnlockedMutex:
def __init__(self, mx: Mutex):
self.mx = mx
self.__freeze__()
def get(self):
return self.mx.protected
def set(self, obj):
# previous = self.mx.protected
self.mx.protected = self.mx.__protect__(obj)
# self.__unprotect__(previous) <---- is this needed? is it implicit?
@contextlib.contextmanager
def lock(self):
with self.__mutex__:
yield self.UnlockedMutex(self)
Given the above, what would be the result of this code?
my_mutex = Mutex({})
with my_mutex.lock() as unlocked_mx:
current = unlocked_mx.get()
updated = current | {"spam": True}
unlocked_mx.set(updated)
# current is now the sole reference to the empty dict used above
print(current) # is this ok? was it implicitly un-protected? or does it raise?
Another question. This table says “no” under immutable x __protect__().
Does that mean the following code would raise an exception?
my_mutex = Mutex(0)
Yet another question. What does this do?
self.__protect__(self)
Do I take it correctly that it means in order to change the attributes of self, from now on you’ll need to hold self.__mutex__? If so, I guess the Mutex.__init__() above needs it, right?
On the benefits of this proposal
I would like to point out a few characteristics that make this simple Mutex worthwhile, because I think they really are benefits of this PEP.
my_list = []
my_mutex = Mutex(my_list) # raises
# as desired: prevents the reference from escaping the mutex's scope.
# => can't mutate the list without calling my_mutex.lock()
my_mutex = Mutex([])
with my_mutex.lock() as unlocked_mx:
lst = unlocked_mx.get()
lst.append("spam")
lst.append("nope!") # raises: this thread does not hold my_mutex.__mutex__
# again, prevents the reference from escaping the mutex.
I think it would be tremendously useful to have the guarantee of holding the sole reference to an object, especially in a multithreading context.
Furthermore, incrementally opting out of the GIL with “SuperThreads” sounds definitely less scary than switching from all-GIL to no-GIL. For anybody, beginners and experts alike. It would probably help the adoption of free-threading in the long run.