After a year of on and off development, I am pleased to present PEP 805.
Abstract
This PEP proposes internal changes to CPython and a new API to support safe, parallel execution of Python. With this PEP, parallel execution of code is race free by default: objects must be explicitly declared to be safe to be shared between parallel threads, or such sharing is prohibited.
This PEP builds on both PEP 703 and PEP 734 to provide a unified execution model that offers better safety than PEP 703, better sharing than PEP 734, and better performance than either of them.
This PEP adds some additional state to each object, so that it is possible to check, at runtime and at low cost, whether an operation is safe and raise an exception when it is not.
I like it. I won’t claim to have fully understood it or thoroughly analysed it, but it largely rings of “doing what I want with few surprises”, which is a good step.
My main desire would be to even further reduce the amount that Python developers need to think about it. The more transparently objects become protected or transferred/claimed when needed, the better.
I’d like to see some small/moderate length samples - at least one in the PEP - doing some kind of task using these primitives. There are micro-samples currently, but nothing at the level that we’d expect people to have to write down. Something like making a set of <waves hands> requests[1] in parallel and updating a central result object would do.
Showing what a user would have to go through to get a regular loop into a parallel loop is key for me. All of this stuff is already possible for someone willing to work hard at it, so the big step is to make it easy while still being safe. (I’d prefer we do the hard work/magic in the runtime than push it onto users.)
And I know I opposed the ABI break in another thread. This looks like it could simply adopt abi3t as a prerequisite, so I’d say that’s your way forward. Don’t let that side of things become the hangup for the rest of the proposal - we’ll sort out the ABI changes if the benefits are compelling enough.
A make_the_request() function call is fine for the sample. ↩︎
In regards to freezing, I think it this PEP should also introduce (or suggest for a near-future PEP) a way to mark a method as Frozen[Self] compatible
i.e.
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def add(self: Frozen[Self], other: Frozen[Self]) -> None:
self.x += other.x
self.y += other.y
def non_frozen_add(self, other: Self) -> None:
self.x += other.x
self.y += other.y
p1 = Point(1, 2)
p2 = Point(3, 4)
p1.add(p2) # OK. A `T` can be used anywhere `Frozen[T]` is used
freeze(p1)
p1.add(p2) # OK
p1.non_frozen_add(p2) # Error. `Frozen[T]` cannot be used anywhere `T` is used
Since this would require a lot of work to update typeshed and 3rd party libraries, some work might need to be done to figure out a gradual way of introducing it. This also might need an “official” way of typing the self parameter of a method, since I think all type checkers assume the type of self is Self (if that is done, then it should also support mixin classes).
Yes please! Anything to get away from C extension authors being expected to Py_BEGIN_CRITICAL_SECTION() ~everything[1] just in case the user decides to pass in an object which they then mutate in another thread.
Even though many libraries are marked as supporting free-threading, it is unlikely that they are all completely safe to use in a free-threading environment given the difficultly of eliminating race conditions.
And note that most of this support has come from a concentrated pool of people contributing patches to major libraries. Very few library authors have shown any know-how or appetite for doing this themselves.
possibly with performance costs to users who knew what they were doing so high that it’s actually faster to switch back to the GIL enabled build and use a single thread than it is to use multiple threads with no-GIL ↩︎
How should the steering council evaluate this given the lack of proof of concept or prototype? PEP 703 had Sam’s nogil fork to point to and that made the discussion much more concrete. I understand that would be a big undertaking and you’d like the Steering Council’s approval before doing the implementation, but doesn’t that make it much harder to evaluate the idea?
I see the PEP has a significant new Python API surface. Is there an accompanying C API? If not, why not? I think in NumPy we’d have to update e.g. ndarray to support the new protocols, in C.
I don’t see a section on adoption. Can you elaborate on how a library like NumPy would adopt this? Is our existing source-level support for free-threading sufficient, or will we have to do things to update our existing support to allow users to use the new interpreter you’re envisioning?
What object state would a frozendict with mutable values-- e.g., lists --have? The PEP says, for instance, dict.__freeze__() will be added, but it doesn’t say what happens if any value in the dict is not immutable. Is that considered immutable enough to be considered immutable? Is that only considered at the time of access for the relevant key, although that would seemingly run counter to wanting to avoid access checks as that’s a driving factor for the PEP.
The proposal puts a “deep freeze” out of scope for the intial work, but I don’t see how the proposal could achieve the claimed safety without it. Maybe I’m missing part of the intent here, but this feels like it comes up short in ways we’ve even seen discussed before about various other tangentially related proposals (and specific issues that kept those from being pursued further).
I’d like to be wrong about this, because if such a proposal can materialize into something usable, then it’s easier for people to write the right code, but I don’t see how this proposal could accomplish it with what it leaves out of scope.
Mark sees ‘shallow freezing’ as minimally sufficient for the purpose. He lists ‘deep freezing’ (PEP795) as a possible future enhancement. EDIT: the difference is that between copy and deepcopy. Shallow copies are often sufficient.
I really like this proposal, but would like to suggest a few points:
The del as an expression is definitely better than tuple alternative; but have you considered a builtin called transfer that might read better, eg channel.put(transfer(x)). Could del/transfer be a separate PEP, it has many other uses?
The ref count is highly dependent on compiler optimisations. Not sure how this will affect your proposal. Are there cases when the ref count is really 2 but shows as 1 because of compiler optimisations? EG if the object is passed through a function but object is in a cpu reg already and therefore ref count not incremented by function call.
I can’t think of a better alternative to the proposed way of sub-classing frozen classes. But I feel the proposed method with a frozen bool arg is a little clunky. Can anyone think of a better way?
Agreed, but I’d still prefer us to figure out how to be clever about this, and maybe formalise checks for “this is the sole reference to the object”. The caller shouldn’t have to worry about it, and since we’re doing parallel stuff there’s a very high chance that the local reference is immediately going to fall off anyway. Detecting multiple references on the other side is easier than scoping on the local side.
I’d rather explain to people that their multithreaded code gets faster when they drop all references (but otherwise works) than explain that you can’t even start multithreading until you learn to apply this magic function at all sorts of places.
Please don’t downplay the disadvantages though.
The Backwards Compatibility section seems incomplete:
Compared to the default build, the only incompatible change is that the lifetimes of some objects (those of primitive types) may be extended, possibly increasing memory use.
The PEP preview lists several incompatible changes, such as:
This PEP will require a one time ABI breakage
The __kwdefaults__ attribute [of functions] can still be changed, but only by re-assigning the whole object, not mutating it.
[some objects] may not be reclaimed immediately if there are no more references to them.
Not to mention things like a builtin function that changes an object’s type in-place. While that’s opt-in from CPython’s point of view, it’s asking library maintainers to make potentially backwards-incompatible changes in order to get the benefits.
The reference passed to protect must be the sole reference to a local object, or a ValueError is raised.
Does this mean that e.g. a debugger taking a snapshot of locals() might cause protect to fail?
It is an error to call acquire or release on a protective lock. Such a lock can only get acquired by using a with statement with that lock, or a compound lock formed from it, as the context manager.
Does that mean you can’t call __enter__/__exit__ directly, either?
Feels like Shareable.LOCAL &c. should be available from somewhere (and added to the new API list).
sys.main_group should probably be sys.main_thread_group.
The __shareable__ flag reminds me a lot of Rust Sync trait which describes at compile time whether a type is safe to access in multiple threads concurrently. It should be pretty straightforward for PyO3 to automatically mark types as local or shareable based on this to avoid asking all users to make an unsafe decision.
I am curious about the motivation for having a semantic difference between synchronized and immutable. It seems like in both cases it’s ok to share objects between threads, does the VM optimize differently based on the distinction?
Similarly why do we need separate synchronized builtin types? Why can’t we re-use the existing containers but promote them to shared (which would be a one-way promotion)?
As you say, providing a reference implementation would be a big undertaking. We wanted to get some community feedback and some positive signals from the SC before committing to it.
Yes, there would be accompanying C-APIs for the new Python APIs. Mentions of this are scattered throughout the PEP, and we didn’t want to write down a full spec yet, but it would be beneficial to summarize them here briefly. There would be C-APIs for:
The case of ndarray is indeed a very interesting example because it would make sense for it to support all possible states. (Unlike e.g. AtomicDict which is an example we looked more closely into, for obvious reasons.)
The smallest change to support this PEP would be declaring all instances of ndarray as synchronized, since they were already made thread-safe for FT. Nonetheless, supporting more states might be an interesting change for Numpy arrays.
If I was freely editing Numpy myself, I’d look into doing something like the following.
Do nothing at __init__, which keeps instances in the local state.
Make use of the assumption that arrays are thread-local by default, changing relevant methods to remove calls into synchronization primitives.[1] (This should give better performance in the common cases, though I can’t tell how much.)
Add a new array.synchronize() method, like we’re proposing for lists and dicts, to change the __class__ of array and replace the methods modified in the previous point (2) to use the thread-safe versions you have now.
Likewise, add a new __freeze__() method to change the __class__ to point to methods that raise exceptions instead of allowing mutations.
The trick of changing the __class__ of strictly local objects is thread-safe because no other thread could observe the change as it was happening. Furthermore, it avoids the need to have checks at each individual method call, which would be somewhat inefficient. (Maybe there’s also some usefulness for ndarray to have different code paths for e.g. immutable arrays?) The main constraint here is keeping the memory layout of all states identical, though I don’t think that would be a significant limitation.
This change for Numpy wouldn’t be small, but note that the biggest change of making arrays synchronized, under the terms of this PEP, was a change already made in support of FT.
While it may be desirable for Numpy to have a non-trivial change such as this (to fully take advantage of the possible performance improvements contained in this PEP), I don’t think many libraries should find the need to do so.
What object state would a frozendict with mutable values-- e.g., lists --have?
A frozen dict is immutable. It doesn’t matter what its key or values are, as the state of an object is “shallow”.
Freezing a dictionary has no effect on the objects in it. It is the same as creating a frozendict with mutable values now; the values are unchanged:
It should be pretty straightforward for PyO3 to automatically mark types as local or shareable based on this to avoid asking all users to make an unsafe decision.
That would be great.
I am curious about the motivation for having a semantic difference between synchronized and immutable.
They are more or less identical, from the point of view of the user and the VM. The big value in the distinction is for optimization. For example, attribute lookup can become constant lookup if the object is immutable.
There are some semantic differences for the C API though. For example, a class with a MemberDef can’t be synchronized as the access would be unsynchronized. If the MemberDef were READONLY, it could be immutable
have you considered a builtin called transfer that might read better, eg channel.put(transfer(x))
channel.put(transfer(x)) is no different to channel.put(x): it still leaves a reference to the object in the local variable x. To transfer a local to another ThreadGroup, there can be no remaining references held by this ThreadGroup.
Right, so my question is what does it look like when you share a frozendict with “local” values to another thread? Is the check for that when the value is accessed from the frozendict, when the frozendict is shared, …? Since deep immutability isn’t directly covered it isn’t clear to me from reading the PEP how mutable objects inside an immutable container get handled (the PEP covers direct sharing but not this sort of indirect sharing).
