What is the Long-Term Vision for a Parallel Python Programming Model?

Ideas
41

OT, but I think the original inspiration for rayon was Cilk, admittedly a somewhat obscure reference. rayon predates JuliaFolds by at least a few years.

1 Like
42

OK, maybe I understand better. Still not enough to answer your questions directly, but perhaps this is the kind of info you wanted?

(All of this is my understanding; I’m not an expert.)

    1. Done is printed after 10 seconds.

  • 2-7:

    Per PEP-703:
    For lists & dicts, getting obj[x] and setting obj[x] each lock obj, and each is atomic (since every relevant operation on obj also uses that lock, or an equivalent).

    For regular attributes, getting/setting obj.attr is equivalent to getting/setting obj.__dict__['attr']. (obj.__dict__ itself is a descriptor, not a regular attribute.)

    x.y += z and x[y] += z consist, in your examples, of three atomic operations: get, add, and set. (Get and set are atomic for built-in collections, as above. Add is atomic for integers: it reads immutable objects and produces a value that’s not yet shared.)
    (For other types, += could be two operations: get and inplace-add.)

    self.list[n] += 1 is four operations: getattr, getitem, add, setitem. Each is atomic for built-in containers and simple classes.

    The VM is free to optimize, if it can maintain the illusion of the above. Note that yet another thread could, for example, modify the class so that accessing self.list now runs arbitrary code with an arbitrary result.

2 Likes
43

Notably, this is NOT a single atomic operation, meaning that the three separate actions can be interleaved (and thus having a too-low total). You can’t corrupt internal state with it but you can miscalculate external state.

1 Like
44

Ok. Of course this puts strong constraints on what optimizing compilers are allowed to do. For instance on a JVM-like memory model, the read would be considered loop invariant and moved out of the loop.

Ok. From this I take there is an intuition of operations being broken down into atomic steps, which are by themselves safe, but can interleave possibly arbitrarily. Is this a fair representation of your point of view?

This is me mostly trying to categorize what the current understanding and expectation towards these programs is to get a feeling of what might be desired/expected from Python implementation.

45

From my point of view, it’s a very complex topic, and I try to get some intuition where people are with their expectations. I thought @encukou’s response was in a way useful, because it gives me one personal point of view. Others might have different thoughts or perhaps a different vision of what would be useful. For instance, seeing everything as steps of atomic operations that might interleave arbitrarily might not be a programming model people find easy to use and reason about.

1 Like
46

And in general it can’t be atomic, because once we’re not dealing with simple builtins, each of the operations can involve calling arbitrary Python code. And Python doesn’t e.g. know that x.y is an int before executing the x.y.
Of course, an optimizing compiler can do lots of magic in avoiding unnecessary unlock-lock pairs, but I don’t think such optimizations should be part of the language guarantees. (And in practice, optimizations only kick in after a warmup of a few repetitions.)

Yes.
But, determining which steps are atomic is based mainly on maintaining invariants at the C level (the hold “don’t corrupt the Python VM state”). As you point out, it’s not very practical as a programming model.
As far as programming models go, bring your own locks or don’t share mutable data would work much better.

47

Python already places constraints just as strong on optimizing compilers. For example, consider this non-threaded version of your original code.

from sneaky import BaseClass

class Example1(BaseClass):
    def __init__(self):
        self.keep_looping = True
  
    def thread1(self):
        while self.keep_looping:
            pass
        print("Done")

Example1().thread1()

Will this print Done? Will it loop forever? If I provide the file sneaky.py, would I be able to ensure that this does actually loop for ten seconds and then terminate?

Answer: yes. All it needs is for keep_looping to be a property.

48

Ehm, no, not really. Sure, if your sneaky BaseClass turns keep_looping into a property the behavior and optimization opportunities are different. But if the optimizer can proof, typically after inlining the property accessor that there’s no write, or more generally state change inside the loop body, in a single threaded world, it can move things out of the loop.

I put this example in mostly as a demonstration of the pitfalls. Not being able to move a read out of a loop can be a major blocker for many common optimizations, which is why many languages require other means to indicate that a read needs to stay in a loop. I am not saying these languages are right, I am just trying to raise awareness of the tradeoffs of desired properties.

1 Like
49

Is this true even in the presence of signal handlers? It’s not as cross-platform, but the same thing can be done like this:

import signal

class BaseClass:
	def __new__(cls):
		self = super().__new__(cls)
		signal.signal(signal.SIGALRM, self.halt)
		signal.alarm(10)
		return self
	def halt(self, sig, frm):
		self.keep_looping = False

(Since I didn’t have any super().__init__() in the original code, I stuck the magic into __new__, but the point remains.)

Signal handlers provide a measure of concurrency already. So optimizations that assume that there are no other interspersions of code are going to be, at best, fragile. Yes, there are optimizations that you can’t do, but that’s the nature of Python - everything IS dynamic, for better or for worse. Anything that isn’t a fast local can be manipulated externally.

1 Like
50

Well, since there’s no spec, it’s all a question of what semantics one wants/thinks is correct.

Since CPython 3.10, the only check point for signals is the loops back edge, as I understand it.

Depending on the compiler/language to take as reference, there are all kinds of options. JVMs would typically make an informed guess of how likely the loop is going to be problematic, and if it is found to be not so, for instance because it’s trivial and has a loop bound, it would remove such check points from the loop.
In our specific case, it might decide to indeed leave the check for the signal in, but because of JVM memory model semantics, I would still be free to move the read out of the loop.

51

I’ve had a chance to look over these prompts, and since you’ve said you’re interested in people’s expectations I’ve tried to answer based on intuition, without “thinking about it too hard.”

I would expect that “Done” would be printed after about 10 seconds.

I’d want for thread1 and thread2 to run in parallel without any need for synchronization. This seems reasonable to me since they are not mutating the same attributes.

I would say thread1 and thread2 run in parallel without any need for synchronization, since they do not write to the same indices and the structure of the list doesn’t change.

Same as (3) and for the same reasons. But “ thread1 and thread2 only need some minimal check that the dictionary is “consistent”” is also interesting. I suppose that would come down to the cost of the checks vs the benefit of the guarantees provided but I’m not sure how to further evaluate it.

Without the experience to know any better, “ thread1 and thread2 run their loop, without any need for synchronization, and only need some form of synchronization to add the fields” sounds excellent to me. I can’t comment on how practical it is to implement it.

How terrifying. :grimacing: At this point if the runtime just made sure the lists were “consistent” it would make sense to me, and “on your own head be it” for not synchronizing around mutations.

This does give me pause about my own answers to earlier questions.

I suppose “ thread1 and thread2 only need some minimal check that the dictionary is “consistent”, but thread3 and thread4 need to synchronize somehow before accessing the dictionary.” (Again, without any insight into how practical that actually is.)

Well, off the top of my head…

  • I don’t want multithreading to be able to crash the interpreter
  • It would be great if objects wouldn’t become corrupted such that they are no longer readable
  • Without knowing how practical it would be in such a dynamic language, I think it’s really nice when my tools can tell me if I’ve introduced a data race

I don’t know how helpful any of the above will be to you, but this has certainly been food for thought and a nice reminder of how much more there always is to learn. :slightly_smiling_face:

2 Likes
52

I would like to revisit @ntessore’s points, that

I think this is somewhat different from the main discussion here. In particular, I wonder if it calls for any additions/changes at the syntax level? OpenMP, in particular, uses C language #pragma which doesn’t have a direct equivalent, although it’s possible that it could be handled through a combination of decorators and the context handler mechanism.

Is this appropriate here, or should it be split off into a separate thread?

53

I think designing new syntax is premature optimization at this point. We need to see how people use free-threaded Python for a little while first.

Particularly for the scientific Python stack, I expect it will take some time to really incorporate the changes, but I also expect eager adoption[1]. A lot of extensions were written because of the gil. Some might be rewritten in Python, others might be simplified, still others may need substantial changes because they relied on internal behavior that needs to be changed.

Given the implementation schedule and support for previous versions, it might take years before new syntax could reasonably be designed.

As you point out, a lot can be done with decorators and context handlers. So the ecosystem can develop without syntax changes.

  1. if this group doesn’t embrace free threading it’s a big warning sign for pep 703 ↩︎

3 Likes
54

+1. I would love to see an ecosystem of free-threading libraries that implement parallelism under PEP 703. Even if they required some compromises to avoid needing new syntax, I’d be fine with that initially. Once we know what works best, and free threading is the standard build, not an option, then let’s think about whether adding syntax for parallel programming is worthwhile.

8 Likes
55

Yes; but I also have my eye on a quite different field: network servers, notably web apps. Typically, a simple web app written in Python will be single-threaded, multiplexing incoming requests as it can, but handling only one request at a time - and to handle more requests, you need something that juggles processes rather than threads. I’m very interested to see whether tools like gunicorn decide to switch to a multithreaded model instead, and what the consequences are on things like database connection pools.

2 Likes
56

You’re right about rayon being first, thanks! I should have been more clear that JuliaFolds was a reference for high-level (Pythonic) parallel programming, not for inspiring rayon in particular.

I had no idea about Cilk, though! Thanks for the link–I always thought Clojure transducers were the first implementation of this.

1 Like
57

GIL was sparing us thread complexity. How to stay safe without a new protective syntax, enforcing good practices from the start of this new era ? kittens may be armed, eco-system may fork into an hydra of several incompatible Way-of-Working-through-it, …

58

Applications that are IO bound have been solved well with green threads and/or asyncio.

It’s compute-bound applications which could benefit from no-GIL.

1 Like
59

That is a fallacy, the GIL did not give you thread safe code.
You always needed to use locks to make your code thread safe.

3 Likes
60

Thank you for starting this thread @smarr.

I find that even in the current GIL world, the state of the python thread programming model is very frustrating for a user. Every time I try to write a piece of threaded code I come to the realization that it is impossible to write correct threaded python code, simply due to the fact that no description exists for what that would be. The best I can hope for is to produce code that appears to work today on my system and hopefully works tomorrow somewhere else. For something as complicated and difficult to test as concurrent programming, this is very unsatisfactory.

So, given that the free threading model will likely make it more difficult to write working threaded code, my expectations are:

  1. Clear, unambiguous, specification for (formulated in terms of python source code, not byte code or other implementation details that are non-obvious to the programmer at the source level):
    a. How python defines a race condition
    b. What are the possible outcomes of a race condition
    c. What is required to avoid a race condition
    d. What all of the above truly means when interacting with non-trivial python objects

  2. Concrete guidelines and best practices for how to do this in real code

  3. That there is a high chance that any moderately competent python programmer can successfully follow these guidelines in practice.

You will notice that I don’t specify any expected concrete behaviour above. That is because I expect I will try to avoid those details as much as possible, regardless of what they may be. My expectation is that due to pythons complex and dynamic nature and lack of truly immutable objects, I would find it almost impossible to exploit such details correctly. Operator overloading, exceptions, object patching, etc can cause arbitrary code to run at even the most innocuous statements.

I suspect my own guidelines will end up trying to mimic process isolation for the threads as much as possible, something like:

  1. Minimize shared state to a brutal degree, copying objects where needed/possible. I would try to avoid even read-only sharing of any complex object because I would find it extremely difficult to make sure the access was truly read only (hello memoization, caching, lazy evaluation, statistics, logging, …). I would probably only trust read-only access to list, dict, tuple, str, etc. for which I would assume a clear specification would exist.
  2. Communicate through message passing as much as possible.
  3. When I run into the limits of what can be achieved with the above, add one big lock so that I don’t have to think about what the exact semantics are when more than one thread is accessing complex objects.

I guess my implicit assumption above is that at least this would be safe and correct in all cases, which may be naive of me, given the complexity of python code.

Oh, one last thing: I would assume the standard library to thoroughly test the concurrency guarantees of all objects (true, randomized, stress testing with varying load characteristics, exception injection, etc). This does not seem to be the case today where things like concurrent.futures doesn’t seem to have any true testing, only some “kick the tires” unittests. I find this slightly horrifying…

1 Like