Exactly
I believe bubbling up that functionality to Python code would be beneficial.
Many other languages have them, there’s no reason why Python shouldn’t.
(The example you brought up is also implemented in AtomicInt64 where += x
is an atomic lock-free operation.)
Totally agree
I was also considering having an AtomicNumber
, instead of having AtomicInt
/ Float
/ Complex
/ MyCustomNumber
.
Python is not Java, we don’t necessarily have to restrict an atomic numeric class to one kind of numbers.
Do you perhaps have any thoughts on this idea?
[nit]
That wouldn’t implement a behavior like cmpxchg
, but more like xchg
.
Also, that would not be atomic (here atomic is the proper word), despite it being a one-liner and looking like it’s atomic, in the same way as C’s a++
is not atomic.
If a and b were shared and other threads were operating on them, this swap would not be thread-safe.
Furthermore, it’s entirely possible that the upcoming JIT will try to be clever optimizing that line, producing weird results for other threads observing a
and b
.
Thank you for sharing that!
I actually had no idea multiprocessing.Manager
existed
For this typical setup of yours, do you have any thoughts about free-threading?
It looks to me like exactly the sort of hoops you would go through just to circumvent the GIL.
Here I think AtomicDict
would be a nice fit as well.
That is, you could rewrite it like this:
from cereggii import AtomicDict, NOT_FOUND
from threading import Thread
import time
...
def manager_dict_update(proc_num, shared_dict, start, end):
# Perform heavy computation on a subset of the range
result = count_primes_in_range(start, end)
# Update shared dictionary with the result for this process
# shared_dict[proc_num] = result # this would work, but
shared_dict.compare_and_set(key=proc_num, expected=NOT_FOUND, desired=result)
# with AtomicDict.compare_and_set you either get:
# - the desired update (if proc_num not in shared_dict), or
# - an exception otherwise
# raising here might be helpful to catch a bug somewhere else
# Benchmarking function
def benchmark():
range_size = (RANGE_END - RANGE_START) // NUM_PROCESSES # Divide range into equal parts
shared_dict = AtomicDict() # Shared dictionary for results
processes = []
for i in range(NUM_PROCESSES):
start = RANGE_START + i * range_size
end = start + range_size if i < NUM_PROCESSES - 1 else RANGE_END
p = Thread(target=manager_dict_update, args=(i, shared_dict, start, end))
processes.append(p)
start_time = time.time()
for p in processes:
p.start()
for p in processes:
p.join()
manager_dict_time = time.time() - start_time
...
Do you have any thoughts?
No, it actually doesn’t.
Most methods of AtomicDict
are lock-free and don’t block other threads.
I’ll make it explicit in the documentation, thank you for bringing it up!