Hi Mark ![]()
I like this PEP, in part.
I haven’t read it in full details yet, but I can say that I definitely liked the inspirations from the Pyrona project.
The part I’m quite against is the general sentiment that mutable shared objects are just ![]()
![]()
![]()
I think the guardrails you lay down make a lot of sense and I would gladly see them in Python.
The problem is that mutable shared state can only exist, under this PEP, when guarded by a mutex.
Correct me if I’m wrong, but the special synchronized state is only assignable to built-in objects.
That would completely stifle development of concurrent data structures (that is, shared mutable objects) that can ultimately provide more thread safety guarantees and better multithreaded scaling than their sequential counterparts that have to be mutex protected.
I know this is a small niche, but it is my niche, so I’ll be vocal about it ![]()
We briefly talked about this issue at the last PyCon US, did you give it any more thought?
Honestly, I can only see AtomicDict working under this proposal if I can set a trust-me-bro flag/state, otherwise a concurrent hash table would be pointless to develop.
I wouldn’t like a solution where we say: we have concurrent data structures in the stdlib, so we don’t need this flag.
Because, even if we did, someone ten years from now could come up with a novel, better approach for some concurrent data structure, and there should be no reason why they can’t have that published on PyPI and be meaningfully useful.
Please, let me know if I’m missing something from the PEP that would enable this.
I know the counter argument is: how do I, the VM, know that what is marked as trust-me-bro can actually be trusted?
On one hand, we’re all consenting adults here.
On the other, we may wish to be stricter in order to provide more guards against data races, but as you write in the PEP, it “eliminat[es] most race conditions.”
For instance, if I read the ... in ThreadSafeList as including a __setitem__(), then it’s easy to write the usual my_not_so_thread_safe_list[item] += 1 data race.
If instead it’s an append-only list, then we can see how a programmer might rewrite it with __setitem__() and fall down to the same mistake.
(Better clarity in the examples would help.)
Under this PEP, data races can still exist at the upper level, where the mutexes are acquired and released, possibly at the wrong places.
(I’m saying this to elicit the synchronized state be available to normal humans as well, not because I think the PEP is unsound. I like the PEP and I believe it should move forward.)