PEP 734: Multiple Interpreters in the Stdlib

I briefly skimmed through the PEP. One thing I have in mind: instead of hardcoding the list of types that support sharing, would it be possible to make this a protocol? Something like __reduce__, but for in-process sharing.

Also, I think interpreters.Queue deviating from the threading and multiprocessing queue semantics by only allowing shareable objects will be annoying users. Perhaps you could have two queue classes, for example a LowLevelQueue for shareable objects only, and a regular Queue that would use pickle for most objects?

Finally, I think a benchmark of sharing vs. pickling/unpickling would be useful to assess the benefits of the sharing mechanism. Right now it’s not obvious whether this really can make a difference.

(note that with pickle 5 out-of-band buffers, you could perhaps simply share those buffers while still using pickle for most datatypes?)

5 Likes