Freezing data type for parallel access

Ideas
12

While the protocol proposal could potentially be the same, I think we do need a new PEP to explain what has changed since PEP 351 was rejected. (The python-dev post linked from the PEP describes some still legitimate concerns, but the utility of MemHive’s HAMT mappings also provide a counterexample to the dict case)

It isn’t the same situation as when older PEPs were deferred or withdrawn by their authors.

In addition to the open questions still in PEP 351, a modern PEP would also need to discuss the interaction with type checkers.

Both the default dict use case and the more general “locking a mutable container after populating it” use case would also benefit from the “freeze in place” version of the protocol that makes mutation methods start throwing exceptions rather than removing them entirely.

I also think the relationship to the copy module needs to be considered further:

  • if the base protocol freezes in-place (perhaps even calling the method __ifreeze__ rather than __freeze__), then freeze and its protocol method should be mutating operations that return None
  • by analogy with sort vs sorted, the make-an-immutable-copy version would be frozen. If the relevant methods are defined, that could use the copy protocol with in-place freezing, but __frozen__ could also be implemented directly (either for efficiency or because in-place freezing isn’t supported)
  • the shallow freezing vs deep freezing question needs to be considered (I would suggest shallow freezing by default, with deep_freeze and deeply_frozen as recursive versions)
  • rather than builtins or a new module, this could all be added to the copy module to emphasise the relationship between the protocols, especially if thawed variants are added (I realised that using the copy protocol for thawing doesn’t work when the mutable and immutable versions of a container are genuinely different types). Thawing should always produce a new object, though (due to the concurrency problems with thawing in-place)

Finally, I think the PEP 603 thread at PEP 603: Adding a frozenmap type to collections would be worth reviewing for potential ideas (especially around temporarily thawing objects to more efficiently produce derived versions).

3 Likes