An ABC for locks

Ideas
1

There are a few types in the stdlib with an identical locking interface (acquire, release, etc). Unfortunately, users can’t subclass some of these types because these are actually factories, not real types. For example, the following isn’t allowed:

class Spam(threading.Lock):  # TypeError!?
    pass

Similarly, this breaks some duck-typing when using locks with type annotations:

def spam(lock: threading.Lock):
    ...

spam(multiprocessing.Lock())  # Type checker error!

So, if the user wants to accept any generic locking interface rather than a specific lock type, they would have to define their own protocol or ABC for that. My proposal is to standardize this: it would be nice if there was something in collections.abc that defined the actual lock interface.

1 Like
2

threading.Lock is no longer a factory but a regular type in Python 3.13. It’s just that you cannot make subclasses: Make `threading.{Lock,RLock}` factory functions subclassable

1 Like
3

I don’t think an ABC for locks makes sense. The minimum most locks provide is… they’re contextmanagers. not all contextmanagers provide locking, and different locks provide different behavior. While the standard library locks might provide .acquire/.release, it is reasonable that 3rd party locks would insist they only be used as a context manager and not provide these.

From a type system perspective, if you’re allowing any lock, including those provided by users, you don’t want to mandate more than the minimum, but you can’t guarantee that the minimum has any kind of locking behavior associated with it.

If the intent is that you accept anything that can be used as:

def spam(lock: ...):
    with lock:
        ...

you don’t need a lock ABC

from typing import ContextManager
from contextlib import nullcontext

def spam(lock: ContextManager = nullcontext()):
    """
    Parameters
    ----------
    lock: ``ContextManager`` :
        An optional context manager which will be used to ...
    """
    with lock:
        ...

you can then document parameters appropriately.

Note: both this and the proposed ABC would be unable to handle both threading locks and asyncio locks simultaneously.

1 Like
4

Hmm, do you have an example of a library that only exposes the context manager without acquire and release methods?

In my experience, it’s a bad idea to expose only half the interface, because a). you sometimes need to hold a lock for something that doesn’t fit in a context e.g. a lock that needs to be held for the duration of a class b). context managers aren’t very fun to work with from the C API, so you generally resort to acquire and release there and c). acquire normally has a timeout parameter – there’s no way to specify that from a context manager.

AFAIK, there’s only one asynchronous locking interface: asyncio.Lock, which can be subclassed anyway (maybe trio has one, I don’t know) – I don’t see the need for an ABC specifically for that. But maybe it is a good idea to have a collections.abc.Lock as well as a collections.abc.AsyncLock.

5

That’s the thing though, it’s not half the interface, that’s the intended exclusive interface in the case I’m aware of. The intent was to force it’s use as a context manager. So an ABC that assumes .acquire/.release would exclude this, even if the function using that type only ever used it as a context manager. The involved code isn’t public, but was done this way for reasons similar to why everyone encourages with open(...) as f: rather than f = open(...), context managers are very good at helping avoid mistakes with this and there were repeat cases of broken lock invariants (resulting in both permanently held locks, and locks that were acting as n=2 semaphores unintentionally), due to such errors.

As for the timeout, One I’m aware of in a private corporate code base, the lock was implemented such that both of the below were valid.

with lock:
     ...

and

with lock.timeout(10):
    ...

I also don’t agree that there are things that don’t fit into a context, but there’s a lot more to that which gets more into my issues with people getting stuck in very rigid OOP design patterns when python supports a mix of this and other patterns which when mixed appropriately, may be better suited to their use[1]

An ABC which assumes a specific interface wider than used conflicts with the concept of duck typing. If you only use it as a context manager, some of the below would be things that have nothing to do with the “duck type” of it.

class ThreadingLockLike(Protocol):
    def __enter__(self) -> bool: ...
    def __exit__(
        self, exc_type: Any, exc_val: Any, exc_tb: Any
    ) -> None: ...
    def acquire(self, blocking: bool = ..., timeout: float = ...) -> bool: ...
    def release(self) -> None: ...
    def locked(self) -> bool: ...

Everything you specify beyond your actual use is no longer duck-typed. This is instead prescriptively assuming what a lock should be rather than typing your required interface. Such an ABC might have value to someone, but it’s not the duck type of a parameter in a function taking a lock that uses only some parts of the above.

Even in the protocol here, I’ve left out the precise typings of exc_type, exc_val, and exc_tb, because the types of those will never matter to you using it as a context manager while remaining blind to the actual concrete implementation. I’ve also left off the undocumented acquire_lock, release_lock and locked_lock methods that you can find on threading.Lock

  1. It’s my opinion that every python developer interested in concurrency could benefit from a week of Elixir, if for no other reason than exposure to the idea of functions being the top level and data being passed as needed, such a design works wonderfully with context managers to lock shared resources even in languages with mutable data structures which might need explicit synchronization such as python ↩︎

6

Personally, I’m not a fan of with lock.timeout – I don’t really see the benefit over acquire (and it’s also an uncommon interface, which if I’m accepting a “generic lock,” such as with an ABC, then quite a bit of support is lost).

I guess this proposal comes down to a philosophical question in typing: should you specify only exactly what you need, or use a type that gives you a little more leverage? For example:

def add_one(obj: Incomplete) -> None:
    obj.append(1)

Do you annotate obj as any object with an append method that takes one argument, or do you simply make it a list?

If the precedent for the stdlib is the former, then I agree, an ABC for locks is no good! But otherwise, I could see it being useful. Generally, I would think that if you want a lock, you shouldn’t allow things that aren’t locks (i.e., no acquire and release) to be passed.