# Asyncio RLock - reentrant locks for async Python

**URL:** <https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509>\
**Category:** Ideas\
**Tags:** asyncio\
**Created:** [November 29, 2022, 9:12am UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509 "2022-11-29T09:12:57Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![robsdedude](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/robsdedude/32/9990_2.png) [@robsdedude](https://discuss.python.org/u/robsdedude)\
**Post date:** [November 29, 2022, 9:12am UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/1 "2022-11-29T09:12:57Z")

</div>

I’d like to re-open the discussion started here [https://github.com/python/asyncio/issues/439](https://github.com/python/asyncio/issues/439), which requested reentrant locks to be added to asyncio.

GvR argues that locks in eventloop-based concurrency are less used because the context switching happens more controlled, and I agree. But there are still use-cases where a complex state change in a component involves I/O, requiring async locking (and no-one argued against that).

What I don’t understand is how those observations make a point against RLocks in async. It’s true that RLocks are not a necessity. But that holds for async as well as threaded concurrency. Their just a convenience. You could always refactor your code to not need them ([example](https://github.com/python/asyncio/issues/439#issuecomment-254776968) given in the same issue). In the most extreme case, that refactoring could mean implementing your own RLock, which isn’t hard and only requires 1. a simple mutex/lock 2. a way to identify the current thread/task.

So they’re not necessary, but they do help to reduce the complexity of code and make it more robust, concise, and easier to reason about.

I ended up implementing my own async RLock and it was a somewhat alien experience having to do that because I’m very used to Python being that language that comes with batteries included.

Worth noting that I’m not the only one that felt the need for async RLocks. There’s even a package to ship this bit of code: `asyncio-rlock` (imagine the link to pypi here, I was only permitted to post up to 2 links)

---

<div class="post-metadata">

**Author:** ![asvetlov](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/asvetlov/32/55_2.png) [@asvetlov](https://discuss.python.org/u/asvetlov)\
**Post date:** [November 29, 2022, 9:46am UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/2 "2022-11-29T09:46:41Z")

</div>

Any functionality in stdlib has its own maintenance cost.  
If a third-party library on PyPI solves your needs – that’s fine, please use it.  
When we start feeling the high demand for the library, this situation can be a reason for embedding it into stdlib.

For asyncio-rlock I see very few usages on the github. Maybe the feature request is not very popular?

---

<div class="post-metadata">

**Author:** ![robsdedude](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/robsdedude/32/9990_2.png) [@robsdedude](https://discuss.python.org/u/robsdedude)\
**Post date:** [November 29, 2022, 12:42pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/3 "2022-11-29T12:42:55Z")

</div>

> For asyncio-rlock I see very few usages on the github. Maybe the feature request is not very popular?

Might be. Or maybe people just wrote their own implementation. Or they swallowed the pill of having to refactor their code to work with simple locks.

Looking at [Code search results · GitHub](https://github.com/search?q=AsyncRLock&type=code) you can see 4 libs or so that rolled their own solution. Possibly more that I didn’t find because they could’ve named their class whatever.

Given that it’s not a lot or complex code and there is some (hard to measure) demand, I think it would be a cheap and worthy addition to `asyncio`.

To elaborate on my use-case: I had a decently sized, sync code-base that I wanted to port to async Python. The code used `threading.RLock`. So I had the option to rethink and refactor the whole thing to only use simple locks or write my own RLock. As said it felt very awkward to have async ship without batteries.

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [November 29, 2022, 11:17pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/4 "2022-11-29T23:17:58Z")

</div>

Given the async code is not using threads you do not need a thread type lock.  
You can use normal python variables you hold state that you check.  
Complex I/O needs one or more state machines I find, not locks.

If you do have threads then you could use the thread locks in your async code. But must use mechanisms that allow communication with the threads that do not block the event loop.

---

<div class="post-metadata">

**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\
**Post date:** [November 30, 2022, 1:39am UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/5 "2022-11-30T01:39:59Z")

</div>

To anyone who thinks locking in asyncio is simple, check out the git history of [asyncio.Semaphore](https://github.com/python/cpython/commits/main/Lib/asyncio/locks.py).

---

<div class="post-metadata">

**Author:** ![ConnorSMaynes](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/connorsmaynes/32/10002_2.png) [@ConnorSMaynes](https://discuss.python.org/u/ConnorSMaynes)\
**Post date:** [November 30, 2022, 2:33pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/6 "2022-11-30T14:33:59Z")

</div>

I have created an `asyncio.RLock` implementation internally to avoid adding another external dependency. It is widely used to avoid all the _shadow methods_ (i.e. `remove()` and `_remove()`, with `remove()` being locked because it is public and `_remove()` being used internally with one surrounding lock used by the caller) you get when using a regular `asyncio.Lock` to avoid deadlocking.

I completely agree with @robsdedude that similar reasons for needing this apply both in `thread-land` and `task-land`. `asyncio` copied nearly every primitive and method interface. Leaving this out of `asyncio` seems a bit unintuitive and inconsistent.

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [November 30, 2022, 4:50pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/7 "2022-11-30T16:50:21Z")

</div>

I guess i am missing something.  
In my mind locks only have meaning between threads.  
Why does an event loop want a lock at all?  
Surely that will block the event loop defeating the purpose of async?

---

<div class="post-metadata">

**Author:** ![ConnorSMaynes](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/connorsmaynes/32/10002_2.png) [@ConnorSMaynes](https://discuss.python.org/u/ConnorSMaynes)\
**Post date:** [November 30, 2022, 5:20pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/8 "2022-11-30T17:20:24Z")

</div>

> Why does an event loop want a lock at all?

For the same reasons you might want locks in threaded code: to protect critical sections of code which might lead to a corrupted state otherwise.

Here is a (somewhat long) example where an async lock is useful.

In the output, you can see how publishing of the `create` message completes after publishing of the `delete` message when no lock is used. When a lock is used, the messages are published serially.

```python
import asyncio

# lets say we have a collection of resources and a pub/sub system
# which we want to send all signals related to those resources to

# lets also say that we want all messages (created, deleted, updated, etc.)
# to be sequential for each resource so that, for example a deleted message
# does not arrive to some subscribers before the created message.
# to do that, we use a lock for each resource.

async def publish(msg, delay):
    print(f"starting publish: {msg}")
    await asyncio.sleep(delay)
    print(f"ending publish: {msg}")

async def create_resource(resource_lock):
    print("create - wait lock")
    await resource_lock.acquire()
    print("create - acquire lock")
    try:
        print("create - resource")
        # once we have created the resource, we have to finish publishing
        # or the other parts of the system wont know about it
        # we have to hold the lock or publishing of the created message
        # may interleave with publishing of the deleted message
        await publish("create - resource", delay=3)
    finally:
        resource_lock.release()
        print("create - release lock")

async def delete_resource(resource_lock):
    # the lock is outside the shield so we can still cancel waiting on it
    # if there is contention with the lock. maybe we come back later.
    print("delete - wait lock")
    await resource_lock.acquire()
    print("delete - acquire lock")
    try:
        print("delete - resource")
        await publish("delete - resource", delay=2)
    finally:
        resource_lock.release()
        print("delete - release lock")

class DummyLock:
    async def acquire(self):
        ...

    def release(self):
        ...

async def main():
    print("\nwith lock...")
    resource_lock = asyncio.Lock()
    creator = asyncio.create_task(create_resource(resource_lock))
    await asyncio.sleep(0)
    deleter = asyncio.create_task(delete_resource(resource_lock))
    await asyncio.wait([creator, deleter])

    print("\nwith no lock...")
    resource_lock = DummyLock()
    creator = asyncio.create_task(create_resource(resource_lock))
    await asyncio.sleep(0)
    deleter = asyncio.create_task(delete_resource(resource_lock))
    await asyncio.wait([creator, deleter])

asyncio.run(main())

```

Output:

```auto

with lock...
create - wait lock
create - acquire lock
create - resource
starting publish: create - resource
delete - wait lock
ending publish: create - resource
create - release lock
delete - acquire lock
delete - resource
starting publish: delete - resource
ending publish: delete - resource
delete - release lock

with no lock...
create - wait lock
create - acquire lock
create - resource
starting publish: create - resource
delete - wait lock
delete - acquire lock
delete - resource
starting publish: delete - resource
ending publish: delete - resource
delete - release lock
ending publish: create - resource
create - release lock

```

> Surely that will block the event loop defeating the purpose of async?

All the `asyncio` primitives are specially designed using futures to avoid blocking the event loop. When one of them would block, a future is created in the background and the event loop moves on to other tasks and comes back to the task when the future is done (cancelled or result is set).

If you were to use `threading` primitives in `asyncio` code, that would block the event loop, but that is why `asyncio` has its own version of almost every primitive from `threading`.

---

<div class="post-metadata">

**Author:** ![Rosuav](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rosuav/32/3429_2.png) [@Rosuav](https://discuss.python.org/u/Rosuav)\
**Post date:** [November 30, 2022, 6:10pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/9 "2022-11-30T18:10:21Z")

</div>

> [@ConnorSMaynes](#):
>
> For the same reasons you might want locks in threaded code: to protect critical sections of code which might lead to a corrupted state otherwise.

Notably, they are only important in asyncio code when the critical section includes an await point, in contrast with threaded code where everything is an await point. So that makes them _less_ important, but still of value.

Recursive locks, then, are only important if (a) the critical section must be entered by at most one task at a time; (b) this critical section can be entered by the same task more than once, which isn’t a problem; and (c) there are await points within this critical section. Rare, but definitely possible.

---

<div class="post-metadata">

**Author:** ![EpicWink](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/epicwink/32/17968_2.png) [@EpicWink](https://discuss.python.org/u/EpicWink)\
**Post date:** [November 30, 2022, 9:26pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/10 "2022-11-30T21:26:47Z")

</div>

If it’s common for users to write their own recursive lock, perhaps it makes sense to include some links to some third-party libraries providing RLocks in the documentation for [`asyncio.Lock`](https://docs.python.org/3/library/asyncio-sync.html#asyncio.Lock).

---

<div class="post-metadata">

**Author:** ![JoshuaAlbert](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/joshuaalbert/32/12921_2.png) [@JoshuaAlbert](https://discuss.python.org/u/JoshuaAlbert)\
**Post date:** [June 15, 2023, 12:25am UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/11 "2023-06-15T00:25:45Z")

</div>

Here’s a well-tested standalone implementation of a fair reentrant lock for conncurrent programming with asyncio. [GitHub - Joshuaalbert/FairAsyncRLock: This is a well-tested standalone implementation of a fair reentrant lock for conncurrent programming with asyncio.](https://github.com/Joshuaalbert/FairAsyncRLock)

Disclosure: I’m the author too.

---

<div class="post-metadata">

**Author:** ![mattyoungberg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mattyoungberg/32/25275_2.png) [@mattyoungberg](https://discuss.python.org/u/mattyoungberg)\
**Post date:** [January 16, 2025, 6:09pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/12 "2025-01-16T18:09:41Z")

</div>

I wrote up a small class to show how a reentrant lock could be useful, only to see how easy it was to solve my problem. So I’m now empathetic to the argument that a reentrant lock isn’t strictly needed.

However, Python’s big strength is its usability and the “batteries included” approach alluded to in this thread.

In my head, I already knew how to solve my concurrency problem if I only had a reentrant lock. It took me some extra time to figure out how to solve it without an reentrant lock.

That “extra time spent” figuring out how to do this alone would hopefully be a strong enough argument to merit serious consideration of adding a reentrant lock. It’s a common sychronization primitive, even included in the `threading` library. There may be a maintainability cost, but if absorbed, it would make asyncio development more accessible in the end to programmers that already have a decent grounding in concurrency.

---

<div class="post-metadata">

**Author:** ![robsdedude](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/robsdedude/32/9990_2.png) [@robsdedude](https://discuss.python.org/u/robsdedude)\
**Post date:** [October 22, 2025, 9:11pm UTC](https://discuss.python.org/t/asyncio-rlock-reentrant-locks-for-async-python/21509/13 "2025-10-22T21:11:02Z")

</div>

Please excuse the “necromancing” of this thread. The thing doesn’t let me go and pops back into my mind every now and then.

I’ve further refined the GitHub search and looked at it again: [Code search results · GitHub](https://github.com/search?q=AsyncRLock+language%3A%22python%22+NOT+is%3Afork&type=code)

Over the years it has become more. At the time of writing, the search yields 81 hits. Granted, some of these are duplicates as people have checked in their venvs 🙃, but then again, like mentioned above, this search is very fuzzy anyway as people can call their implementation whatever and I decided to just search for the most obvious name that came to my mind.

On top, there’s [`fair-async-rlock`](https://pypi.org/project/fair-async-rlock/) with some mid-single digit thousands of downloads a week (trending upwards) and if nothing else 6 stars on GH.

I think why this doesn’t let me go is that I learned Python as this lovely language that comes with batteries included… But here the battery is missing and I feel like I’m writing Go or something where I have to take care of crafting my own primitives. It’s a feeling in the end, and with my nearly 20 years of Python experience this one feels surprisingly unpythonic to me.

Now, I don’t want to just be demanding things here. I’m more than happy to give something back to this wonderful community (in fact I already have contributed to Python’s ecosystem across different projects). Would somebody be willing to consider a PR if I craft one and potentially help/guide me through the process if I get stuck?