Is `asyncio.to_thread` always threadsafe?

Hello! I am building a system that has already developed TCP/IP clients using the socket module. For new clients and for the overall system, my plan is to adopt asyncio. To call the existing clients, the method that I planned on is using asyncio.to_thread.

My question is: Is asyncio.to_thread always threadsafe? In the sense of, can it be used to call non-threadsafe methods on a class?

My understanding is that the default executor for asyncio.to_thread is concurrent.futures.ThreadPoolExecutor](concurrent.futures — Launching parallel tasks — Python 3.12.2 documentation), which is obviously a pool or collection of different threads. So my assumption is that asyncio.to_thread will not run on the same thread everytime. The only way that I know of to do this is to create a custom ThreadPoolExecutor with max_workers=1, however, that greatly limits the threads that can be used with asyncio.to_thread.

But, it seems that since await asyncio.to_thread(...) itself runs on a single thread, will call out to a separate thread, block that thread until it is returned, that if the class it is calling a method on is not running on any other thread, then the call would be threadsafe.

Can someone please help me to check if my understanding is correct? The main context of my question is centered around leaning on asyncio.to_thread to turn existing blocking clients into asyncio clients.

Thank you!

It looks like you’ve figured this out for yourself already: The function passed to to_thread() will be submitted to an executor, and then it will run on some thread (but never the thread that is running the asyncio loop). So if you call to_thread() twice before waiting for the results using await, the calls may run on two different threads (in the same executor), without further synchronization.

I wouldn’t recommend using a custom executor with max_workers=1 – while that serializes the calls, if that’s what you’re after, you could just as well create your own lock and use that to serialize calls (writing a simple wrapper around to_thread()).

4 Likes

Thanks for the confirmation and the clarification! I’ll give using this to asyncio-ify some existing TCP clients, so I will see how it goes.