Support for running async functions in sync functions

Async-SIG
#1

Feature or enhancement
Support for running async functions in sync functions
Pitch
Currently only running async functions inside sync functions is only available with
asyncio.run(func(x)) or loop.run_until_complete(); however when there is a running loop, these calls cannot be used.
The only way is running the function in background, and using callback functions(which is very limiting)

event_loop = asyncio.get_event_loop()
task = event_loop.create_task(func(*args))
task.add_done_callback(callback_func)

See example PR

A bit Related:

1 Like
#2

You could use janus to send and receive arbitrary objects between a thread and another thread that runs an asyncio event loop.

Here is a real-world example: backend.ai/session.py at main · lablup/backend.ai · GitHub
Session is a synchronous version of AsyncSession and uses a separate thread to run the asyncio operations. In the perspective of its caller it looks like just a set of synchronous blocking APIs. (The purpose is to keep a single internal async implementation but to provide both sync/async interfaces.)
_SyncWorkerThread is the sync-to-async task thread implementation.

The above implementation does not use janus because we assumed no concurrency is required for the client-side sync wrapper. But if you replace queue with janus and add another queue for receiving results, you can achieve concurrency in the worker thread.

#3

That PR does not give me any idea about the practical use case. Are you trying to run a coroutine function from an asynchronous callback, or worker thread? Which is it?

#4

Could you take a look at this issue and let me know if there is anything unclear?