Support for running async functions in sync functions

Feature or enhancement
Support for running async functions in sync functions
Pitch
Currently only running async functions inside sync functions is only available with
asyncio.run(func(x)) or loop.run_until_complete(); however when there is a running loop, these calls cannot be used.
The only way is running the function in background, and using callback functions(which is very limiting)

event_loop = asyncio.get_event_loop()
task = event_loop.create_task(func(*args))
task.add_done_callback(callback_func)

See example PR

A bit Related:

1 Like

You could use janus to send and receive arbitrary objects between a thread and another thread that runs an asyncio event loop.

Here is a real-world example: backend.ai/session.py at main · lablup/backend.ai · GitHub
Session is a synchronous version of AsyncSession and uses a separate thread to run the asyncio operations. In the perspective of its caller it looks like just a set of synchronous blocking APIs. (The purpose is to keep a single internal async implementation but to provide both sync/async interfaces.)
_SyncWorkerThread is the sync-to-async task thread implementation.

The above implementation does not use janus because we assumed no concurrency is required for the client-side sync wrapper. But if you replace queue with janus and add another queue for receiving results, you can achieve concurrency in the worker thread.

That PR does not give me any idea about the practical use case. Are you trying to run a coroutine function from an asynchronous callback, or worker thread? Which is it?

Could you take a look at this issue and let me know if there is anything unclear?