Upfront apologies if this email is late to the party and we’ve overlooked previous closure / guidance on this subject. If there is a resolution here, It’d be very kind if you could provide an authoritative link. I will make sure to share it broadly as I am talking with a few other folks wondering about the same thing.
If possible, perhaps adding a link or documentation to the asyncio docs would be good as well as this is a fairly broadly encountered issue and the hacks being recommended are leading to a lot of problems.
Please See here -
The issue is that folks write code, and then may (or may not) try to execute that code in the context of a repl. That repl may (or may not) have an asyncio event loop already running. Sometimes one repl may not be sufficient, and sometimes folks may decide to move onto another repl.
There are many repls in the ecosystem. They all satisfy different needs and requirements. And in fact, we are writing a new one as well which takes a more visual approach than previous ones. It will not be based on iPython or jupyter.
Repls are critical to the success of python, especially in the realm of data science and notebooks, which has lead to a great deal of industry investment. Usually these types of users are not engaged or concerned about the subtleties of Python itself, they want it to be very clear and just simply work. It usually is and does, which makes Python great for this use case.
But in writing our own repl, we are faced with an issue everyone else is who makes a repl like solution - what is the asyncio contract for executing asyncio code within the context of a repl?
This comes when pasted code samples for various reasons benefit from asyncio. And indeed, for our network use cases we really like coroutines as python threads have issues.
Note that configuring the environment or cherry picking the right asyncio code to execute in the right context is not a user friendly solution. It leads to user surprise and endless support. The user can’t just copy/paste/execute like they usually do in the other countless samples they’re leveraging.
So the extra steps required is not how repls are generally used, and goes against their value proposition of just working. For our product, we’d like there to be a way to call code which will work on both async and sync contexts while minimizing behavioral differences without surprising the user with some special case treatment that is orthogonal to what they are trying to achieve.
Ideally, we would have an agreed upon asyncio API we can run in repls that works in either context.
I’ve seen in a few places folks have solved this problem by fudging their execution environments a bit and suggesting using asyncio.get_event_loop().run_until_complete, however the documentation for asyncio specifically discourages the use of this API. Some environments do sub optimal (IMHO) like allowing await outside the context of an async API. I imagine this will inevitably lead to confusion, though I agree it’s tricky to disallow.
The get_event_loop() solutions out there currently are not very solid and only work in inconsistent circumstances, which is not surprising as there is no contract here. For example, calling asyncio.run() can break the api above.
However, the solution is actually reasonable and we will use the get_event_loop() API if that is the consensus. It supports our users fairly well. We will investigate ways to discourage use of await outside of functions (perhaps just a warning).
So, second question - If using get_event_loop().run_until_complete() is the right way is it possible to add something to the documentation to this effect to provide some reassurance to both repl dev and users that efforts will be made to avoid breaking this?
It can also be used to inform other folks about the ‘right way’. Some maintainers are recommending monkey patching, which leads to poor outcomes.
Even better would be to add implementation support for this, but I don’t have high expectations there. Guidance on how best to support it and what will be forwards compatible would be nice, tho.
It doesn’t have to be forever, but it would be good to have at least some documented if temporary agreement among the language/stdlib designers and the repl community.
If not the right way, what is the way to support non intrusive code?
Fwiw, if there is no consensus opinion on a non intrusive solution, likely we will default and follow suit of other repls and defacto support get_event_loop().run_until_complete(). Not ideal, in my honest opinion, but practical.