Removing the asyncio policy system (asyncio.set_event_loop_policy) in Python 3.15

Async-SIG
1

I have opened a PR deprecating asyncio.set_event_loop_policy and scheduling its removal for Python 3.15. I also plan to submit more PRs deprecating asyncio.get_event_loop_policy and the policy classes

In 3.15 I propose that asyncio.new_event_loop will import and return the new asyncio.EventLoop and asyncio.get_event_loop/asyncio.set_event_loop will be re-implemented to store in a threadlocal independent of the policy system matching the default policy behaviour (unless asyncio.get_event_loop becomes an alias for asyncio.get_running_loop as originally planed)

The current leading usecase for asyncio.set_event_loop_policy, running uvloop, is already deprecated in favour of uvloop.run. The next most popular usecase, using a SelectorEventLoop on win32 is available in the loop_factory kwarg of asyncio.run, eg asyncio.run(main(), loop_factory=asyncio.SelectorEventLoop). A number of patterns rely on the “spooky action at a distance” behaviour of the policy system by setting the policy in one location so that a user in another location will use the SelectorEventLoop without having to configure it, this will still be available by running asyncio.EventLoop = asyncio.SelectorEventLoop of course this is not recommended but naturally will be supported python behaviour.

Here is a link to the PR gh-94597: deprecate asyncio.set_event_loop_policy by graingert · Pull Request #110728 · python/cpython · GitHub

2

Here’s an example of some PRs showing how to avoid the deprecated api by using the new loop_factory kwarg in run:

3

The issue with some discussion. More can be added here.

4

Perhaps the most relevant comment there has this:

Deprecating policies: Yes please. The policies no longer serve a real purpose. Loops are always per thread, there is no need to have a “current loop” when no loop is currently running. The only thing we still need is a loop factory, so perhaps instead of an API for getting/setting a global “policy”, we could have an API for getting/setting a global “loop factory”.

(That’s me summarizing the feeling at the 2022 core dev sprint.)

1 Like
5

What is the (“default”?) event loop implementation on Windows? ProactorEventLoop? Does it implement all features of SelectorEventLoop on Windows?

6

The default event loop on Windows is indeed ProactorEventLoop.

I don’t know how to answer your second question – I believe it can do everything SelectorEventLoop can do, but better, except for things involving Windows “socket file descriptors”. I think the latter is the only reason we’re keeping it around.

7

Without policies, how do I use SelectorEventLoop on Windows? From what I saw, policies were mostly useful to opt-in for SelectorEventLoop on Windows, no? Well, there was also an API to select the “child watcher” implementation. It also goes away, no?

8

You call run with asyncio.run(main(), loop_factory=asyncio.SelectorEventLoop)

2 Likes
9

That’s already deprecated in 3.12 asyncio uses the best available child watcher

1 Like
10

I would like to avoid adding more global things like a global default loop factory, practically asyncio.run it needs to be called only once in an entire application so explicit specifying loop factory seems better.

IMO we need to do all this in a planned fashion like it was done for child watcher which is now entirely deprecated. For policy, first design a way in which current “usual” asyncio users can use asyncio without knowing about it at all, currently if a user knows nothing about child watchers it all magically works “good enough” same should be for policy. Then you can start deprecating all the policy things in one go like getting, setting it and all the classes etc.

11

Most users already don’t know or don’t care about policies. The policy is a way to hold a dynamic global default. Instead, Thomas has introduced a static global default, asyncio.EventLoop. On UNIX this is asyncio.SelectorEventLoop and on Windows it is asyncio.ProactorEventLoop. This comes down to the same default that was being used before if no explicit policy was set.

Requiring users to pass an event loop factory to run() seems overkill. Allowing them has been possible for a while, we just make it the only way to override an event loop. (You’re not supposed to overwrite asyncio.EventLoop, of course.)

12

As a data point when considering this deprecation, I grepped the primary Meta monorepo for usage of (g|s)et_event_loop_policy, and got O(100) hits in the internal codebase and O(300) hits in third-party libraries we use.
It shouldn’t be a huge pain for us to handle this removal, but would be great if:

  1. 3rd party libraries have enough time to remove their usage and release versions that support pre-3.15 python releases (so we could upgrade the internally before we start upgrading to 3.15+)
  2. The docs are clear about how to avoid these APIs where they are used (ideally it would be a mechanical change we can automate with LibCST, bonus points if the changes can be applied to code that uses 3.10+)

For comparison, I see thousands of hits for get_event_loop(, which is already deprecated - this one is going to be a more painful removal


13

can you break down how many hits you get for get_event_loop_policy vs set_event_loop_policy?

can you also see how many hits are get_event_loop_policy().get_event_loop() which was used to incorrectly avoid the get_event_loop() DeprecationWarning?

14

Also asyncio.get_event_loop() is only deprecated when no event loop is set. If a loop is set (running or not) it does not warn. Not sure if this matter, since setting an event loop may also be deprecated in favor of using asyncio.run(..., loop_factory=...).

Thomas, if we’re deprecating set_event_loop_policy(), that means we’re also deprecating everything that comes with policies (e.g. get_event_loop_policy() and the policy classes). Would we still support asyncio.set_event_loop()? How?

I have a feeling that we may be way too optimistic with our deprecation schedule for policies – I’ve already heard people say they want to be able to have code that, without version checks, works on all supported versions, which currently means back to 3.8. And only emitting deprecation warnings for set_event_loop_policy() isn’t going to catch all uses (some people may just be inspecting the policy and deciding things based on what they find there or call its methods).

15

We can make get/set_event_loop still work without policies:

16

sure!
for get_event_loop_policy it’s under 50 for internal and close to 300 for third-party
for set_event_loop_policy it’s close to 100 for internal and under 200 for third-party
for get_event_loop_policy().get_event_loop() it’s only a few for internal and close to 200 for third-party
(sorry it’s all approx numbers, I am unable to share exact numbers)

17

I’m in favor of dropping event loop policies, they always felt janky to me and I never actually needed them since asyncio.run was added.

Why do we need this exactly? So there’s a way to get a platform’s best loop class?

18

Good question. It was a year ago and I don’t recall. More recently I suggested that we might still want to support set_event_loop(), which sets the actual event loop to use, not the class. The platform’s best loop class should be EventLoop, so we don’t need a way to set it.

I guess there’s a general API design issue here where for some reason we tend to like passing a loop factory around instead of a loop object – e.g. the run() function and Runner class do this, even though both of these just instantiate the class once, without additional parameters. Does anyone recall or understand why? There’s a comment in asyncio/runners.py in _lazy_init() that suggests there’s some special treatment for child watchers – but maybe that’s no longer an issue either now?

19

I don’t know if it’s currently true for us, but quite often these kinds of loops have thread affinity for where they start. So if you wanted to change the event loop used on a different thread, you’d want to pass it the factory rather than the loop.

Maybe there are thread locals or contextvars somewhere? Or there could be in a 3rd party loop?

20

As a data point, asyncio GUI integration tools (which need to listen to native windowing events in addition to IO) currently tend to use set_event_loop_policy.

I assume they can adapt to the new way of doing things. But please keep them in mind for documentation & porting notes. It’s not just uvloop & SelectorEventLoop

2 Likes