Make asyncio eager task factory default

Not sure where to reply…

I expect an important use case will be to have a library that works with lazy and eager tasks. This helps with transitioning applications to eager tasks. It would also help with other event loops.

But this use case really requires using the task factory or another bit of public writable state attached to the loop, not a different way of spawning tasks.

I would also avoid changing create_task to improve usability as part of this project. Do that as a separate project if you want.

Not sure if I understand you well.
Say, aiohttp (both client and server) works well in both eager and lazy mode. Well, actually two very minor tests out of 3500 required a little tune to be eagerness-independent; the actual code is not changed. I’m pretty sure that the same is true for other aio-libs libraries; I’ll check it quickly for libraries that actively create tasks. Maybe, aiokafka will be another good candidate.

Or did you mean something else?

1 Like

Yeah, exactly that. It works because you don’t have to change every create_task call in the library. So we should keep that property, not require create_task(…, eager_tasks=True).

That’s was my reason for initial proposal: switch to eager tasks ultimately and quickly.
Now I see guys who expressed their objections; I respect their voices.
Perhaps I don’t feel the real pain until the global switch happens.

Anyway, from my experience the switch is more or less safe. Say, my job’s code (about 30 micro services and 20 internal libraries) shouldn’t suffer.
I will finish the switch eventually, and report the result. Pretty sure, I’ll not find any issue except maybe very trivial fixes for a couple of our tests

Maybe we need to do some research exercise? Say, take a FOSS library and check how it’s test suite works with eager tasks enforced.
Very many libs don’t create any task, especially client ones.
I’d like to check FastAPI and it’s basement Starlette.

If somebody wants to help – you are welcome. This work could be done even without a collaboration with libraries owners if we can run corresponding test suite locally.

Maybe Grand Majors did this task for their proprietary code already? Facebook guys and others, I very appreciate if you share your experience.

The amount of create_task()lines is surprisingly low in every particular project.
Say, aiohttp creates a task per request to server, but is is located in the only place.
A crawler can spawn a lot of tasks, but again it is most likely done by the only code line.

Task groups are a little different but still the exact amount of create_task() lines is very low usually.

There were two pieces of feedback in this thread, raised by folks other than me:

  • Even if we switch the default to eager tasks in the future, the ability to create lazy tasks will still be valuable. This could imply we would need the ability to pick at the task creation site, presumably.
  • Eager tasks somewhat break the asyncio contract, because the current task gets switched out outside of a suspension point.

How we proceed depends on what we think of this feedback. I’m personally neutral on the first point, and I think the second point is valid. What do you think, @guido ?

2 Likes

(TLDR: Keep lazy tasks the default forever, encourage use of eager tasks, add eager_tasks flag to run() but not to create_task().)

Having initiated, helped with, and observed many, many of these “should be nearly 100% compatible” changes, I believe we should be very conservative. I don’t know how far back in terms of Python versions libraries like aiohttp or anyio or other libraries built on top of asyncio go, but in other areas I’ve often hear from library maintainers who insisted that a certain way of spelling some thing should remain valid (with the same semantics) for the entire range of Python versions they support, and preferably 1-2 versions more, before they are willing to change their code to use the new spelling across all versions. And nobody wants to maintain support for both spellings, even if it could be done using some helper function (always better than in-lining a version check).

This means we can’t require them to use eager_tasks=False anywhere until all Python versions they need to support have that flag.

I’m not actually sure how being able to pick at task creation helps any use case, so I’m so far neutral on the first bullet as well. Someone who has a need for this should raise their hand and explain their use case (and we’ll have to guess how real that use case is, since this is all hypothetical). AFAIK Instagram, who spearheaded eager tasks, didn’t give their developers a choice.

There was mention of wanting to start with eager tasks but going back to lazy tasks during shutdown. I honestly didn’t understand the reason for doing that, but as long as we can still call set_task_factory() with the lazy task factory at that point, the requirement is satisfied.

All in all I currently support the proposal someone made several posts back: keep lazy_tasks as the default “forever” but encourage using eager tasks to all new users, and encourage libraries to fully support (and test!) both.

PS. Whenever I hear someone say both “let’s switch ASAP” and “I only had to change a few tests out of thousands” I cringe a little bit – those few tests that didn’t work represent many hours of debugging for every developer whose code happens to set up the same scenario as one of those tests (but much more complicated – a failing unit test is much easier to debug than a mysterious occasional application error, whe all you know is that it worked in 3.12 but occasionally broke in 3.14).

Also app developers may skip multiple Python releases since they can’t defend the effort of testing and upgrading to their management, making them miss deprecation warnings. (We can sort of morally oblige libraries to support every version, but not developers of proprietary applications.)

3 Likes

Ok, let’s be conservative. I understand the reason and I agree with it.
eager_tasks flag for runners is a good step forward anyway.

1 Like

I think an eager_tasks flag for runners is an unusual api, it would be better to go via the existing loop_factory api. eg calls will be asyncio.run(corofn(), loop_factory=asyncio.new_eager_tasks_loop) there can be a create_eager_tasks_loop_factory that let you use it with uvloop: asyncio.run(corofn(), loop_factory=asyncio.create_eager_tasks_loop_factory(uvloop.new_event_loop))

it’s annoying to add in uvicorn a cross product of options of loop factory and eager_tasks, when we can just accept loop factory (which is currently in a PR)

1 Like

Yes, that’s true. Cinder 3.10 bakes asyncio eagerness very deep into the runtime, making it a non-optional feature.

Within Instagram, and all other internal users of Cinder 3.10 across Meta, I am not aware of any issues we observed as a result of the difference in semantics.

I was actually thinking about what would be the cleanest way to make the eager task factory enabled by default for all of our 3.12 applications. While adding flags to asyncio.run would be more convenient, it would still require users to maker a code change to opt-in to this behavior, which doesn’t scale well when there are hundreds of thousands of different applications (and perhaps more concerning, would likely lead to diverging application runtime vs test environment semantics, as those would be configured in different places).

I was considering patching our 3.12 to add a -X flag and/or environment variable to make eager task factory default – would that be something that upstream might consider?
With something like this, it would be trivial for me to globally enable eager tasks at the build-system layer, for example.

1 Like

Ok, so let’s agree we need to support lazy and eager tasks forever. Now the questions are about the defaults and the coarseness of the configuration.

I was a little tepid on my suggested approach, which is to add an additional (async) method to taskgroups that would spawn eager tasks, because it makes the api larger and so increases cognitive complexity. But if we’re keeping both lazy and eager tasks forever, the complexity is already there, just hidden behind esoteric, coarse apis. So I think we should reconsider this approach.

Why give the ability of choice? It makes the configuration very fine grained, allowing libraries and applications to opt into it at their own pace, if they want and where they want. It makes what’s going on explicit, minimizes the blast radius and nudges codebases towards task groups.

I would also start recommending against the eager task factory, so TaskGroup.create_task would still be guaranteed to spawn lazy tasks.

That does sound annoying. OTOH your proposed spelling feels esoteric and not very user-friendly, if we want to recommend this. How hard is it really to maintain that matrix of options? (I don’t know anything about uvicorn, maybe you can explain more details?)

Hi Tin, I’m sorry, but I don’t understand what you are proposing concretely. Could you clarify?

I had been thinking the same thing (an env var). But maybe it should only affect run() calls that didn’t explicitly pass eager_task=True or False? Or maybe encode in the env var whether to force or to suggest.

1 Like

One option is that asyncio.TaskGroup can default to creating tasks that are eager or lazy based on the parent task’s eager or lazy status

Or we could store the eager/lazy flag in a contextvar and use that?

The task factory approach would still work but could be phased out.

Instead of a flag on runner it would be a context manager that could be used outside run, or before a task is created or around a TaskGroup

I don’t think the singular task factory is the wrong approach here. I wouldn’t want to add branching at every task creation. I haven’t heard anyone need the ability to select which per-task.

1 Like

I see, you prefer very fine-grained control, whereas the original implementation (in Cinder 3.10) chose for global control (if that – but presumably much of Instagram’s code could also still run with the original CPython 3.10?)

I think I would prefer very course-grained control (just in run() or with an env var, keeping the factory) but I could be swayed if you presented a realistic use case. (Or is this your way of avoiding having to provide explicit support for this in uvcorn? In that case please explain or link to docs or code that would be affected.)

The use-case is various, but particularly in anyio’s start_soon where we want to keep trio’s lazy semantics, but don’t want to introduce a 2 event loop cycle delay to lazy tasks, and want to keep task.get_coro() working. We could work around this, perhaps the minimal change is having some way to know if the task will be eager before we create it

I also think 3.14 is probably too early to do any of this. I think I’ll try to get as many suites onto 3.14 and swapping out the use of policy for loop_factory as possible, then running all suites testing both eager and lazy using a parameterized fixture. Then once we have some research on what changes are needed there we can work out what to do here?

1 Like

I’m not sure I’m following your use case. If the application has set an eager task factory, why would anyio not respect that?

The eager_task=True API doesn’t exist yet (and will never exist in 3.12), so I didn’t really consider that :slight_smile: If this API lands soon in main, and the patch is manageable, I would definitely consider backporting it to our 3.12, as well as an env var based configuration once we decide on the exact spelling (maybe that should be a separate discussion).

We never tried that, so I can’t say for sure, but my intuition is that this is mostly the case (maybe except for some places that implicitly rely on inlined comprehension semantics that we had in Cinder forever, and were upstreamed in PEP-709).

We do run the Instagram Server test suite and canary with a “minimally-modified Python 3.12” (most significant patches are Lazy Imports, and asyncio awaiter tracking), and I think it works with both lazy and eager tasks (I can verify this after the holidays).

Just wanted to chime in. AnyIO’s test suite has 14 tests that fail with the eager task factory. A handful of them fail due to the assigned task name not being set, and the rest (from what I can tell) fail due to the unexpected task scheduling order. Fixing the former may require the issue to be fixed upstream. I’m not sure how to go about fixing the latter to work regardless of task scheduling order.

My exact proposal is this:

  • We leave lazy tasks as default
  • We do not implement any more coarse-grained switches
  • We stop recommending switching the task factory to the eager one
  • We add an additional method to TaskGroup, async def create_and_start, that can be used to create and start eager tasks. We start recommending it over TaskGroup.create_task

That’s it.

You yourself expressed concern about edge cases. I think this is the only approach that avoids the chance of blowing up downstream libraries and codebases that spawn tasks. If we don’t do this, every task-spawning library will need to test against both types of factories, forever. This approach allows libraries to choose which they support and use, and migrate over time.

Bonus points: since the new method would be async, we do not break the asyncio contract about switching tasks without a suspension point.

1 Like