asyncio and generally async programming is meant to break down a long running IO-bound operation into a series of small tasks that can be scheduled and executed concurrently on an eventloop. The above mentioned methods do not benefit from this since their runtime execution speed is about as constant O(1) time as it can get.
For example, len()just returns a length counter on a data structure, it doesn’t go about counting items in a data structure itself. setattr just sets an attribute of an object, its as fast as running an assignment a = 1. The rest of the methods fall under the same catergory as well, they don’t apply to async programming at all. They can just be used the say they are since, as you mentioned, they’re implemented in C which is fast enough.
There’s already a known pattern for this. The library can continue maintaining its synchronous code base if providing first class support for it will introduce strain on the maintanence part. You as a developer relying on this sync library can run the library’s functionality in a different thread from the one executing you asyncio eventloop. You can do this by using asyncio.to_thread. This will keep you application in async mode while running the sync code in a different thread.
It is theoretically possible for these to involve network traffic, though - ORMs often map operations like these into queries. (For example, len(some_table) might run a query to SELECT COUNT(*) FROM table_name and return that.) I would, however, strongly recommend against the use of setattr in this way, as it’s extremely confusing for what looks like simple attribute assignment a.b = c to perform database actions. Plus, it’s very common to need to change multiple attributes at once. This is why a lot of ORMs have some sort of .save() method to actually apply the changes.
Agree completely. For example, you wouldn’t want a mysocket.recv() in your async defed function. What I’m proposing is to remove the need to have every function call be an awaited one from top to bottom of your Task’s stack (I’ll get onto why asyncioio.to_thread doesn’t always help in a moment).
Other than the speed improvement from your function calls being functions calls rather than awaited coroutines, and the less typing, you are able to use the overrides Python provides - @property, __getattr__(), __setattr__(), __len__() etc without a thought about whether somewhere deep in your call stack there is an await myfuture. Consider your example:
This is, for ORMs like Django, an await-needing operation - the __len__() override refers to the database to count the selected records. __getattr__(), similarly, may go to the database to retrieve a record, and need to await.
Not needing to be in an async defed routine to await also means you can usually wrap or patch your blocking low-level thing (socket, for example (see my earlier comments), but also Lock comes to mind) with asyncio-aware versions of its routines. So a blocking thing becomes a non-blocking thing. As several people have commented, you then need to be more aware of inter-Task interactions - I’m sure any asyncio user would have those in mind anyway.
Let’s look at asyncio.to_thread(). This runs sync code on a thread in a thread pool. The two problems with this are: you’ve only got a limited number of threads; you’ve got a deadlock hazard. The thread limit may not be a problem for your use cases, but in a web server where you’d like to be able to serve 1000s of pages concurrently, having a 32 thread bottleneck is a problem. The deadlock hazard comes from the limited pool to threads, say, 32. It happens when the 32 running jobs are all waiting on a resource to be released, which can only be release by the 33rd job running, and that can’t run because the thread pool is full. This deadlock is not theoretical - it happens in practice.
Fair, unless you’re aiming for a website which can scale to XXL. In my use case there’s a bunch of requests.post()s too (mostly inter-server API calls).
Which gets to where I’m at with asgi-ing my website. Had a thread-based website using Django. Realised some parallelism would be good to improve page serving speed. asyncio was the way to go. Try converting part of my code, and realised asyncio is unusable ‘as is’ due to the function colouring requirement. Take a half year to convert cpython to not care about the function colours, to show it can be done (didn’t think it’d take quite that long). Which gets me to today - persuade the Python community this light-touch adjustment to the language gives big benefits (surprisingly hard so far!). Next steps: get a PEP for it accepted; work to integrate into cpython; continue with website (with requests and psycopg2 non-blocking).
… which is why I’ve not async-ed my requests calls.
I suspect you could have converted your website in much less than half a year. Suggesting that maybe you directed your energy at the wrong target…
By the way, I assume you are fully aware that Django has at least some level of async support. I have no idea how relevant that is to you, but I think it’s important to point out that using asyncio in a Django website certainly isn’t “unusable” in the general sense, so the problems you have are likely related to your own codebase rather than the libraries and framework you’re using.
What makes you think this is a “light touch” change? Have you even considered how much documentation and course material would need to be changed? Have you estimated how much effort libraries like trio, anyio, httpx, etc., would need to invest to adapt to your proposed change to the core language? Especially if they want to continue to support older versions of Python from the same codebase (which they will!). On a note that might affect you more directly, how long do you expect Django to take before it supports your new feature? Have you looked into what would be involved? As the author of the proposed language change, and a user who has a direct need for this in Django, preparing at least a proof of concept Django PR adding support for this seems like a pretty reasonable thing to expect you to provide.
A PEP would absolutely need answers to the questions I raise above, so you should probably start doing that research now. And you should also be aware that a PEP which makes a change this significant to the language[1] could easily take another half year, quite possibly longer, to develop and get into a state for pronouncement - with no guarantee of acceptance even then.
Some other questions a PEP would need to answer:
How does this proposal interact with the free-threaded build of Python? Or for that matter with threading in general?
Why does this proposal not have the problems that previous attempts like stackless Python had?
The implementation might, in your view, be straightforward, but the impact is definitely significant! ↩︎
When I read this, I felt that your motivation and execution where on different pages, and in that, your choice of solution was also off. (edit): For example, could you share what exact aspect of your page serving needed parallelism or concurrency?:(end edit). Asyncio doesn’t run tasks in parallel. As such, you might have been using the wrong solution for the problem.
Asynchronous programming invites you to think about programming in a different way. It might not always be about sprinkling coroutines around and expecting a performance improvement. Without proper structure and architecture, your async code will run just the same as your sync code thus not ripping the benefits that come with async programming.
I’ve not seen your code but I’ve encountered similar complaints many times with developers that haven’t grasped how to program asynchronously. So I believe its less of “changing python to do something” and more of “learning how this thing is actually done”
Maybe, maybe not - it’s quite a large code base, and so it isn’t clear which would have been quicker.
Yup - that’s where I started, before concluding asyncio ‘as is’ and Django don’t mix well.
All existing code should work, without change (see my very early note on incompatibilities in the proof-of-concept. These can be addressed).
The new feature is: await, async for and async with in def myfunc() (non-async) functions now generates working code where they used to generate syntax errors. That’s the change on the Python side. There’s quite a lot more on the C side so that these non-async tagged functions work (only await myfuture needs to be in an asyncio.Task). In doing that, I’ve also aimed for those changes to have as small an impact as possible on extensions (mostly none, but there are some cases where more work needs to be done).
I have to admit not, beyond knowing that would be have to be part of getting it ready.
Yes: none. See earlier - all existing code should work unadjusted. I know that, until tested, you can never be certain - make test being clean, though, is a good start. The tricky bit for library maintainers will be writing new sync functions which have async elements, whilst remaining backwards compatible.
You’ve got a point - I’ll add that to my list. Before really looking into the code, I know I’ll need to look at: psycopg and psycopg2 non-blocking; simplifying the sync/async transitions; connection management (to make sure it’s async happy). Non-blocking sockets will be part of cpython-await-anywhere, so HTTPRequest and HTTPResponse and very likely to work ‘as is’, although the asgi/request/response interface might need simplifying.
Many thanks, also, for your PEP advice, and libraries to try to make sure they’re OK.
Seems OK so far (make test MacOS debug has 3 fails, but they seem related to closing the asyncio loop from a Task - I’m working out whether the difference in behaviour is driven the free threading or my work).
Different approach to parking a coroutines. Not having read stackless’s code, I’m guessing it made sure as much as possible went onto the Python datastack, and kept the C stack un-nested (stackless), so parking the coroutines only needed the datastack to be switched. That’s part of how I park coroutines, and I add managing the C stack so that you can switch to different C stacks at the same time (different parts of the standard C stack). This might sound like a hazard for cpython’s cross-target-compatibility, but since there’s no assembler, and only well supported, C standard library functions are used in a way the standard allows, I thought it was probably OK.
Including code that relies on the fact that calling a function that wasn’t defined with async will never do a context switch? Yes, that code won’t break until the called function has an await added to it, but that’s still a breakage - especially in code that takes arbitrary callbacks from user code.
Honestly, you’re being incredibly naive here in your interpretation of what people would consider “breaking” existing code.
Have you tested on Linux and Windows as well? What about other platforms like iOs or Android, or emscripten? Or platforms that, while not technically supported by the core team, Python is still used on (like IBM - zOS, I think)?
I started to write out some of the difficulty of having long jump based coroutines without undefined behavior across supported systems, but it’s more of a distraction when the core idea has some issues, so I want to focus on that first.
This and similar topics have come up before. There have been other implementations proposed before. Attacking the function color problem by eliminating the barrier on where await can be used won’t ever work; The reason we have async and await as syntax is to indicate what functions are capable of yielding to the event loop to facilitate reasoning about concurrency.
Which means it’s not a “light-touch”, it’s undermining the entire reason we have async.
We had coroutines, event loops and green threads before we had async. async didn’t directly enable anything new there, it’s syntax sugar and an effective boundary for context switching that enables people to write more complex concurrent programs while still holding all of the possible context they need to know in their head.
I don’t think the current async documentation helps people learn enough to hit the ground running right now, and I don’t want to sit here and blame you when this is the efforts of being frustrated at it not being obvious how to leverage async.
If your first thought was add async everywhere as it looks to be from the examples you’ve given, then let’s talk about the purpose of async: it’s to cooperatively schedule other work. eg. “I’m waiting on IO, let another task do it’s thing”.
In this example, you went with treating each << as needing an await of it’s own, resulting in 3 awaits. Is the expectation that you have 3 separate writes to the stream with a full trip around the event loop between each one? Does anything else running on the event loop hold a reference to that stream that might end up with some interleaved content then?
as Chris said, asyncio streams typically have a non-async def write with an async def drain. When you’re working within the bounds of “you can’t await outside an async def function”, you sometimes don’t have to worry about synchronizing access because other tasks aren’t allowed to be scheduled if you just don’t await between writes that need to be in order.
The await is there for letting other work happen during IO, it doesn’t help when placing values to later write it into a some backing deque, as that’s a cheap operation.
If you allow await in any function, a bunch of currently safe code is no longer safe without introducing explicit synchronization. Even attribute access might yield back to the event loop, causing other parallel tasks doing writes to be interleaved unexpectedly, and you have to inspect everything from operators to properties for it.
If you await only where you want other work to be allowed to happen rather than awaiting everywhere, function color becomes a benefit.
That’s not entirely true. The way we got here historically is that back
when I invented yield from, one of the uses I had in mind for it was
getting a form of green threads without requiring platform-dependent
hackery. The fact that you had to use special syntax for calling
suspendable functions was something that I regarded as a necessary evil.
However, it turned out that some people saw it as a virtue rather than
an evil, and it stuck around.
Personally I’d be quite happy to replace the entire async/await
infrastructure with a single-colour green threads system, but that seems
unlikely to happen, because the people who like the two-colour system
will always be opposed to it.
At some point though, the system has to know which functions run eagerly and which can be suspended. A vocabulary to distinguish the two is necessary. Not only for differentiation purposes but also to choose the right execution path for each. Even the languages like Go with builtin support still use the go keyword to schedule and run these goroutines, which is similar to await. The alternative route would be to use stackless, which offered a rather mechanical way of doing things IMO.
No specific opinion on your suggestion, however I have noticed the same thing re. Django’s async support. I have found that for IO-bound scaling use cases, standard sync Django + gunicorn/gevent is a better approach. It essentially means you get the benefits of async io with standard sync code. YMMV