A long time ago (September 2018) asyncio’s StreamWriter class gained awrite(data) and aclose() async methods. See
But they never made it into a release and awrite() was merged into write() which became a hybrid sync/async method. See
But that also never made it into a release and was reverted in
Reading the related discussion Revert the new asyncio Streams API · Issue #82423 · python/cpython · GitHub which has motivation for the revert, it seems that the bigger picture for the revert was to design a better streaming API, not that the awrite() itself was a bad method to add.
It’s a shame that StreamWriter.awrite(data) got caught up in this (failed) stream API redesign, because IMO it was a good addition that allows a no-copy write.
Wearing my hat as a MicroPython maintainer, we would really like this awrite(data) method to come back! Many of the uses of asyncio in MicroPython are for streaming data between devices/networks, and it’s mandatory that the streaming can work without buffering everything in RAM. For example downloading a large XML file (eg with weather data) straight to the filesystem because it’s too big to fit in RAM. Or streaming a large audio file from filesystem/ROM out to audio hardware (eg DAC or I2S).
The problem with using the standard pattern for write:
writer.write(big_buffer)
await writer.drain()
is that big_buffer cannot go out all at once to the underlying device and instead must be drained in chunks in writer.drain(). But due to the API design of writer.write() a copy of big_buffer must be made and stored in the writer instance (eg big_buffer could be a bytearray). It’s impossible to make a copy of big_buffer if it’s more than half available RAM.
As far as I know, there’s currently no way in asyncio to do a no-copy write-and-drain of a buffer of data.
But if we had StreamWriter.awrite(data) then that would work perfectly for this use case, because the implementation of awrite can take “ownership” of data until it has all been written out.
In MicroPython we try hard to be compatible with CPython, and so I’m wondering if we could come up with a solution to the above problem (no-copy write-and-drain) that would work in both CPython and MicroPython. We could just implement awrite() ourselves, but then code written using awrite() would not work under CPython and that’s not a good situation for many reasons.
Would CPython consider bringing StreamWriter.awrite(data) back? Or is there a better solution?
(Note: we did actually implement both awrite and aclose in a past version of MicroPython’s asyncio long ago, and due to legacy reasons still have those methods. But they are not documented and not used in any new code.)