Reworking "Buffered I/O" in CPython

Core Development
16

A year later and time for an update. I have built numerous prototypes, triaged the current IO implementation, and made multiple passes through I/O issues both open and closed. The core design I have come up with for BufferedIO, nibbler, parallels io.TextIOWrapper internals and reduces overheads. I prototyped in _pyio and plan to build out a C implementation after PyConUS. While the changes pass the existing test suite they modify a substantial percentage of the code and likely change subtle untested behaviors which makes them PEP scale. To reduce risks I’d like to get eyes on the work as soon as possible for 3.16.

The core change is moving from a single pre-allocated, contended, and locked buffer with continual userspace position tracking to a nibbler. The nibbler gathers buffers into a list and produces a morsel once buffer_size is exceeded. That morsel is written zero-copy using writev if possible or coalesced into a contiguous buffer and written with write.

In my most recent _pyio prototype I have focused on BufferedWriter as write has the most challenges and changes but I plan to unify io.BufferedReader and io.BufferedRandom to use the nibbler list of buffers model as well (protype: Unify _pyio.Buffered* around _pyio._Nibbler). The current _io C implementation uses a common buffered struct while _pyio.BufferedReader and _pyio.BufferedWriter currently have distinct buffers and implementations. That means the _io changes to do all three are simpler than the _pyio changes.

Currently in _io.BufferedIO “write” copies data to a shared buffer under a lock as well as updating a sometimes present virtual position. With nibbler it just appends a buffer to a list. The initial implementation I am developing matches critical section locking of the existing BufferedIO for easy but the model enables moving to a zero-copy and lock-free implementation for further performance benefits in the future. The system resolves some longstanding I/O issues, reduces buffer allocations, reduces copies, and lays the foundation for more feature and performance work.

One of the big changes I want to enable with this is for open(buffering=0) to emit a BufferedIO with buffer_size=0 rather than the current FileIO. FileIO intentionally matches POSIX write which requires callers to check return size to detect partial writes. With all other open() calls either all data is consumed or an exception is raised. This case has no copies and gives the expected write retry loop with minimal overheads in the BufferedIO nibbler model.

Core aspects:

  • Move from a single pre-allocated, locked buffer to nibbler model
    • Collect buffers then either writev (preferred) or allocate + merge + write on flush.
    • Mutable objects which are buffered require a copy to guarantee behavior as modification of the buffer after write is explicitly allowed.
    • The nibbler model reduces allocations, reduces copies, and reduces critical section length increasing performance in contended cases.
  • Remove internal position tracking from BufferedIO. This means .seek() always queries the system for file position and adds buffer size to that as an offset. This reduces contended variables, significantly simplifies implementation, and removes two seek() system calls in non-seeking cases.
  • Support buffer_size=0 with BufferedIO.
  • Always flush on interleaved read + write. Most code paths already do this as a result of bug fixing but not all. To efficiently build in-memory use io.BytesIO.
  • Unify the open builtin so all forms retry partial writes by changing open(buffering=0) to use BufferedIO with buffer_size=0. Interrupted writes are already always retried (PEP 475 – Retry system calls failing with EINTR).

Possible extensions:

  • Explicit write control: print(“a”, “b”, “c”, flush=True) could explicitly mark “try to make a single write call” to reduce multi-threaded printing issues.
  • Explicit copy control of mutable buffers: Enables using pre-allocated mutable buffers, such as bytearray, when writing with a no-copy guarantee.
  • lockfree read + write: This changes the ordering much more substantially; should have significant performance benefits with shared I/O objects but also likely to subtly change behavior of a lot of code.
  • Zero-copy io.TextIOWrapper.write() to device: Requires a new API to access utf-8 bytes in str without creating a bytes (discussion). This also removes internal buffering by deferring to the underlying BufferedIO. Gets closer to supporting non-blocking I/O at the text layer as well as better support for interleaved read and write.
6 Likes