# Reworking "Buffered I/O" in CPython

**URL:** <https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019>\
**Category:** Core Development\
**Created:** [April 29, 2025, 12:14am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019 "2025-04-29T00:14:48Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [April 29, 2025, 12:14am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/1 "2025-04-29T00:14:48Z")

</div>

Ever since [gh-120754: Speed up open().read() by reducing the number of system calls](https://github.com/python/cpython/issues/120754) I have been trying to reduce Buffered I/O overhead. The I/O stack is really effective in a lot of cases as is evident by the minor evolution needed since [PEP 3116](https://peps.python.org/pep-3116/) in 2007. Over the past decade I/O devices have improved substantially in latency and throughput as well as CPython developing async, free-threading, and multiple interpreters. With those improvements I think there is good reason both from systems and Python to look at reducing overheads in Python I/O. In particular for the “Buffered I/O” layer I’ve been studying:

1. [~15% overhead reading small files](https://github.com/python/cpython/pull/122111). Code in CPython optimizes around this by using `FileIO` and/or `TextIOWrapper` directly instead of using `open()` (ex. `.pyc` writing). This is also in broader community where `open(buffering=0)` is somewhat common but can lead to partial writes (`FileIO` doesn’t retry partial writes; Other `open` invocations guarantee write-all or exception)
2. `seek` system calls when no seeking is used or needed (ex. read-only `stdin`, write-only `stdout`, read/write a single buffer, etc)
3. Locking and critical sections: `buffered` uses a single shared buffer and tracks position inside the file absolutely (every call must touch it currently).
4. Reducing copies of buffer data.

I think by reworking the “Buffered I/O” layer (`BufferedReader`, `BufferedWriter`, `BufferedRandom`, …) can fairly significantly improve performance and setup for longer term I/O performance projects (ex. utilizing newer system I/O APIs, batching operations, …). I’ve brought this up offline with a couple groups as well as prototyping locally. At a high level, I think there are significant improvements available:

1. Remove the absolute position tracking (and contention coming from it)
2. Operate on a list of buffers as core primitive  
a. When possible, just refer to user-provided buffer rather than copying (ex. bytes)  
b. Use `writev` to write multiple buffers simultaneously
3. Reduce/remove critical sections and locking in at least read-only and write-only cases (`stdin`, `stdout`, `stderr`)

I have the next several months free and am hoping to focus directly on this project. Definitely need core developer help in figuring out the right tradeoffs (ex. handling seek, flush, buffer protocol / `bytes-like`, …), drafting a PEP for actually shipping, finding the right benchmarks to measure performance, right tools to validate behavior, etc.

I am attending PyConUS and hope to discuss in person there as well as online here.

I want to take a moment to thank those who have helped review my Issues, PRs, and ideas so far. I’ve found engaging with the Python development really rewarding, and hope I can help contributing back to a language I’ve had the fortune to use for over 15 years.

Looking forward to spelunking,  
Cody

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [April 29, 2025, 7:50am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/2 "2025-04-29T07:50:00Z")

</div>

> [@cmaloney](#):
>
> I think by reworking the “Buffered I/O” layer (`BufferedReader`, `BufferedWriter`, `BufferedRandom`, …) can fairly significantly improve performance and setup for longer term I/O performance projects (ex. utilizing newer system I/O APIs, batching operations, …)

I think a good start would be to expose those capabilities at the raw IO level, e.g. `FileIO`. It is also a less risky endeavour than introducing additional complexity in the buffered IO layer.

> [@cmaloney](#):
>
> - Operate on a list of buffers as core primitive  
> a. When possible, just refer to user-provided buffer rather than copying (ex. bytes)  
> b. Use `writev` to write multiple buffers simultaneously

You seem to be hinting that it’s important to provide multi-threaded _writes_ of disjoint buffers. Is that a common use case?

> [@cmaloney](#):
>
> - Reduce/remove critical sections and locking in at least read-only and write-only cases (`stdin`, `stdout`, `stderr`)

Your 3 examples here are non-seekable streams that are not capable of explicitly-positioned IO. How do you plan to remove locking while keeping correct behavior with them?

---

<div class="post-metadata">

**Author:** ![facundo](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/facundo/32/53_2.png) [@facundo](https://discuss.python.org/u/facundo)\
**Post date:** [April 29, 2025, 10:32am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/3 "2025-04-29T10:32:26Z")

</div>

Hello Cody, thanks for working on this!

Having a leaner and faster I/O subsystem would be great! Let’s just pay extra attention to not sacrifice simplicity or safety because of speed. For example, the current situation that you mention that passing `buffering=0` to `open` may lead to partial writes; that is surprising and probably unexpected.

I’m not saying that we should not allow particular ways of using the I/O subsystem, like “let me do this, I don’t want buffering because whatever, I’ll take care of some trickyness because of that”, but high level interface should be always simple and correct.

Thanks again!

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [April 29, 2025, 3:50pm UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/4 "2025-04-29T15:50:27Z")

</div>

> [@facundo](#):
>
> For example, the current situation that you mention that passing `buffering=0` to `open` may lead to partial writes; that is surprising and probably unexpected.

You mean in the case where the write length is bigger then the kernel will accept?  
Usually when buffering=0 is used the user want each write to be written atomically (as defined by the kernel) and it does not get split.

---

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [April 30, 2025, 3:03am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/5 "2025-04-30T03:03:04Z")

</div>

> [@pitrou](#):
>
> I think a good start would be to expose those capabilities at the raw IO level, e.g. `FileIO`. It is also a less risky endeavour than introducing additional complexity in the buffered IO layer.

👍. I also think fairly independent in implementation from BufferedIO changes I’m proposing. The BufferedIO changes I’m proposing _should_ reduce overhead in existing code without any changes, and with additional projects can get even bigger improvements (ex. batched I/O).

Main reason for including these in proposal is to try and validate the designs, engineering, and investment in this project works well with those projects. I prototyped using a context manager to hold a `io_uring` ring which served as a “deferred” I/O error collection point + asynchrony. There’s some cool improvements possible with that (ex. dispatch reading lots of small files in parallel with a single interpreter/thread), but is also a lot of new code and primitives that I’m not sure are the right ones. Likely needs to be an experimental third party module first.

> [@pitrou](#):
>
> You seem to be hinting that it’s important to provide multi-threaded _writes_ of disjoint buffers. Is that a common use case?

My goal is `open().write()`, `open().read()`, and `print()` even in just single-thread code should be faster. For `print()` today each object passed in is serialized to string then written (usually with a `memcpy`) into the BufferedIO buffer (ignoring [`-u`](https://docs.python.org/3.13/using/cmdline.html#cmdoption-u) and `tty` vs. non-tty stdout, as well as `WindowsConsoleIO` for brevity here). Based on my measuring reducing the copies (particularly of 1KB-100KB objects) into the shared buffer should notably increase performance.

Ideally the model also scales well for multi-threaded code but that is a want to me rather than a requirement currently. With free-threading I think it’s likely in “adopting multi-threaded” code that `print()` (or logging) will occur in multiple distinct OS threads simultaneously and am hoping this work improves scalability of that / helps make all the threads faster. A lot of library or more optimized code I’ve encountered does things like a logger per thread / process + logging aggregator to keep things fast. Hoping to make that less necessary.

> [@pitrou](#):
>
> Your 3 examples here are non-seekable streams that are not capable of explicitly-positioned IO. How do you plan to remove locking while keeping correct behavior with them?

One of the questions I’m hoping for help to answer 🙂. To some extent, if I/O is multi-threaded and unordered can Python rely on the system Kernel to order it? Does the IO stack need to synchronize it (which BufferedIO somewhat does implicitly today)? What are acceptable orderings there?

My current thought is to use `tarfile` and `zipimport` / `zipapp` as validators for this to help measure correctness and ensure performance is maintained or improved. A couple ideas I’ve looked at but don’t have any strong opinions on yet, and am definitely looking for more ideas:

1. `seek` working as a deferred / buffered operation (just like a read or write). Get position `seek` sums buffered operation state to calculate needed result.
2. Treat position-changing `seek` as a “flushing” operation (all buffered changes to that point need to be sent to OS).

Note for both of these, if the user seeks / moves around underneath the BufferedIO without explicitly flushing first behavior would change. I think that is a reasonable behavior change to have? (The existing `buffered` has a lot of edge cases around this itself).

> [@facundo](#):
>
> Having a leaner and faster I/O subsystem would be great! Let’s just pay extra attention to not sacrifice simplicity or safety because of speed. For example, the current situation that you mention that passing `buffering=0` to `open` may lead to partial writes; that is surprising and probably unexpected.

Discussed changing that in another thread, generally agreed it’s unexpected: [Change `open().write()` to guarantee all bytes will be written or an exception will be thrown](https://discuss.python.org/t/change-open-write-to-guarantee-all-bytes-will-be-written-or-an-exception-will-be-thrown/71082).

My personal thought / idea for resolving is users can manually make a `FileIO` if they want, and it works exactly as specified and documented; a special-purpose tool. Changing `FileIO` is possible but not worth the risk. I want to change `open(buffering=0)` to actually return a `Buffered{Reader,Writer}` with a buffer size of 0 which would mean “dispatch I/O immediately when get data” (don’t buffer in Python) with the no partial writes without an exception behavior. All `open()` invocations then guarantee write-all or exception. I think that is a reasonable feature change path. Definitely changes return type and could break some code, but I think worth it for the improved semantics with `FileIO` still existing for people who require the previous behavior.

I just found [open() of read-write non-seekable streams broken · Issue #64273 · python/cpython · GitHub](https://github.com/python/cpython/issues/64273) which may complicate that plan some…

> [@barry-scott](#):
>
> You mean in the case where the write length is bigger then the kernel will accept?  
> Usually when buffering=0 is used the user want each write to be written atomically (as defined by the kernel) and it does not get split.

The Python I/O code at the underlying call level (`_Py_write_impl`) doesn’t actually do this today as some kernels have limits how big a `write()` is allowed to specific device types…

For the `open()` builtin `buffering=` just lets you choose between line buffering, fixed-size in-python buffering, or `0` which is a special value that (currently) the “Raw I/O” (`FileIO` typically) should be returned directly without a buffering wrapper. Note that reading text via open (ex. `open("README.rst", "rt")`) requires buffering today / disabling it is explicitly disallowed in that case.

(the docs don’t currently match implementation of that… an open issue I really should get back to, but have been hoping to just make it so “open() always does a cleaner behavior”, [Update FileIO comments, documentation to match implementation · Issue #129011 · python/cpython · GitHub](https://github.com/python/cpython/issues/129011), [open() built in's doc does not say what the buffering default of -1 means · Issue #93600 · python/cpython · GitHub](https://github.com/python/cpython/issues/93600))

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [April 30, 2025, 7:22am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/6 "2025-04-30T07:22:27Z")

</div>

> [@cmaloney](#):
>
> To some extent, if I/O is multi-threaded and unordered can Python rely on the system Kernel to order it?

I’m not sure what the question means, but ultimately Python file objects are an abstraction over kernel-provided IO facilities. So it makes sense that what Python can do is bound to kernel-exposed semantics.

The problem here is more to ensure that Python file objects don’t have race conditions or inconsistencies of their own.

> [@cmaloney](#):
>
> Does the IO stack need to synchronize it (which BufferedIO somewhat does implicitly today)?

Buffered IO merely synchronizes its own state (which strikes me as necessary, unless you come up with an extremely clever lockless implementation). It might also synchronize raw IO as a side effect, but that’s not part of the semantics.

What’s _dictated_ by the current semantics, though, is the reliance of all reads and writes on an implicit “file position” that’s also updated by those reads and writes, and that has to be carefully handled when several threads issue IO operations on the same file object. It might be a good idea to add “read at” and “write at” primitives that ignore the internal file position.

(beware that [`ReadFile`](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-readfile) on Windows is annoying in that regard: even if you give it an absolute position to read from, it will still update the file’s current position… is there a modern alternative to that @steve.dower ?)

> [@cmaloney](#):
>
> Note for both of these, if the user seeks / moves around underneath the BufferedIO without explicitly flushing first behavior would change. I think that is a reasonable behavior change to have?

At what points buffers are flushed is indeed an implementation detail, so you can freely change it (except when `flush` is called, of course).

> [@cmaloney](#):
>
> I want to change `open(buffering=0)` to actually return a `Buffered{Reader,Writer}` with a buffer size of 0

Can you make a separate discussion for this? That’s unrelated to the rest of the topic.

> [@cmaloney](#):
>
> > [@barry-scott](#):
> >
> > You mean in the case where the write length is bigger then the kernel will accept?  
> > Usually when buffering=0 is used the user want each write to be written atomically (as defined by the kernel) and it does not get split.
> 
> The Python I/O code at the underlying call level (`_Py_write_impl`) doesn’t actually do this today as some kernels have limits how big a `write()` is allowed to specific device types…

I think you misread it? `_Py_write_impl` does not retry on partial writes. At most a single _successful_ `write` call is issued.

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [April 30, 2025, 9:09am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/7 "2025-04-30T09:09:26Z")

</div>

> [@pitrou](#):
>
> It might be a good idea to add “read at” and “write at” primitives that ignore the internal file position.
> 
> (beware that [`ReadFile`](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-readfile) on Windows is annoying in that regard: even if you give it an absolute position to read from, it will still update the file’s current position… is there a modern alternative to that @steve.dower ?)

Kind of. Using overlapped IO (the `OVERLAPPED` structure) with a synchronous\[1\] file handle lets you set the position to start reading from (see [this section of the ReadFile docs](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-readfile#synchronization-and-file-position)), though it still updates the file pointer afterwards. But if every reader is doing it this way, then none of them will care about the initial file pointer position.

This seems like a proposal that can pretty easily be put together as a standalone library initially. The current `_io` module isn’t critically tied into the core runtime\[2\], and its type hierarchy could be replicated as much or as little as someone liked. Most of the IO hierarchy is transparent to callers, so `from modernio import open` is likely enough to upgrade any existing code, and I think we’d be incredibly open to merging it into the core runtime after it’s proven itself (provided it’s designed for being merged in - so pure C, etc.).

* * *

1. It doesn’t have to be an _async_ file object to use `OVERLAPPED`, though you do have to open it with a specific flag initially even for non-async. The async version is pretty easy to synchronise, because it just returns a wait object that you can wait on immediately - as a bonus, we could get Ctrl+C interrupts and better parallelism for scattered reads if we went to full async here. 

2. And anywhere it is, its semantics will be critical, so you couldn’t just change them anyway.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [April 30, 2025, 9:47am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/8 "2025-04-30T09:47:13Z")

</div>

> [@steve.dower](#):
>
> Using overlapped IO (the `OVERLAPPED` structure) with a synchronous[[1]](#footnote-244323-1) file handle lets you set the position to start reading from (see [this section of the ReadFile docs](https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-readfile#synchronization-and-file-position)), though it still updates the file pointer afterwards. But if every reader is doing it this way, then none of them will care about the initial file pointer position.

I agree that people should use either the explicitly-positioned APIs or the implicitly-positioned APIs, but not both on the same file object (and arguably, any multi-threaded reader should only use explicitly-positioned APIs).

If we introduce explicitly-positioned APIs, we’ll have to document that, and also mention that the behavior when mixing both API styles is system-dependent (because POSIX [`pread`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/read.html) doesn’t update the file position).

> [@steve.dower](#):
>
> Most of the IO hierarchy is transparent to callers, so `from modernio import open` is likely enough to upgrade any existing code

That’s a good point.

---

<div class="post-metadata">

**Author:** ![facundo](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/facundo/32/53_2.png) [@facundo](https://discuss.python.org/u/facundo)\
**Post date:** [April 30, 2025, 9:58am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/9 "2025-04-30T09:58:13Z")

</div>

> [@barry-scott](#):
>
> You mean in the case where the write length is bigger then the kernel will accept?  
> Usually when buffering=0 is used the user want each write to be written atomically (as defined by the kernel) and it does not get split.

I mean more generically (and totally leave you to your good understanding): the general / external /simple interface should be as safer and with less surprises as possible, and the very particular cases where the user may seek for special behaviours should be exposed in ways that are obvious that “stuff may not behave as we always expected”.

Thanks again for working on this!

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [April 30, 2025, 10:07am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/10 "2025-04-30T10:07:37Z")

</div>

> [@pitrou](#):
>
> If we introduce explicitly-positioned APIs, we’ll have to document that, and also mention that the behavior when mixing both API styles is system-dependent

Absolutely, though the IO stack could manage its own current position and still make it transparent for regular users. Users who want to directly mess with file descriptors are already in trouble on Windows (it’s emulated… barely), and those who get as far as a native `HANDLE` can cope with caveats.

Years ago I dreamed of replacing the IO stack on Windows with a properly native one, rather than trying to wrap up the C runtime’s emulation of POSIX semantics. `fileno()` is the only real problem, but I daresay it could be made to be slow/limited emulation when called and not before (while the CRT can be made to “open” arbitrary `HANDLE`s, it makes all sorts of assumptions that make it generally not a good idea).

---

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [April 30, 2025, 9:36pm UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/11 "2025-04-30T21:36:57Z")

</div>

> [@pitrou](#):
>
> I’m not sure what the question means, but ultimately Python file objects are an abstraction over kernel-provided IO facilities. So it makes sense that what Python can do is bound to kernel-exposed semantics.
> 
> The problem here is more to ensure that Python file objects don’t have race conditions or inconsistencies of their own.

Currently BufferedIO has a lock and for _every_ write (and most reads) acquires it exclusively with `ENTER_BUFFERED`; that means BufferedIO forces an order to the I/O calls with that lock.

`FileIO` much more closely exposes the underlying POSIX semantics just adapting from Python objects to system calls and doesn’t do any locking; rather multiple simultaneous or independent `read()` or `write()` calls are handled by the underlying system/kernel as to what happens (ex. ordering writes from multiple threads).

As a specific example, I tend to think even in `-u` / `PYTHONUNBUFFERED` mode developers want a single `print()` to not be interleaved with any other output even in the presence of threads with multiple `print()` happening simultaneously, but that is not what the implementation actually guarantees today. With unbuffered mode, each individual write happens immediately. This results in `strace python -u -c "print('test', '1')"` outputting 4 system calls (on Linux each tends to be fairly atomic / non-interleaved):

```python
write(1, "test", 4test) = 4
write(1, " ", 1 ) = 1
write(1, "1", 11) = 1
write(1, "\n", 1
) = 1

```

There I think ideal for user would be to actually have one “atomic” per POSIX standard call (ex. `writev`) that is ordered before/after other threads through the newline (line buffering). Some code works around this by pre-building an individual buffer and writing once. I’d like to get tools to be able to address that more precisely. It comes up in “line buffered” mode vs. fixed-size buffered mode as well (is python writing to a `tty` for `stdout` or not; CPython run as a subprocess vs at terminal will result in different set of system calls today). I think people want the whole line to not be interleaved across threads. I think it’s part of the reason `stdout` and `stderr` have quite so many “modes of operation” today (see: [`sys.stdin` docs](https://docs.python.org/3/library/sys.html#sys.stdin)).

Python’s current “Buffered I/O” layer doesn’t expose primitives for that. Moving to a list of buffers I think can add primitives to help express that at least for in-cpython cases; if those are found to be nicely simple/general can expose them more broadly. Make existing cases faster, resolve issues, and in time enable new features and optimizations.

This is some of the design tradeoffs that I’m hoping to work on with developers to figure out the right options / directions. There are a number of open issues especially around writing / printing during interpreter/thread shutdown as well as “read + write” on buffered I/O objects that I think with some thought can close the issues or at least significantly improve behavior.

> [@pitrou](#):
>
> If we introduce explicitly-positioned APIs, we’ll have to document that, and also mention that the behavior when mixing both API styles is system-dependent (because POSIX [`pread`](https://pubs.opengroup.org/onlinepubs/9699919799/functions/read.html) doesn’t update the file position).

Note I’m explicitly not looking to expose “write at” or “read at” here, that would definitely be new additional features. I think that’s interesting (particularly `pwritev` with `RWF_ATOMIC`), but a lot differently scoped project. I want to focus on making existing code and cases faster, lower overhead. In particular, the ~15% measured overhead in BufferedIO writing a single pre-composed buffer to a single file. Similar overheads reading files in fixed size chunks (`readall` bypasses that somewhat, but still can have moderate buffered overhead).

> [@pitrou](#):
>
> Buffered IO merely synchronizes its own state (which strikes me as necessary, unless you come up with an extremely clever lockless implementation). It might also synchronize raw IO as a side effect, but that’s not part of the semantics.

If I remove the BufferedIO lock, that may change behavior. I think we can keep compatible behavior with a atomically appended to list of operations / buffers / `byte` objects and when the aggregate buffer size passes a threshold (ex. DEFAULT\_BUFFER\_SIZE) do a single `writev` which I think will mirror current behavior sufficiently. That is really hard to measure without testing against lots of cases. Fortunately, there’s lots of usage in CPython community and the test suite to validate all the edge cases :).

In particular, the Buffered locks around “absolute position tracking” and having a single fixed-size in-memory buffer in my studying of the APIs aren’t required to meet the requirements of `BufferedReader`/`BufferedWriter`/`BufferedRandom` today. Definitely how the current implementation works, but I think both the special common cases (ex. write-only and read-only) as well as regular cases (ex. write this bytes/buffer to a regular file / `pathlib.Path.write_text`) and complex cases (ex. lots of seeking with small reads/writes ala `tarfile` or `zipapp`) can be implemented with less overhead without those two pieces of state explicitly tracked using a list of operations/buffers and calculating position only when needed / requested.

I’d like to work with CPython developers to prove that theory out with code. Definitely if it doesn’t prove true, then will be lessons learned and will delete the experimental work / keep the current code as is. I hope with the gh-120754 / FileIO changes helped demonstrate building piece by piece to something definitely better/faster/simpler and listening to feedback both from people and tools / measurement to improve existing code without breaking critical guarantees.

> [@pitrou](#):
>
> Can you make a separate discussion for this? That’s unrelated to the rest of the topic.

It’s not implementable without reworking `buffered` to allow a size-zero buffer which is part of what I want to add as part of this work. This work is a prerequisite to me 🙂. Agreed out of scope for initial work though, linked to the existing discussion which didn’t seem to get enough traction to actually motivate changing it independently.

> [@pitrou](#):
>
> I think you misread it? `_Py_write_impl` does not retry on partial writes. At most a single _successful_ `write` call is issued.

Agreed it doesn’t retry partial writes. It’s more that it may itself _split_ any write / buffer passed to which means code using it doesn’t have a guarantee the buffer will be written “atomically” in any form, esp. across platform. On all platforms no write will ever be passed through that goes over `_PY_WRITE_MAX`. On some platforms and for some “Raw I/O” devices that split size is smaller.

> [@steve.dower](#):
>
> Most of the IO hierarchy is transparent to callers, so `from modernio import open` is likely enough to upgrade any existing code, and I think we’d be incredibly open to merging it into the core runtime after it’s proven itself (provided it’s designed for being merged in - so pure C, etc.).

I looked at this but because Buffered I/O lives in the middle of the I/O stack it is particularly difficult to do. The python test suite works to do this with `_pyio` and `_io` across the existing io tests but often what ends up being tested isn’t what is intended (ex. test cases that call `open` directly instead of `self.open` which defers to the two implementations).

I definitely think a compile time or ideally run-time way to swap between the two implementations would make validating new vs. old a lot easier. `_io` is used in early startup and the behavior / performance there, particularly within `importlib` and `zipimport` matters a lot to me to validate and not regress. I don’t know a way to test/validate that without being in the `cpython` repository.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [May 1, 2025, 10:16am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/12 "2025-05-01T10:16:08Z")

</div>

> [@cmaloney](#):
>
> Currently BufferedIO has a lock and for _every_ write (and most reads) acquires it exclusively with `ENTER_BUFFERED`; that means BufferedIO forces an order to the I/O calls with that lock.

The order is undefined as far as users are concerned, so that doesn’t matter. The goal of the lock is not to force a particular (undefined) order, but to ensure that the internal state of the Buffered object remains consistent.

> [@cmaloney](#):
>
> I think people want the whole line to not be interleaved across threads

I think so, but I’m not sure that’s relevant to the “reworking buffered I/O” discussion. Up until now, we were implicitly talking about binary I/O, not text I/O.

> [@cmaloney](#):
>
> Note I’m explicitly not looking to expose “write at” or “read at” here, that would definitely be new additional features. I think that’s interesting (particularly `pwritev` with `RWF_ATOMIC`), but a lot differently scoped project.

Well, I’m not sure which kind of multi-threaded IO you’re hoping to do without such primitives? Unless you’re talking about very simplistic file formats, you want to know at which file position your IO calls will operate, and that’s not possible if you issue implicitly-positioned IO from multiple threads.

> [@cmaloney](#):
>
> If I remove the BufferedIO lock, that may change behavior. I think we can keep compatible behavior with a atomically appended to list of operations / buffers / `byte` objects

I don’t think it’s worth discussing such details until there’s an implementation that shows it to be technically possible (and reasonably debuggable/maintainable as well).

> [@cmaloney](#):
>
> Agreed it doesn’t retry partial writes. It’s more that it may itself _split_ any write / buffer passed to which means code using it doesn’t have a guarantee the buffer will be written “atomically” in any form, esp. across platform.

Such a guarantee is impossible to make due to the semantics of the corresponding system calls, so what is the problem exactly? The only guarantee you get is that _the partial write_ that is actually done is atomic.

---

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [May 2, 2025, 12:34am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/13 "2025-05-02T00:34:15Z")

</div>

> [@pitrou](#):
>
> The order is undefined as far as users are concerned, so that doesn’t matter. The goal of the lock is not to force a particular (undefined) order, but to ensure that the internal state of the Buffered object remains consistent.

Agreed it is behavior users should not be relying on. Unfortunately when a system maintainer upgrades the system python and the behavior of a tool they rely upon changes they often report it against CPython. To me acknowledging and planning for that is important to try and make as smooth an upgrade path as possible for community, developers investigating, and triagers handling reports.

> [@pitrou](#):
>
> I think so, but I’m not sure that’s relevant to the “reworking buffered I/O” discussion. Up until now, we were implicitly talking about binary I/O, not text I/O.

I can build the an equivalent demo with `sys.stdin.buffer` / the `BufferedWriter` avoiding the `TextIOWrapper`, just not as short of a one-liner 🙂. From my reading of `_io_TextIOWrapper_write_impl` and `perf` profiles I’ve looked at of writing text to files there isn’t any locking in that layer, the locking per write in user space is entirely in `BufferedWriter` at the moment. With Python in UTF8 mode the Text I/O work is a really small part of the profile which is really nice.

> [@pitrou](#):
>
> Well, I’m not sure which kind of multi-threaded IO you’re hoping to do without such primitives? Unless you’re talking about very simplistic file formats, you want to know at which file position your IO calls will operate, and that’s not possible if you issue implicitly-positioned IO from multiple threads.

The big one I think users will run into with free-threading is multi-core scalability of writing to `sys.stdout`. A single `print()` in a thread could have a significant performance impact. Currently _all_ threads printing to `sys.stdout` are serialized in user space by `BufferedWriter`’s lock. From my understanding and experience POSIX compatible kernels handle multi-threaded contention writing to a single file descriptor and often do so with significant optimizations. I’d like to get Python closer to the speed of the underlying kernel here; reduce the Python `BufferedWriter` overhead. To reiterate though: my main focus is improving single thread cases, multi-thread gains are a nice to have.

As a concrete example `.pyc` file reading in a single thread currently does this set of system calls in `main`:

```python
openat(AT_FDCWD, "<cpython dir>/Lib/ __pycache__ /platform.cpython-314.pyc", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=46027, ...}) = 0
lseek(3, 0, SEEK_CUR) = 0
read(3, "'\16\r\n\0\0\0\0R@\360g\276\272\0\0\343\0\0\0\0\0\0\0\0\0\0\0\0\24\0\0"..., 46028) = 46027
read(3, "", 1) = 0
close(3) = 0

```

There Python, in the `io.open_code` hook, opened the `.pyc` file for binary buffered reading. That created a `BufferedReader` which calls `seek` to get the absolute position, allocates a internal buffer, and creates a lock. `BufferedReader.read()` is then called without a size which defers to `_bufferedreader_read_all` and in this case, since the `BufferedReader` buffer is empty, defers entirely to `FileIO.readall`. `FileIO.readall` uses the `fstat` result from its initial open to allocate a file size + 1 byte `bytes` object, uses one `read` system call to fill it, and a second to validate it’s actually at the end. `readall` guarantees handling of cases where the file size changes between `open()` and the `readall` call so always goes until a non-zero length `read` system call returns size 0.

In that particular case the `BufferedReader` internal buffer, position tracking, and locking are all unnecessary / overhead introduced by “Buffered I/O”. They happen because of how `BufferedReader` is structured today. By reworking “Buffered I/O” I want to reduce or preferably eliminate that excess work.

In my measuring and experiments many cases can be improved by reworking Buffered I/O. That includes ones with many small reads, a mix of read + seek, a single big write, many small writes, and a mix of write + seek. The overheads of the current structures were small compared to older kernel and I/O device speeds/overheads. Currently they’re a significant portion of the runtime.

> [@pitrou](#):
>
> I don’t think it’s worth discussing such details until there’s an implementation that shows it to be technically possible (and reasonably debuggable/maintainable as well).

Chicken and egg a bit here. I’m trying to build consensus that it’s worth core developer time and effort to review if written, what features are important to people far more experienced and familiar with CPython than me, find mentor(s), and get buy in for the work it would take to ship such changes in a release. If the only thing needed to make progress is prototype code happy to sit down and write that.

* * *

To increase viability of this work I’m trying to align the docs, code comments, and code behavior for `FileIO` and `Buffered*`. One of the issues I found, and what led me to work on `_pyio`, is behavior differed from what was anticipated by developers or documented. [Currently have a PR around `BufferedReader.read` docs and non-blocking](https://github.com/python/cpython/pull/130653). Once get agreement on the docs for that, plan to make similar doc updates to resolve [FileIO.readinto() breaks the contract on io.RawIOBase.read() · Issue #118276 · python/cpython · GitHub](https://github.com/python/cpython/issues/118276).

---

<div class="post-metadata">

**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\
**Post date:** [May 2, 2025, 8:33am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/14 "2025-05-02T08:33:54Z")

</div>

> [@cmaloney](#):
>
> Chicken and egg a bit here. I’m trying to build consensus that it’s worth core developer time and effort to review if written, what features are important to people far more experienced and familiar with CPython than me, find mentor(s), and get buy in for the work it would take to ship such changes in a release. If the only thing needed to make progress is prototype code happy to sit down and write that.

Prototype code is certainly progress. And there’s a set of people who will be convinced by working code (and another by the performance of that code), as well as people who will be convinced by a clear architecture, nicer type hierarchy, or other less concrete aspects of the design. Doing a bit of all of them is the best way to build a consensus, as they’ll all convince someone, and that gets more people convinced.

That said, working, production-ready code is the hardest to argue against. If you can build a library that code can opt into and it’s good enough to run benchmarks on, you’ve already got the best chance of it being taken seriously by the most people. There are a few people around (I won’t name them - they’re reading already I’m sure) who might take something standalone and integrate it into massive apps at huge companies and report back. _That_ sort of thing is convincing in a way that logical arguments without code are not.

The key point is not to benchmark and criticise what’s already there, but to create something that _could_ be there and benchmark that. If it has to be done in place, then you’ll be building it in a fork of CPython for the time being. If you can do it as a library, it’s much much much easier for people to test and get involved in. It’s incredibly rare that any significant contribution will get merged without changes anyway, so the process of integrating a library is going to be about the same as integrating work that’s been done in-tree.\[1\]

* * *

1. Provided you’ve planned for it, e.g. by not relying on too many external build tools or code generators. We can’t take a dependency on Cython, for example, because Cython has a dependency on us. So if you wrote it all in Cython, then merging would become a rewrite.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [May 2, 2025, 9:01am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/15 "2025-05-02T09:01:20Z")

</div>

> [@cmaloney](#):
>
> In that particular case the `BufferedReader` internal buffer, position tracking, and locking are all unnecessary / overhead introduced by “Buffered I/O”. They happen because of how `BufferedReader` is structured today. By reworking “Buffered I/O” I want to reduce or preferably eliminate that excess work.

Perhaps the import mechanism can switch to unbuffered file I/O then? If the additional system calls are really significant, it might be a quick win.

(of course, that doesn’t mean that `BufferedReader` cannot be improved)

---

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [May 15, 2026, 12:20am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/16 "2026-05-15T00:20:27Z")

</div>

A year later and time for an update. I have built numerous prototypes, triaged the current IO implementation, and made multiple passes through [I/O issues](https://github.com/python/cpython/issues?q=is%3Aissue%20state%3Aopen%20label%3Atopic-IO) both open and closed. The core design I have come up with for BufferedIO, **nibbler** , parallels [`io.TextIOWrapper`](https://docs.python.org/3/library/io.html#io.TextIOWrapper) internals and reduces overheads. I prototyped in `_pyio` and plan to build out a C implementation after PyConUS. While the changes pass the existing test suite they modify a substantial percentage of the code and likely change subtle untested behaviors which makes them PEP scale. To reduce risks I’d like to get eyes on the work as soon as possible for 3.16.

The core change is moving from a single pre-allocated, contended, and locked buffer with continual userspace position tracking to a **nibbler**. The nibbler gathers buffers into a list and produces a **morsel** once `buffer_size` is exceeded. That **morsel** is written zero-copy using `writev` if possible or coalesced into a contiguous buffer and written with `write`.

In my most recent [`_pyio` prototype](https://github.com/cmaloney/cpython/pull/2) I have focused on `BufferedWriter` as write has the most challenges and changes but I plan to unify `io.BufferedReader` and `io.BufferedRandom` to use the nibbler list of buffers model as well ([protype: Unify \_pyio.Buffered\* around \_pyio.\_Nibbler](https://github.com/cmaloney/cpython/pull/3)). The current `_io` C implementation uses a common [`buffered` struct](https://github.com/python/cpython/blob/main/Modules/_io/bufferedio.c#L223-L267) while `_pyio.BufferedReader` and `_pyio.BufferedWriter` currently have distinct buffers and implementations. That means the `_io` changes to do all three are simpler than the `_pyio` changes.

Currently in `_io.BufferedIO` “write” copies data to a shared buffer under a lock as well as updating a sometimes present virtual position. With nibbler it just appends a buffer to a list. The initial implementation I am developing matches critical section locking of the existing BufferedIO for easy but the model enables moving to a zero-copy and lock-free implementation for further performance benefits in the future. The system resolves some longstanding I/O issues, reduces buffer allocations, reduces copies, and lays the foundation for more feature and performance work.

One of the big changes I want to enable with this is for `open(buffering=0)` to emit a BufferedIO with `buffer_size=0` rather than the current `FileIO`. `FileIO` intentionally matches [POSIX write](https://pubs.opengroup.org/onlinepubs/9799919799/functions/write.html) which requires callers to check return size to detect partial writes. With all other `open()` calls either all data is consumed or an exception is raised. This case has no copies and gives the expected write retry loop with minimal overheads in the BufferedIO nibbler model.

Core aspects:

- Move from a single pre-allocated, locked buffer to nibbler model
  - Collect buffers then either `writev` (preferred) or allocate + merge + `write` on flush.
  - Mutable objects which are buffered require a copy to guarantee behavior as modification of the buffer after write is explicitly allowed.
  - The nibbler model reduces allocations, reduces copies, and reduces critical section length increasing performance in contended cases.

- Remove internal position tracking from BufferedIO. This means .seek() always queries the system for file position and adds buffer size to that as an offset. This reduces contended variables, significantly simplifies implementation, and removes two `seek()` system calls in non-seeking cases.
- Support `buffer_size=0` with BufferedIO.
- Always flush on interleaved read + write. Most code paths already do this as a result of bug fixing but not all. To efficiently build in-memory use `io.BytesIO`.
- Unify the open builtin so all forms retry partial writes by changing `open(buffering=0)` to use BufferedIO with `buffer_size=0`. Interrupted writes are already always retried ([PEP 475 – Retry system calls failing with EINTR](https://peps.python.org/pep-0475/)).

Possible extensions:

- Explicit write control: print(“a”, “b”, “c”, flush=True) could explicitly mark “try to make a single write call” to reduce multi-threaded printing issues.
- Explicit copy control of mutable buffers: Enables using pre-allocated mutable buffers, such as `bytearray`, when writing with a no-copy guarantee.
- lockfree read + write: This changes the ordering much more substantially; should have significant performance benefits with shared I/O objects but also likely to subtly change behavior of a lot of code.
- Zero-copy `io.TextIOWrapper.write()` to device: Requires a new API to access utf-8 bytes in `str` without creating a `bytes` ([discussion](https://discuss.python.org/t/encode-str-utf-8-uncodeobject-without-a-copy/105572)). This also removes internal buffering by deferring to the underlying BufferedIO. Gets closer to supporting non-blocking I/O at the text layer as well as better support for interleaved read and write.

---

<div class="post-metadata">

**Author:** ![vstinner](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/vstinner/32/15130_2.png) [@vstinner](https://discuss.python.org/u/vstinner)\
**Post date:** [May 15, 2026, 11:00am UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/17 "2026-05-15T11:00:57Z")

</div>

This is a very promising plan! Would it be possible to have incremental changes? So we can properly measure and test each step?

---

<div class="post-metadata">

**Author:** ![cmaloney](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cmaloney/32/15851_2.png) [@cmaloney](https://discuss.python.org/u/cmaloney)\
**Post date:** [May 15, 2026, 5:25pm UTC](https://discuss.python.org/t/reworking-buffered-i-o-in-cpython/90019/18 "2026-05-15T17:25:18Z")

</div>

The change from single locked buffer to nibbler there isn’t an incremental implementation path but my plan is to copy `bufferedio.c` to a new `bufferedionibbler.c` and setup so both can be used in parallel in a fork for performance comparison, testing, and iteration. If/when a merge to `python/cpython`, swap of default, and deletion of pre-nibbler `bufferedio.c` should happen will need to evaluate and build consensus around.

The pieces beyond that such as the `open()` buffering change, changing locking, and `TextIOWrapper` work I plan to do incrementally. One of my big questions is: Is it going to be better to spread the known I/O behavior changes out over time / CPython major releases or make them all experimental options in the short term and do a larger “swap” later? (on PEP acceptance?)