The Steering Council asked for this post so the proposal gets broader visibility (steering-council#372). If you use
_pyioin a project, I’d like to hear from you so we can make sure we have a good path forward while making the CPython I/O stack easier to develop and maintain.
I’m working on some significant changes to Buffered I/O in CPython. One challenge in making improvements to the I/O stack is that there are two implementations which currently differ significantly but need to be maintained to have:
Unfortunately _pyio and _io have distinct algorithms and behavior around major cases. _io for instance has a unified buffered C object which is used by the various Buffered* classes while _pyio has distinct buffering logic implementations in BufferedReader and BufferedWriter.
What I would like to do, at least in the short term, is to drop _pyio as it has no known users. That needs a PEP 399 exception: C _io would no longer have a pure-Python eqivalent in _pyio. This is effectively codifying the current maintenance state: It sometimes gets changed with other _io changes but not always.
With just one implementation it will be easier to land features such as free-threading safety as well as performance improvements. Security changes and long-standing bugs take fewer steps to fix.
This proposal does not imply there could never be another _pyio. I personally find it useful for prototyping changes. The goal is to reduce development and maintenance burden.
Known cases for keeping _pyio
- PyPy or some unknown third party might have a hard requirement on it. PyPy uses a native
_io. I audited open-source projects and couldn’t find active users. If your project has a hard dependency on it, please let us know. - Contribution and interest in
_pyiois up (History for Lib/_pyio.py - python/cpython · GitHub). - It is useful for teaching and understanding I/O in Python. Two core developers raised this during the sprint.
What I’d like to hear
- Do you import
_pyioanywhere? Production code, tests, tooling and teaching material all count. What do you use it for? If your project depends on it, I’m happy to work with you on a path to a maintainable state. - Do you contribute to
_pyio? What draws you to it, and what would you want to see kept?_pyiocould also be developed outside of CPython’s core, reducing barriers to entry. - Is there a reason for CPython to keep maintaining
_pyiothat isn’t listed above?