Extending open() to accept custom RawIOBase, BufferedIOBase, and TextIOBase classes

Ideas
1

I’ve been working with Python’s I/O system and ran into a use-case that I believe could be improved with a small, backward-compatible extension.

The Problem

Currently, the built-in open() function is hardcoded to use io.FileIO for creating the raw binary stream. While this is perfect for most file operations, it can be limiting when you need lower-level control.

In my specific case, I needed to open a file and acquire an exclusive lock on it immediately at the raw file descriptor level:

class LockedFileIO(io.FileIO):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        fcntl.lockf(self.fileno(), fcntl.LOCK_EX)

The standard open() doesn’t expose a way to customize the RawIOBase implementation it uses internally, making this awkward without creating the entire I/O stack manually.

Proposed Solution

I propose we add optional keyword arguments to open() to specify the classes used to construct the I/O layers.

The new signature could look like:
open(file, mode='r', ..., raw_io_class=None, buffered_io_class=None, text_io_class=None)

How it would work:

  • If raw_io_class is provided (and mode is appropriate), it would be instantiated instead of io.FileIO. It would need to be a callable that returns a RawIOBase instance (e.g., raw_io_class(file, mode, ...)).

  • Similarly, buffered_io_class could override the default buffering logic, and text_io_class could override the text wrapper. If the mode necessitates using a buffered or text stream, the provided class will be used. Otherwise, the argument is safely ignored.

  • The function would retain its current behavior if these arguments are None (the default), ensuring full backward compatibility.

2

Did you consider using the opener argument of open()?

import os
import fcntl

def opener(path, flags):
    fd = os.open(path, flags)
    fcntl.lockf(fd, fcntl.LOCK_EX)
    return fd

open('filename', 'x', opener=opener)
4 Likes
3

:+1: I think opener allows you to do what you want here today. I’m really hesitant to add more knobs to open; it’s already a really complicated function.

There is a similar proposed feature `open()`able objects which gives the ability to do arbitrary transformations / processing for opening and reading. That would enable using open on things like zipfile.Path or other “virtual” filesystems such as HTTP accessible blob storage. For this particular case I think it would provide a good abstraction to encapsulate “work with files in this directory or bit of codebase in this particular way”.