# Simple \`Reader\` and \`Writer\` protocols

**URL:** <https://discuss.python.org/t/simple-reader-and-writer-protocols/61510>\
**Category:** Typing\
**Tags:** typing\
**Created:** [August 21, 2024, 12:20pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510 "2024-08-21T12:20:21Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 21, 2024, 12:20pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/1 "2024-08-21T12:20:21Z")

</div>

As we all know, I/O is a particularly difficult part to type, especially safely. When the typing module was introduced, the `IO`, `TextIO`, and `BinaryIO` classes were supposed to represent “file-like” objects. Unfortunately, the introduction of these classes precedes protocols, so they are concrete classes that need to be sub-classed for other classes to be considered compatible with them. They are also fairly broad and many I/O classes don’t fully implement the required protocol, sometimes leading to unsafe calls.

In typeshed we have defined a few fairly tight protocols to alleviate these problems and we aim to use these tight protocols if possible. I also encourage library authors to do the same. Still, they are sometimes a bit clunky to use (especially since we don’t have a convenient method to compose protocols in type annotations) and they have a discoverability problem.

Therefore I suggest we add two fairly simple, fairly tight protocols to the typing module that will probably be good enough for 90% of use cases (number entirely made up) where `IO`, `BinaryIO`, and `TextIO` is currently used, a reader and a writer class. Something along the line of this:

```python
@runtime_checkable
class Reader[AnyStr](Iterable[AnyStr], Protocol):
    def read(self, n: int = ..., /) -> AnyStr: ...
    def readline(self) -> AnyStr: ...

@runtime_checkable
class Writer[AnyStr](Protocol):
   def write(self, s: AnyStr, /) -> int: ...

```

This is not a final proposal, and we’d need to put a bit more research into which methods are most used in practice, but just to give an idea. This splits the tasks of reading and writing (since consumers of file-like objects will usually do either but no both), and leaves out the more esoteric features like file seeking and physical file management, including closing files. These are still available for `IO` and its sub-classes or more specific protocols.

I think this could reduce a lot of the problematic uses of `IO` etc. and would be a big step forward for the safe, easy-to-use typing of I/O in Python.

---

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 21, 2024, 12:22pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/2 "2024-08-21T12:22:05Z")

</div>

The `Writer` class should probably not use `AnyStr`, but a custom type var bound to something like `str | Buffer` or even be unbound. (Same for `Reader`, really.)

---

<div class="post-metadata">

**Author:** ![rchen152](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rchen152/32/16955_2.png) [@rchen152](https://discuss.python.org/u/rchen152)\
**Post date:** [August 22, 2024, 12:52am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/3 "2024-08-22T00:52:29Z")

</div>

I like this idea. These seem like they would be much easier to use and understand than the existing `IO` base classes.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 2:35am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/4 "2024-08-22T02:35:50Z")

</div>

> [@srittau](#):
>
> Unfortunately, the introduction of these classes precedes protocols, so they are concrete classes that need to be sub-classed for other classes to be considered compatible with them.

Isn’t that a benefit? It gives the writer a guarantee that their subclass is an instance of the superclass they want (otherwise they have to assert on the type), it makes isinstance checking explicit rather than implicit, and probably faster. And if the base classes ever change, then the subclasses will reflect those changes.

Can you illustrate some benefits of `Reader/Writer` protocols rather than `Reader/Writer` base classes?

---

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 22, 2024, 2:58am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/5 "2024-08-22T02:58:25Z")

</div>

> [@NeilGirdhar](#):
>
> Can you illustrate some benefits of `Reader/Writer` protocols rather than `Reader/Writer` base classes?

I’m not sure I understand the question. There currently are no `Reader`/`Writer` base classes, only the very broad `IO` classes. But using base classes instead of protocols would mean that every existing class supporting I/O would need to be changed to include these (abstract) base classes. I don’t see that happening and don’t see any benefit over using protocols. Also, `isinstance` checks are not mixing well with duck typing, which is very prevalant when it comes to I/O in Python.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 3:11am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/6 "2024-08-22T03:11:52Z")

</div>

> [@srittau](#):
>
> . But using base classes instead of protocols would mean that every existing class supporting I/O would need to be changed to include these (abstract) base classes.

Right, that’s what I’m asking. I’m still confused as to why that would be? Couldn’t you make IO derive from these, or were you planning on removing those from IO?

> [@srittau](#):
>
> `isinstance` checks are not mixing well with duck typing, which is very prevalant when it comes to I/O in Python.

In the ABC world, isinstance always works, which Is one reason I prefer that pattern.

---

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 22, 2024, 7:56am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/7 "2024-08-22T07:56:24Z")

</div>

> [@NeilGirdhar](#):
>
> > [@srittau](#):
> >
> > . But using base classes instead of protocols would mean that every existing class supporting I/O would need to be changed to include these (abstract) base classes.
> 
> Right, that’s what I’m asking. I’m still confused as to why that would be? Couldn’t you make IO derive from these, or were you planning on removing those from IO?

Of course `IO` could derive those if we’d use ABCs instead of protocols, but I can’t find any good reason in favor of ABCs here, only disadvantages: Every I/O class (even third-party and legacy ones) would have to derive from those ABCs (and that not only takes years, it’s also unlikely to happen comprehensively), it adds extra base classes to the MRO, and would require extra imports. The same arguments basically apply to any protocol.

> [@NeilGirdhar](#):
>
> > [@srittau](#):
> >
> > `isinstance` checks are not mixing well with duck typing, which is very prevalant when it comes to I/O in Python.
> 
> In the ABC world, isinstance always works, which Is one reason I prefer that pattern.

`isinstance` also works with protocols. (I’ve sneak edited the required decorators into my example above.)

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 12:31pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/8 "2024-08-22T12:31:25Z")

</div>

> [@srittau](#):
>
> Every I/O class (even third-party and legacy ones) would have to derive from those ABCs

Don’t they already derive from `IO`?

> [@srittau](#):
>
> Of course `IO` could derive those if we’d use ABCs instead of protocols, but I can’t find any good reason in favor of ABCs here,

Should we itemize the benefits of ABCs and compare them to protocol?

Benefits of ABC:

- `isinstance(A, B)` always returns true if and only if `A` implements `B` whereas for protocols, it can be fooled by methods that don’t quite match the interface just because they have the same name. These false positives are going to be pretty common when you have common method names like `read` and `write`.
- `isinstance(A, B)` will always returns true even if `A` or `B`’s definition changes whereas for protocols changing either `A` or `B` can _silently_ break `isinstance`.
- If `A`’s definition changes in a way that’s incompatible with the interface `B`, then in the ABC case, the type checker will complain whereas with a protocol the type checker stays silent.
- Having an MRO is an explicit declaration of intent: It says, I expect my class to implement a given interface.
- It makes `isinstance(A, B)` faster since it will simply scan the `MRO` rather than verifying that all of the methods are in the interface.
- Type checkers process `isinstance/issubclass` constraints faster with MROs than with protocols (AFAIK).

Benefits of protocols:

- It requires old inheritors who want to inherit from `Reader` or `Writer`, but not `IO` to explicitly mention the ABC. Are there many such classes?
- It requires new inheritors who want to inherit from `Reader` or `Writer`, but not `IO` to explicitly mention the ABC. (Generally, explicit is better than implicit, so I personally don’t find this convincing.)
- The MRO gets larger. (I’m not sure why you think that’s such a big deal though?)
- Did I miss any benefits?

I think in general protocols are really useful when you can’t add the base class, for example, when:

- the concrete classes are basic types like `int` that don’t admit new base classes, or
- when the amount of existing code that would have to change is large enough that adding ABCs is too onerous.

If the latter is true in this case, then I agree with you that protocols make sense. Is that the case? Could you name a few `Readers/Writers` that don’t derive from `IO`?

---

<div class="post-metadata">

**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\
**Post date:** [August 22, 2024, 1:30pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/9 "2024-08-22T13:30:18Z")

</div>

> [@NeilGirdhar](#):
>
> > [@srittau](#):
> >
> > Every I/O class (even third-party and legacy ones) would have to derive from those ABCs
> 
> Don’t they already derive from `IO`?

The `IO` classes in the standard library are not all the `IO`-like classes there are. There’s many frameworks that implement their own duck-typed `IO` classes that don’t inherit from `IO` and likely never will for the sake of simplicity.

A few come to mind, but just to name one, so that one has been named: [webob.response.ResponseBodyFile](https://github.com/Pylons/webob/blob/2df6d0cefadc781da3c098af06a912c1a87a9958/src/webob/response.py#L1512-L1553)

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 1:32pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/10 "2024-08-22T13:32:20Z")

</div>

Interesting! But FYI, the one you linked wouldn’t work with the given protocols since it has no `read` or `write` methods 😄

---

<div class="post-metadata">

**Author:** ![Daverball](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/daverball/32/15931_2.png) [@Daverball](https://discuss.python.org/u/Daverball)\
**Post date:** [August 22, 2024, 1:34pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/11 "2024-08-22T13:34:14Z")

</div>

It has a sneaky `write` instance attribute. The object is basically a proxy object for the `Response` instance that created it.

---

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 22, 2024, 1:35pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/12 "2024-08-22T13:35:45Z")

</div>

> [@NeilGirdhar](#):
>
> Don’t they already derive from `IO`?

Not necessarily. I don’t think any stdlib class does at runtime, although we pretend they do at type checking time. `isinstance` doesn’t work:

```python
Python 3.12.3 (main, Jul 31 2024, 17:43:48) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import IO
>>> isinstance(open("x", "w"), IO)
False

```

There are also classes that can’t derive from `IO`, since they don’t implement the full interface, like `io.BufferedIOBase`.

I don’t expect many third-party I/O classes to derive from it.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 1:36pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/13 "2024-08-22T13:36:00Z")

</div>

Ah, my mistake! Didn’t see that. I guess type checkers will be okay with that too.

---

<div class="post-metadata">

**Author:** ![srittau](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/srittau/32/237_2.png) [@srittau](https://discuss.python.org/u/srittau)\
**Post date:** [August 22, 2024, 1:49pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/14 "2024-08-22T13:49:37Z")

</div>

> [@NeilGirdhar](#):
>
> `isinstance(A, B)` always returns true if and only if `A` implements `B` whereas for protocols, it can be fooled by methods that don’t quite match the interface just because they have the same name. These false positives are going to be pretty common when you have common method names like `read` and `write`.

The same is true for abstract base classes:

```python
>>> from abc import ABCMeta, abstractmethod
>>> class ABC(metaclass=ABCMeta):
... @abstractmethod
... def foo(self) -> None:
... raise NotImplementedError()
... 
>>> class Impl(ABC):
... def foo(self, x: int) -> None:
... pass
... 
>>> isinstance(Impl(), ABC)
True

```

> [@NeilGirdhar](#):
>
> `isinstance(A, B)` will always returns true even if `A` or `B`’s definition changes whereas for protocols changing either `A` or `B` can _silently_ break `isinstance`.

I don’t see how? Can you give an example?

> [@NeilGirdhar](#):
>
> If `A`’s definition changes in a way that’s incompatible with the interface `B`, then in the ABC case, the type checker will complain whereas with a protocol the type checker stays silent.

Again, I don’t see how? A type checker will notice the incompatibility when trying to pass the changed object. If you want to notice these changes at definition time, you can always derive a class from the protocol.

> [@NeilGirdhar](#):
>
> It requires old inheritors who want to inherit from `Reader` or `Writer`, but not `IO` to explicitly mention the ABC. Are there many such classes?

Yes, basically every class in the stdlib, and potentially many, many third-party classes.

> [@NeilGirdhar](#):
>
> I think in general protocols are really useful when you can’t add the base class

And we can’t. Because there are potentially thousands of third-party classes out there over which we don’t have any control. Also, if we’d add `IO` or the `Reader` and `Writer` classes starting with Python 3.14, we still have 5+ years of incompatibility to deal with.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 2:28pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/15 "2024-08-22T14:28:11Z")

</div>

> [@srittau](#):
>
> The same is true for abstract base classes:

I think you’ve misunderstood. The ABC case is returning true because `Impl` derives from `ABC`, and returns false if it doesn’t. The protocol case can exhibit false positives when some class happens to have a `write` method even though that method doesn’t match the interface. This is one way that protocols are inferior to ABCs.

I get literally hundreds of false positives doing `rg "def write\("` in my src directory from projects like tensorflow, MyPy, NumPy, cpython, etc.

> [@srittau](#):
>
> I don’t see how? Can you give an example?

If you were to add a method to the protocol, various classes that previously would have returned true to `issubclass` may now return false. Also, if someone modifies a derived class and changes the name of a method slightly, then it will no longer inherit from the protocol. This will not trigger any error at the definition. This is unlike ABCs.

> [@srittau](#):
>
> Again, I don’t see how? A type checker will notice the incompatibility when trying to pass the changed object. If you want to notice these changes at definition time, you can always derive a class from the protocol.

I’m talking about definition time. If you’re going to explicitly inherit from the protocol, then you may as well explicitly inherit from the ABC. The main benefit of protocols is their implicit nature, in my opinion.

> [@srittau](#):
>
> Yes, basically every class in the stdlib, and potentially many, many third-party classes.

The stdlib is easy to repair. If there are many existing their party classes that don’t already inherit from related stdlib classes (which could easily be modified), then I agree with you that this would be a good use of protocols.

> [@srittau](#):
>
> Because there are potentially thousands of third-party classes out there over which we don’t have any control. Also, if we’d add `IO` or the `Reader` and `Writer` classes starting with Python 3.14, we still have 5+ years of incompatibility to deal with.

Right, good point.

From my point of view, it’s too bad there’s no way to maneuver it in such a way that in 5 years, we are in a place where `Reader` and `Writer` are ABCs rather than protocols.

---

<div class="post-metadata">

**Author:** ![Jelle](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jelle/32/1049_2.png) [@Jelle](https://discuss.python.org/u/Jelle)\
**Post date:** [August 22, 2024, 4:03pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/16 "2024-08-22T16:03:00Z")

</div>

@NeilGirdhar just so I understand better, is your argument that ABCs are always preferable over Protocols? If not, what makes these Reader/Writer classes different from other classes that makes it so Protocols are preferable?

> [@NeilGirdhar](#):
>
> The stdlib is easy to repair.

Not necessarily, since many of the relevant classes are implemented in C and cannot easily use multiple inheritance or inherit from a Python-defined ABC or protocol.

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 22, 2024, 4:21pm UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/17 "2024-08-22T16:21:04Z")

</div>

> [@Jelle](#):
>
> just so I understand better, is your argument that ABCs are always preferable over Protocols? If not, what makes these Reader/Writer classes different from other classes that makes it so Protocols are preferable?

For the list of reasons I gave, I prefer ABCs when they’re not too onerous to use. But I concede from Sebastian’s last post that this seems to be a case where it is onerous.

> [@Jelle](#):
>
> Not necessarily, since many of the relevant classes are implemented in C and cannot easily use multiple inheritance or inherit from a Python-defined ABC or protocol.

Oh, I didn’t realize that C classes had trouble adding base classes.

Just so we’re clear though, we’re talking about an ABC in the sense of a class with abstract methods. Not necessarily one that inherits from `abc.ABC`. And this would just be interface inheritance (no member variables, no issues with `super()`, etc.).

---

<div class="post-metadata">

**Author:** ![moi90](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/moi90/32/8835_2.png) [@moi90](https://discuss.python.org/u/moi90)\
**Post date:** [August 24, 2024, 9:48am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/18 "2024-08-24T09:48:56Z")

</div>

> [@srittau](#):
>
> Therefore I suggest we add two fairly simple, fairly tight protocols

I find myself introducing such protocols ad-hoc in my code fairly often because of the aforementioned problem of classes that implement reader/writer/whatever protocols without inheriting from any specific base class. It would be cool to have them included in Python.

---

<div class="post-metadata">

**Author:** ![randolf-scholz](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/randolf-scholz/32/11123_2.png) [@randolf-scholz](https://discuss.python.org/u/randolf-scholz)\
**Post date:** [August 28, 2024, 10:33am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/19 "2024-08-28T10:33:48Z")

</div>

> [@NeilGirdhar](#):
>
> I’m talking about definition time. If you’re going to explicitly inherit from the protocol, then you may as well explicitly inherit from the ABC.

But Protocols are ABCs (`type(Protocol)` directly inherits from `ABCMeta`), so what is the actual advantage/difference at definition time, assuming `@abstractmethod` is added to the methods of the Protocol definition?

---

<div class="post-metadata">

**Author:** ![NeilGirdhar](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/neilgirdhar/32/5776_2.png) [@NeilGirdhar](https://discuss.python.org/u/NeilGirdhar)\
**Post date:** [August 28, 2024, 11:40am UTC](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510/20 "2024-08-28T11:40:22Z")

</div>

> [@randolf-scholz](#):
>
> But Protocols are ABCs (`type(Protocol)` directly inherits from `ABCMeta`), so what is the actual advantage/difference at definition time, assuming `@abstractmethod` is added to the methods of the Protocol definition?

In my comment that you quoted, I’m saying that the benefit of protocols over ABCs is that you implicitly inherit from them, and you don’t have to explicitly inherit from them. In this comment, I’m using ABC in the computer science meaning (an ordinary class with only abstract methods). I’m contrasting ABCs with protocols.

I listed what I think the benefits of ABCs over protocols are in another comment.

[Next page](https://discuss.python.org/t/simple-reader-and-writer-protocols/61510.md?page=2)