How to annotate the argument type for method `write`?

Typing
,
1

I create a class inherited from io.RawIOBase and then manually define its write method with typing hints.

My trys

First, I casually annotatememoryview | bytearray | bytes. Of course, that’s incorrect.

import io

class VoidStream(io.RawIOBase):
    # Argument 1 of "write" is incompatible with
    # supertype "_RawIOBase"; supertype defines
    # the argument type as "Buffer"
    def write(self, b: memoryview | bytes | bytearray) -> int:
        return len(b)

Then, I modify the annotation to collection.abc.Buffer, but:

from collections.abc import Buffer
import io

class VoidStream(io.RawIOBase):
    def write(self, b: Buffer) -> int:
        # Argument 1 to "len" has incompatible
        # type "Buffer"; expected "Sized"
        return len(b)

I learned mypy actually use the typeshed repository. The _io.pyi file also shows:

if sys.version_info >= (3, 12):
    from collections.abc import Buffer as Buffer

ReadableBuffer: TypeAlias = Buffer

def write(self, b: ReadableBuffer, /)

Finally, I worked out two hacking ways for bypassing.

def write(self, b: Buffer):
    b = memoryview(b)
    return len(b)

def write(self, b: memoryview): # type: ignore

Question:

  1. (out of curiosity) Is there any use case where write accepts a not-len-able argument?
  2. How to construct the non-hacking code to simultaneously satisfy both write and len?

Thanks.

2

Your second annotation (write(self, b: Buffer) -> int) is correct. The problem is not the annotation; the problem is that using len(b) at all in an override for io.RawIOBase is unsafe since the Buffer ABC only requires __buffer__, not __len__. Conceptually, that’s because it represents things that have buffers, not necessarily raw memory buffers themselves (an actualy raw memory buffer is just a memoryview). I’m not sure that anything in the standard library actually implements Buffer without Sized, but user-defined code absolutely can.

len(memoryview(b)) is indeed a good solution. You can think of it as extracting the (memory) buffer from the (Python) Buffer. Keep in mind that it’s more likely to be the number of items in the view than the number of bytes; for the latter, use memoryview(b).nbytes.

(I happen to have written a Stack Overflow answer to a very similar question!)

4 Likes
3

You could not use the length of the argument at all, but instead calculate it. E.g.:

from collections.abc import Buffer, Sized
import io

class SomeStream(io.RawIOBase):
    data: Sized # E.g.: The file contents themself

    def write(self, b: Buffer) -> int:
        start = len(self.data) # Or, if implemented `len(self)`
        ...
        # After any update
        now = len(self.data) # Or, if implemented `len(self)`
        return now - start

As far as I remember, this would be more ideomatic anyways, as write should always return the size written to the stream, not the size of what was given.

If we e.g. had some stream (with some way to know it’s length), but the stream can only take 2¹⁰ (1024) elements, and it would just do nothing once you are over that, write should return the length you actually wrote, e.g. if you had 1000 elements in there already, and you want to write 50, it should return 24 and discard the rest. Any write after that should return 0 (unless the Stream is resized or items get deleted).

For your VoidStream, I suppose it’s about having a ‘mock’ stream which doesn’t do anything when written to, and returns some empty string when read. If that is the case, making write return 0 would be the correct return value, although callers might expect the size of the argument being returned, so they can assert everything was written successfully.

As for your first question, I guess as long as the object can be written to the stream, and the length of the stored data (or len(self)) can be calculated still, the argument does not need to be Sized.

1 Like