Why doesn't buffer protocol types have bitwise operators?

Python Help
1

I want to do e.g. this:

b"1\xff\2" ^ b"\xff\xff\xff"

But:

TypeError: unsupported operand type(s) for ^: 'bytes' and 'bytes'

So instead I have to:

int.to_bytes(int.from_bytes(b"1\xff\2") ^ int.from_bytes(b"\xff\xff\xff"), length=3)

Which is… less readable.

This seems like a natural thing to want to do with types that implement the buffer protocol, so it surprises me that it doesn’t work. I would like to understand the reason(s) why bytes et al. do not have bitwise operators.

2

It seems fairly sensible to me that bitwise operators could return bytes, so I can’t really speak to why they don’t. But if you need a quick way to get the job done:

>>> import numpy as np
>>> np.ndarray.tobytes(np.frombuffer(b"1\xff\2", dtype='u1') ^ np.frombuffer(b"\xff\xff\xff", dtype='u1'))
b'\xce\x00\xfd'
1 Like
3

You can probably find reasons here: bitwise ops for bytes of equal length · Issue #63450 · python/cpython · GitHub
(old, but referred to from a recent one)

2 Likes
4

I won’t speak for anyone else, but having read through all of that, I find the arguments in favour far more convincing.

3 Likes
5

Thanks for the link! It’s interesting that this idea was already being discussed pretty extensively from 2013 until 2018 and still appears to be an open, undecided topic!
What I mainly got from the discussion as a reason not to go forward is that the BDFL was pretty skeptical, and was worried about surprises for users working with bytes/bytearrays. For the rest it seemed several core devs where pretty positive about it.

6

My concern is more with readability than performance. Numpy is nice, but it’s too heavy a dependency to add just to make the code look nicer, IMO.

Numpy does enforce same-length operands, which is also nice to have. One problem with using int.from_bytes is that I need to manually check that both bytes instances have the same length.

Thanks! I don’t really understand GvR’s concern about surprising behavior; what possible behavior could be expected from e.g. bytes ^ bytes other than bitwise XOR?