# Get single-byte bytes objects from a bytes object

**URL:** <https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709>\
**Category:** Python Help\
**Created:** [December 22, 2023, 3:32pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709 "2023-12-22T15:32:32Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![InSync](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/insync/32/12531_2.png) [@InSync](https://discuss.python.org/u/InSync)\
**Post date:** [December 22, 2023, 3:32pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/1 "2023-12-22T15:32:32Z")

</div>

A `bytes` object yield `int` values when being iterated over:

```py
>>> list(b'abc')
[97, 98, 99]

```

I find myself wanting something like this:

```py
>>> b'abc'.to_list_of_single_bytes()
[b'a', b'b', b'c']

```

I thought that `.split()` is the method I was looking for, but apparently it rejects empty separators, unlike, say, JavaScript’s `.split()`:

```py
>>> b'abc'.split(b'') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: empty separator 

```

Of course, the result can also be achieved by a number of other ways:

```py
>>> [bytes([byte]) for byte in b'abc']
[b'a', b'b', b'c']
>>> 
>>> [byte.to_bytes(1, 'big') for byte in b'abc']
[b'a', b'b', b'c']
>>> 
>>> import re
>>> re.findall(b'.', b'abc', flags = re.S)
[b'a', b'b', b'c']

```

…but all of them are somewhat verbose and do not look quite as “nice” as `list(b'abc')`.

Is there a better way that I’m ignorant of? If not, would it be suitable to add such a method to the language?

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [December 22, 2023, 3:50pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/2 "2023-12-22T15:50:54Z")

</div>

How about

```auto
list(map(int.to_bytes, b'abc'))
>>> [b'a', b'b', b'c']

```

---

<div class="post-metadata">

**Author:** ![effigies](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/effigies/32/35835_2.png) [@effigies](https://discuss.python.org/u/effigies)\
**Post date:** [December 22, 2023, 4:00pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/3 "2023-12-22T16:00:20Z")

</div>

`int.to_bytes` doesn’t work without additional arguments:

```auto
In [1]: list(map(int.to_bytes, b'abc'))
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 list(map(int.to_bytes, b'abc'))

TypeError: to_bytes() missing required argument 'length' (pos 1)

```

The cleanest way I can think of would be to slice:

```auto
In [5]: b = b'abc'

In [6]: [b[i:i+1] for i in range(len(b))]
Out[6]: [b'a', b'b', b'c']

```

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [December 22, 2023, 4:02pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/4 "2023-12-22T16:02:15Z")

</div>

> [@effigies](#):
>
> `int.to_bytes` doesn’t work without additional arguments:

In \>=3.11 those arguments have defaults so it works.

---

<div class="post-metadata">

**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\
**Post date:** [December 22, 2023, 5:57pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/6 "2023-12-22T17:57:45Z")

</div>

> [@InSync](#):
>
> I find myself wanting something like this:

The real question is what subsequent problem you hope to solve by getting this result, and how common that need is. I think it’s likely a sign that you’re doing something wrong. Conceptually, bytes **do not represent text** , which is why the behaviour changed in 3.x (`list(b'abc')` does exactly what you want already in 2.x, modulo the fact that the `b` prefix is the default and Unicode strings with a `u` prefix are special). Indexing a `bytes` now gives an integer, because conceptually a `bytes` is a sequence of individual bytes, and “individual bytes” are fundamentally integral values, if they can be said to have any meaning at all. (Of course, the choice to interpret them as unsigned is somewhat arbitrary.)

But no, you haven’t really missed a better way to get the result you want. There isn’t a built-in conversion, so you indeed will either need to exploit some higher level functionality (like the byte-regex you show), or apply a conversion to each element in the standard ways (like the list comprehensions you show; of course `list(map(...))` can be adapted to the purpose, but that’s largely bikeshedding).

---

<div class="post-metadata">

**Author:** ![Rosuav](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rosuav/32/3429_2.png) [@Rosuav](https://discuss.python.org/u/Rosuav)\
**Post date:** [December 22, 2023, 6:20pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/7 "2023-12-22T18:20:16Z")

</div>

> [@kknechtel](#):
>
> Conceptually, bytes **do not represent text** ,

While this is true, bytes often do contain ASCII-encoded text, which is why you can do things like this:

```
>>> b"\xFF\x04%04d\xFE" % 123
b'\xff\x040123\xfe'

```

It would make a lot of sense to have an easy way to “chunk” a bytestring up into units of whatever size you like, with single-byte chunks being what the OP asked for. There just doesn’t happen to be an easy way to do that in the core bytes object.

---

<div class="post-metadata">

**Author:** ![davidism](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/davidism/32/3295_2.png) [@davidism](https://discuss.python.org/u/davidism)\
**Post date:** [December 22, 2023, 6:23pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/8 "2023-12-22T18:23:40Z")

</div>

A `bytes` object is sequence of ints. Indexing a sequence yields a single value. _Slicing_ a sequence yields another sequence. This is true of sequences in general, bytes isn’t special here except for its repr. `(value[i:i+1] for i in range(len(value)))` will get you 1-byte sequences of bytes, and the slice can be modified to return other lengths. Libraries such as more-itertools and boltons provide `chunked` functions: [API Reference — more-itertools 10.1.0 documentation](https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.ichunked)

---

<div class="post-metadata">

**Author:** ![abessman](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/abessman/32/8367_2.png) [@abessman](https://discuss.python.org/u/abessman)\
**Post date:** [December 22, 2023, 6:54pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/9 "2023-12-22T18:54:42Z")

</div>

I have needed to do something like this in the past. My usecase was that I needed to send bytes over a serial connection one byte at a time, using pyserial’s `serial.Serial` which only accepts iterables as input, even when only sending a single byte.

I can’t recall why I needed to do that. Possibly for debugging purposes.

---

<div class="post-metadata">

**Author:** ![InSync](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/insync/32/12531_2.png) [@InSync](https://discuss.python.org/u/InSync)\
**Post date:** [December 22, 2023, 6:59pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/10 "2023-12-22T18:59:34Z")

</div>

I’m writing a package that deals with character/string ranges (e.g., `abc`-`azz` with the “base” of only ASCII lowercase letters would yield `abc`, `abd`, …, `abz`, `aca`, …, `azz` when being iterated over).

Since `bytes` and `str` have enough similarities for even some libraries in the stdlib to support both more or less the same (e.g. [`os.PathLike`](https://docs.python.org/3/library/os.html#os.PathLike) or [`re.Pattern`](https://docs.python.org/3/library/re.html#re.Pattern)), I think support for bytes ranges would be a nice feature to have.

I know that `bytes` objects do not generally represent text, but my package is called `character_range`: The user should be fully aware that if `bytes` objects are passed as arguments to the package’s API they would be interpreted (nearly) the same as normal ASCII characters (so `b'abc'`-`b'azz'` would yield `b'abc'`, `b'abd'`, etc.) and not just integers in the range `[0, 0xFF]`.

---

<div class="post-metadata">

**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\
**Post date:** [December 22, 2023, 7:08pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/11 "2023-12-22T19:08:19Z")

</div>

> [@InSync](#):
>
> (so `b'abc'`-`b'azz'` would yield `b'abc'`, `b'abd'`, etc.)

If the purpose is to build new `bytes` objects, then you don’t need to split into single-byte `bytes` anyway. The integers should actually be more convenient for the reassembly step after whatever logic you have for the range. For example:

```auto
>>> import itertools
>>> # Of course, this equally works with `b'abc'`, but the point here
>>> # is to show that we are explicitly working with integers.
>>> list(map(bytes, itertools.product(range(97, 100), repeat=3)))
[b'aaa', b'aab', b'aac', b'aba', b'abb', b'abc', b'aca', b'acb', b'acc', b'baa', b'bab', b'bac', b'bba', b'bbb', b'bbc', b'bca', b'bcb', b'bcc', b'caa', b'cab', b'cac', b'cba', b'cbb', b'cbc', b'cca', b'ccb', b'ccc']

```

If our input used a sequence of single-byte `bytes`, we’d have to concatenate them, which isn’t any easier:

```auto
>>> list(map(b''.join, itertools.product((b'a', b'b', b'c'), repeat=3)))
[b'aaa', b'aab', b'aac', b'aba', b'abb', b'abc', b'aca', b'acb', b'acc', b'baa', b'bab', b'bac', b'bba', b'bbb', b'bbc', b'bca', b'bcb', b'bcc', b'caa', b'cab', b'cac', b'cba', b'cbb', b'cbc', b'cca', b'ccb', b'ccc']

```

I would _guess_ that working with the integers would be more performant, too, but of course one never knows without a benchmark.

---

<div class="post-metadata">

**Author:** ![InSync](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/insync/32/12531_2.png) [@InSync](https://discuss.python.org/u/InSync)\
**Post date:** [December 22, 2023, 7:25pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/12 "2023-12-22T19:25:52Z")

</div>

I did implement my own base-`n` integers to do the counting part. The need arised when I needed to test the code, which requires checking that iterating a range of `b'a'`-`b'z'` yields the expected 26 `bytes` objects representing the letters in `string.ascii_lowercase`.

In other words, I’m too lazy to write out all 26, so I tried `string.ascii_lowercase.encode('utf-8').split(b'')` and the rabbit hole began.

---

<div class="post-metadata">

**Author:** ![gcewing](https://avatars.discourse-cdn.com/v4/letter/g/3d9bf3/32.png) [@gcewing](https://discuss.python.org/u/gcewing)\
**Post date:** [December 22, 2023, 11:10pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/13 "2023-12-22T23:10:29Z")

</div>

That’s what was thought at the time, but subsequently bytes objects  
regained a lot of the functionality that strings had in 2.x, because  
people found it useful. If we were designing it with the knowledge  
we now have, we might have done things differently.

---

<div class="post-metadata">

**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\
**Post date:** [December 23, 2023, 6:49am UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/14 "2023-12-23T06:49:32Z")

</div>

They find it useful because of other people’s hacks that inappropriately pretend the data represents text inherently. Unicode is now older than ASCII was when Unicode first appeared.

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [December 23, 2023, 1:21pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/15 "2023-12-23T13:21:33Z")

</div>

There are important use cases for bytes with text in them.  
HTTP for example and other protocols.

That iterating over a string that yields length 1 strings has been talked about as a mistake that python 3 should have not have continued from python 2.

But that opens up the whole code-point vs. grapheme-cluster debate.

---

<div class="post-metadata">

**Author:** ![Rosuav](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rosuav/32/3429_2.png) [@Rosuav](https://discuss.python.org/u/Rosuav)\
**Post date:** [December 23, 2023, 2:03pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/16 "2023-12-23T14:03:37Z")

</div>

> [@barry-scott](#):
>
> That iterating over a string that yields length 1 strings has been talked about as a mistake that python 3 should have not have continued from python 2.

Out of interest, who calls it a mistake? It’s certainly a quirk, but it’s also an extremely practical one, and I generally prefer subscripting of strings to yield strings rather than integers. Although it’s perhaps worth separating subscripting from iterating here; maybe it would have been better to change iteration but keep subscripting? But that would break other things.

---

<div class="post-metadata">

**Author:** ![barry-scott](https://avatars.discourse-cdn.com/v4/letter/b/e9c0ed/32.png) [@barry-scott](https://discuss.python.org/u/barry-scott)\
**Post date:** [December 23, 2023, 2:25pm UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/17 "2023-12-23T14:25:16Z")

</div>

> [@Rosuav](#):
>
> Out of interest, who calls it a mistake?

One of the core devs as I recall - but I cannot figure out how to web search for a reference. I recall it being discussed but I cannot recall the context.  
Could have been during python 3 design or shortly after 3.0 discussing a problem that this change would have solved.

FYI I’ve been reading python dev for \>25 years somewhere in there I suspect I read this. Or maybe on this forum.

---

<div class="post-metadata">

**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\
**Post date:** [December 24, 2023, 8:02am UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/18 "2023-12-24T08:02:53Z")

</div>

> [@barry-scott](#):
>
> But that opens up the whole code-point vs. grapheme-cluster debate.

Putting that aside, one would need to establish a _type_ for whatever was subsequently considered to be an element of a string. Even code points aren’t really integers; their mapping to text characters (!= graphemes) _really is_ determined, by the Unicode standard, which is global this time (and even includes a bunch of stuff that one wouldn’t ordinarily think of as textual, while still having plenty of expansion room).

---

<div class="post-metadata">

**Author:** ![Rosuav](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rosuav/32/3429_2.png) [@Rosuav](https://discuss.python.org/u/Rosuav)\
**Post date:** [December 24, 2023, 8:55am UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/19 "2023-12-24T08:55:36Z")

</div>

> [@kknechtel](#):
>
> Even code points aren’t really integers; their mapping to text characters (!= graphemes) _really is_ determined, by the Unicode standard

I’m not sure what you mean by that. Quoting from the Unicode spec:

> **[ch01.pdf](https://www.unicode.org/versions/Unicode15.0.0/ch01.pdf)**
>
> 167.59 KB

“”"  
The Unicode Standard specifies a numeric value (code point) and a name for each of its  
characters.  
“”"

In other words, the code point IS the integer value, and it, along with the name (eg “LATIN SMALL LETTER Q”) , are fundamental parts of a character’s definition.

---

<div class="post-metadata">

**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\
**Post date:** [December 24, 2023, 10:31am UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/20 "2023-12-24T10:31:51Z")

</div>

Ah, good point, I got the terminology wrong.

The argument I want to make is that the string conceptually consists of _characters_, not the corresponding integral code points. Whether you consider the characters clustered into graphemes or separately, they’re a meaningful abstraction that shouldn’t be equated with the code point.

---

<div class="post-metadata">

**Author:** ![Stefan2](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stefan2/32/18492_2.png) [@Stefan2](https://discuss.python.org/u/Stefan2)\
**Post date:** [December 24, 2023, 10:46am UTC](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709/21 "2023-12-24T10:46:28Z")

</div>

Another short one (I like the “trick” of using `zip` on a single iterable):

```python
[*map(bytes, zip(b'abc'))]

```

I was curious about speed, so …

> **Benchmark results**
>
> ```auto
> bs = b'abc'
> 0.7 μs [bytes([byte]) for byte in bs]
> 0.5 μs [byte.to_bytes(1, 'big') for byte in bs]
> 1.0 μs re.findall(b'.', bs, flags = re.S)
> 0.4 μs list(map(int.to_bytes, bs))
> 0.7 μs [bs[i:i+1] for i in range(len(bs))]
> 0.7 μs [*map(bytes, zip(bs))]
> 
> bs = string.ascii_lowercase.encode('utf-8')
> 4.7 μs [bytes([byte]) for byte in bs]
> 2.5 μs [byte.to_bytes(1, 'big') for byte in bs]
> 4.0 μs re.findall(b'.', bs, flags = re.S)
> 1.6 μs list(map(int.to_bytes, bs))
> 3.0 μs [bs[i:i+1] for i in range(len(bs))]
> 3.1 μs [*map(bytes, zip(bs))]
> 
> bs = bytes(range(256))
> 40.8 μs [bytes([byte]) for byte in bs]
> 20.1 μs [byte.to_bytes(1, 'big') for byte in bs]
> 29.4 μs re.findall(b'.', bs, flags = re.S)
> 12.7 μs list(map(int.to_bytes, bs))
> 24.4 μs [bs[i:i+1] for i in range(len(bs))]
> 25.5 μs [*map(bytes, zip(bs))]
> 
> bs = bytes(range(256)) * 100
> 4.1 ms [bytes([byte]) for byte in bs]
> 2.6 ms [byte.to_bytes(1, 'big') for byte in bs]
> 3.0 ms re.findall(b'.', bs, flags = re.S)
> 1.8 ms list(map(int.to_bytes, bs))
> 2.9 ms [bs[i:i+1] for i in range(len(bs))]
> 3.1 ms [*map(bytes, zip(bs))]
> 
> ```

> **Benchmark script**
>
> ```python
> from timeit import repeat, default_timer as time
> 
> codes = '''
> [bytes([byte]) for byte in bs]
> [byte.to_bytes(1, 'big') for byte in bs]
> re.findall(b'.', bs, flags = re.S)
> list(map(int.to_bytes, bs))
> [bs[i:i+1] for i in range(len(bs))]
> [*map(bytes, zip(bs))]
> '''.strip().splitlines()
> 
> def test(case, number, scale=6, unit='μs'):
> print(case)
> setup = f'''
> import re, string
> bs = {case}
> '''
> t0 = time()
> for code in codes:
> t = min(repeat(code, setup, number=number, repeat=100)) / number
> print(f'{t*10**scale:4.1f} {unit} ', code)
> # print(time() - t0)
> print()
> 
> test("b'abc'", 10**3)
> test("string.ascii_lowercase.encode('utf-8')", 10**3)
> test("bytes(range(256))", 1)
> test("bytes(range(256)) * 100", 1, 3, 'ms')
> 
> ```
> 
> [Attempt This Online!](https://ato.pxeger.com/run?1=dVJBbtwwDAR69CuI5CDJUVy7aYJgAb-ix2CxkL3yVoAsG5KMIFnsS3rJpX1F39AH9AP9RknJuyha1Bfb5HA4Q_LLt_klfp7c29vXJQ63jz_f_Rr8NEI0ozYRzDhPPoLXs1ZRwl4ParFxR1kPKiRYUfTTXgdogTFWPHUvUQeeXlsBw-SBPsE46MI2p6s47TKskcA6c2D_Ar2uBuP2ylresYpJDEoYrDpQI0x-EoU1IfJRzdy4eKEkoBDYJzyZjblptonZEK1X7qC51Y4TBLWUVLxWvZp5DaOLKkSP_6IKszXRGodSRVGgfUBw5L0KWoJbxk57CaFXVrcPEhZnYst-fA9MbArAZ_YoLaFF-g86LjPqH2hSl9kiA7Zzh6Ijb0eCn0hFKok1xmjMPFOQG5o3GUpzz50SFJGjcTxvi1NW5pZnre1Zcoa0TV0LAe_X7IUoyx7YMZZNXZbJ3-Zj1QwnOJLHE-A-iD5Lul4Lskq4Rc3iD_s4uDS0q46prmdXEoj0TqzR7L1SoTdmZ6dn7WkAlXbUgLN0lUz8XZXPJ2_0w_2DSID_JqHE6pogEu7w5EbcUL739ezP5_8b)

[Next page](https://discuss.python.org/t/get-single-byte-bytes-objects-from-a-bytes-object/41709.md?page=2)