# Unusal behavior of python3\* print hex values

**URL:** <https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418>\
**Category:** Python Help\
**Created:** [May 2, 2022, 6:10am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418 "2022-05-02T06:10:52Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![trashford](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@trashford](https://discuss.python.org/u/trashford)\
**Post date:** [May 2, 2022, 6:10am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/1 "2022-05-02T06:10:52Z")

</div>

I’m new to python but have expierences on other languages.  
I discoverd an unusal behavior of python3\* on my kali linux. All I want was to print an output of 4 values given in hex:

This is OK and works how it should be:  
Python 2.7.18 (default, Mar 28 2022, 20:47:09) [GCC 11.2.0] on linux2  
$ python2.7 -c ‘print ("\x37\x8a\x04\x08");’ | hexdump -C  
00000000 37 8a 04 08 0a |7…|

This is my problem:  
Python 3.10.4 (main, Mar 24 2022, 13:07:27) [GCC 11.2.0] on linux  
$ python3.10 -c ‘print ("\x37\x8a\x04\x08");’ | hexdump -C  
00000000 37 c2 8a 04 08 0a  
-------------------^^ ??

Python 3.9.12 (main, Mar 24 2022, 13:02:21) [GCC 11.2.0] on linux  
$ python3.9 -c ‘print ("\x37\x8a\x04\x08");’ | hexdump -C  
00000000 37 c2 8a 04 08 0a |7…|  
-------------------^^ ??

Did I something wrong ? Any suggestions ?  
Thanks to the community for help.

---

<div class="post-metadata">

**Author:** ![sweeneyde](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sweeneyde/32/3689_2.png) [@sweeneyde](https://discuss.python.org/u/sweeneyde)\
**Post date:** [May 2, 2022, 6:34am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/2 "2022-05-02T06:34:19Z")

</div>

According to [this Stackoverflow post by Benjamin Peterson](https://stackoverflow.com/questions/908331/how-to-write-binary-data-to-stdout-in-python-3), you can use

```auto
import sys
sys.stdout.buffer.write(b"\x37\x8a\x04\x08")

```

In Python 3, strings are Unicode strings by default, and in order to use byte-strings, you have to convert things explicitly using `str.encode` and `bytes.decode`

---

<div class="post-metadata">

**Author:** ![trashford](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@trashford](https://discuss.python.org/u/trashford)\
**Post date:** [May 2, 2022, 7:44am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/3 "2022-05-02T07:44:13Z")

</div>

Thank you - that helps. I will keep this in mind.

---

<div class="post-metadata">

**Author:** ![rob42](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rob42/32/8218_2.png) [@rob42](https://discuss.python.org/u/rob42)\
**Post date:** [May 2, 2022, 7:51am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/4 "2022-05-02T07:51:23Z")

</div>

I understand why the `0a` at the end is there in the output, but why is the `c2` there in the output?

---

<div class="post-metadata">

**Author:** ![BowlOfRed](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bowlofred/32/3379_2.png) [@BowlOfRed](https://discuss.python.org/u/BowlOfRed)\
**Post date:** [May 2, 2022, 8:48am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/5 "2022-05-02T08:48:57Z")

</div>

Because the default encoding for the stdout filehandle (on linux) is utf-8. The encoding for \x80 is \<\xc2, \x80\>.  
[https://www.utf8-chartable.de/](https://www.utf8-chartable.de/)

---

<div class="post-metadata">

**Author:** ![rob42](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rob42/32/8218_2.png) [@rob42](https://discuss.python.org/u/rob42)\
**Post date:** [May 2, 2022, 9:00am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/6 "2022-05-02T09:00:56Z")

</div>

Yes. I see that now, thanks.

So, with Python 2.7 (or maybe any 2.x) is the `print()` function handling this is a different way, hence the different output?

---

<div class="post-metadata">

**Author:** ![BowlOfRed](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bowlofred/32/3379_2.png) [@BowlOfRed](https://discuss.python.org/u/BowlOfRed)\
**Post date:** [May 2, 2022, 9:10am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/7 "2022-05-02T09:10:13Z")

</div>

It’s not really `print` itself being different. It’s that python2 `str`s are just bytes, while they’re unicode in python3. And that the default encoding on stdout is `ascii` in python2 and `utf-8` in python3.

---

<div class="post-metadata">

**Author:** ![rob42](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rob42/32/8218_2.png) [@rob42](https://discuss.python.org/u/rob42)\
**Post date:** [May 2, 2022, 9:15am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/8 "2022-05-02T09:15:03Z")

</div>

Thank you. I get it.

I was looking at the bit patterns and noticed the output change when bit 7 became set, but it seems that while my ‘bark’ may have been correct, I had the wrong ‘tree’.

---

<div class="post-metadata">

**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\
**Post date:** [May 2, 2022, 9:46am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/9 "2022-05-02T09:46:40Z")

</div>

The Python 2.7 example uses a _byte-string_. It outputs the bytes

- \x37 = ‘7’ in ASCII
- \x8a = undefined in ASCII; VTS (Vertical Tab Set) in Latin-1; ‘ä’ in MacRoman
- \x04 = EOT (End of Transmission) in ASCII
- \x08 = BACKSPACE in ASCII

followed by a newline (linefeed) generated by the print statement.

The Linux hexdump utility sees those five bytes.

The Python 3 examples use a _Unicode text string_. It outputs the Unicode characters:

- ‘\x37’ = ‘7’
- code point U+008A (VTS)
- code point U+0004 (EOT)
- code point U+0008 (BACKSPACE)

plus the newline generated by print. These have to be converted to bytes before being written to stdout.

On almost all Linux systems, the encoding used to convert Unicode characters to bytes is UTF-8.

In UTF-8, all of those five Unicode characters except the VTS are encoded to a single byte with the same numeric value as the escape code, but the VTS character is encoded to a double-byte sequence 8a 04 (in hex).

So hexdump sees six bytes, not five.

You can learn more about Unicode and encodings here:

[The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)](https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/)

[Pragmatic Unicode](https://nedbatchelder.com/text/unipain.html)

---

<div class="post-metadata">

**Author:** ![rob42](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rob42/32/8218_2.png) [@rob42](https://discuss.python.org/u/rob42)\
**Post date:** [May 2, 2022, 10:35am UTC](https://discuss.python.org/t/unusal-behavior-of-python3-print-hex-values/15418/10 "2022-05-02T10:35:00Z")

</div>

Thanks for posting the extremely useful links.

These go a long way to explain why I get a strange looking cli prompt when I ssh into one of my Linux boxes:  
` NUC6-1  rob  ~ `
