Use of struct.unpack python

Python Help
1

Hi,

I’m sending an float array with my microcontroller with serial communication. I try to read it with python, but I’m struggling with how to do it. I did some research on the internet. I think I should use struct.unpack. I tried some ways but without succes.

I’m sending an array what looks like this in python:

b'\xcdL\x07=\xfc\x00\x00 \x00\x00\x00\x00\x98#\x00 \x01\x00\x00\x00\x00\x00\x04= \x01\x00 \x00\x10\x00@\x00\x00\x00\x00\x01\x00\x00\x00\x9a\x91\xaa?D\x85\x00@\x0c\x0c\x00 \x11\xc6\x01\x00$\x0c\x00 f\xe6.@\x00\x00\x00\x00\xe8\x1d\x00 \x00\x00\x00\x00\xdc\x00\x00\r\n'

This is my python script what I tried, but I know it isn’t the right way. Can someone help me out?

import serial
import matplotlib.pyplot as plt
import struct

plt.ion()
fig=plt.figure()


i=0
x=list()
y=list()
i=0
ser = serial.Serial('COM7',115200)
ser.close()
ser.open()
while True:

    data = ser.readline()
    print(struct.unpack('<f', bytearray(data)))

Kind regards,

JP

2

What happens when you run your code?

Do you get an exception? If so, then please copy and paste (no screen shots!) the full traceback, starting from the line Traceback… to the final error message.

Otherwise, do you get the wrong results? Please tell us what you expect, and what you get instead.

By the way, reading floats using readline is risky. If one of the floats happens to include the byte 0x0A anywhere inside the float, it will be interpreted as a newline and truncate the read, leaving you with half the float at the end of your input and the other half still in the serial port’s buffer.

For example, the float (approximately) 6.788903e-33 would be the bytes b’\0\0\r\n’. The float (approx) 1.285700e-07 would be the bytes b’\x12\r\n\x34’.

Since you are reading 4-byte (32 bit) floats, it is probably safer to read 4 bytes and process one float at a time, and just stop transmitting when you have no more floats to send.

1 Like
3

Yesterday you opened one topic about your serial communication. I offered you a solution for the NUL character which seemed to be unwanted:

I think the solution should work but we have no response from you and you left us speculating a) Why is the NUL there? b) Do you have correct parameters for the serial communication? c) Do not you need flow control? etc.

Please first test if my code resolves the problem, confirm what is the source of the NUL characters and let us know about the results and possible problems with the code I offered. After that let’s continue here.

Something for the preparation here:

Please do not ask us to do reverse-engineering of the data your software produces. Discussion here should be primarily about Python, not reverse-engineering puzzles.

  1. It should be an array, so it is a stream of blocks (array items) of the same size, right?
  2. What is the exact structure of the single block? float what size, what format, endianness, padding, separators, something else?
  3. b'\xcdL\x07=\xfc\x00\x00 ' … It is really hard to read binary data in the default representation of Python bytes. Please show them in hexadecimal, show the blocks and their components. Use the hex() method:
>>> b'\xcdL\x07=\xfc\x00\x00 '.hex()
'cd4c073dfc000020'
  1. Please remove all the code not relevant to the problem (backup your file and copy the removed parts back later if needed). It is just adding distracting noise to our communication. Ideally make minimal runnable code demonstrating the problem.
I think all this should be removed (click to show): 
import matplotlib.pyplot as plt

plt.ion()
fig=plt.figure()

i=0
x=list()
y=list()
i=0
  1. Do you really need to close and open the serial communication? Please try to remove it. Also please test the code from my post (above) which should be better suited for the communication.
ser.close()
ser.open()
  1. data = ser.readline() this expects lines (newline terminated). But you said you are sending an array (binary) so we will probably need a better suited method but we will first need to know the format of the data (points 1-3).
  1. Why? What is wrong? What are the actual results? What are the expected results?
4

The “f” format character should be prefixed by the number of single-precision floats to unpack. For example:

>>> data = struct.pack('<3f', 1.0, 2.0, 3.0)
>>> struct.unpack('<3f', data)
(1.0, 2.0, 3.0)

Or use iter_unpack() to unpack an array of records. For example:

>>> [x[0] for x in struct.iter_unpack('<f', data)]
[1.0, 2.0, 3.0]
1 Like