PySerial not sending string to serial port of Arduino

Python Help
21

Just for the record, while discussing with another friend of mine about the script, he found this:

However, I’m not using Arduino but the TTL converter might also have the DTR pin mentioned in this reply from stackoverflow!

22

Just for the record, while discussing with another friend of mine about
the script, he found this:
python - Have to use sleep with pyserial when opening com port for arduino nano - Stack Overflow

Could you post your current working code so that we can see where the
sleep went, etc?

However, I’m not using Arduino but the TTL converter might also have
the DTR pin mentioned in this reply from stackoverflow!

I’m curious to know your exact setup. (I’ve got nothing like this here,
this is just my curiosity.) Do you have any control over the receiving
end or is it a black box to you? I think I’m wondering how miuch insight
you can can about the receiving end state can be gained.

Cheers,
Cameron Simpson cs@cskk.id.au

23

Hello @cameron.

The time.sleep(1.8) is right before the ser.write() command. The Serial object creation automatically opens the port, if it was provided. So, between the opening of the port and the write command, the stackexchange situation mentioned there might need some time to deal with it. I don’t know to explain exactly what is going on with DTR pins but the time.sleep() seemed to fix it.
Tomorrow I’ll post the code. I’m at home and about to go to sleep.

Thnaks
Psy

24

Thank you.

Cheers,
Cameron Simpson cs@cskk.id.au

25

I’m sorry I couldn’t do it toay. I had a family emergrncy with my father today and I had to leave work before lunch time.
I’ll do it tomorrw!
Psy

26

Please, take whatever time you need. You don’t owe us anything!
Good luck with your father.
Cheers, Cameron

27

Hello,
as promised, here is the code:

import serial, json
import time

ser = serial.Serial(
        port = '/dev/ttyACM0',
        baudrate = 9600,
        timeout = 10,
        xonxoff = False
)
val = 0
command = ''
time.sleep(1.8)
command = str('getData')
command += str('\n')
val = ser.write(command.encode(encoding = 'ascii', errors = 'strict'))
in_data = ''
in_data = ser.read_until(b'}')
print(json.dumps(json.loads(in_data.decode()), indent = 4))

I think I got confused when I mentioned a TTL converter and a DTR pin. Ignore the UART-to-TTL converter
The setup is the following:

                                                            |---------> Sensors
Windows PC -- SSH session --> Rpi -- USB cable -- Arduino --|---------> Sensors
                                                            |---------> More Sensors

Then, in the Windows PC, we access a site that is showing the information gathered in that JSON object the script outputs. The site setup is not known to me. I just know the data from JSON objects is being recorded in a MongoDB, and my colleague has a software running in the Rpi called Tomcat that is related to this (not sure how though).

28

as promised, here is the code:

Thank you!

[…]

in_data = ser.read_until(b’}')
[…]

I was curios about this. The command stream is newline delimited eg
b'getData\n' - is the JSON response newline delimited.

You’re scanning to a closing '}', which will work if the JSON is
fairly flat. And if you’re getting a row of sensor readings I guess it
will be flat. But in theory JSON can contain nested mappings (very
common in, say, web API responses) eg:

 { "foo": 1.3, "bah": {"this": "that"} }

and I was wondering if in fact your device was sending “newline
delimited JSON” which is JSON guarrenteed to be all-on-one-line, with a
trailig newline. This is effectively lines of text, each line being a
JSON record.

Again, just curiosity and I’m sure you’ve wasted enough time on this
anyway.

Thank you again,
Cameron Simpson cs@cskk.id.au

29

I didn’t understand this. Were you asking if the JSON in the Arduino is \n delimited? If so, yes. My colleague built the JSON in Arduino code with a \n per line of the JSON object.
In our case, we won’t have nested lists/arrays, so, we are reading until we find the closing bracket. But yes, I understand that this is a very poor solution in case there are nested objects in the JSON.

30

You could terminate with, say, a null byte b'\x00'.

31

There’s no need for that with newline-delimited-JSON, because it has no
embedded newlines in the data component. - Cameron

32

Nested data are not the only issue. You can also have strings containing
a closing bracket, eg:

 { "size":9, "label": "I saw a {thing}" }

Your data may be entirely numeric etc. But if you know you’re getting
newline delimited records, it is best to parse them that way in 2 steps:

  • read the record by stopping at a newline
  • parse the JSON out of the record

(Handily, you code only needs to change the delimiter - the
json.loads() call is perfectly happy with the trailing newline in the
string.)

This brings a few benefits:

  • it accords with the actual record spec instead of unstated conditions
    about the record contents
  • it consumes the record terminator! leaving you correctly in sync with
    more data
  • it leaves scope to embed different data in the future (nested
    structure or strings containing closing curly brackets) without having
    to make any changes to the JSON-reading code
  • if there’s a bug in the sending code, writing bad JSON, you’re still
    in sync with the lines of data

That last: suppose something buggy wrote:

 { "a": 1, "b": 2, "c":}5, "d":6}

sometimes. By reaing whole lines you’re in sync: complain about this
record, pick up and read the next one. If you stopped at the }5 point
you’ll see 2 bad records.

I think I’m basicly saying: parse the defined structure, not some
arbitrarily chosen aspect of the current data. It is more robust and
flexible.

Cheers,
Cameron Simpson cs@cskk.id.au

33

Ok, I can try if I’m sure I understood the suggestion.
The data is being sent from the Arduino to the Serial port like this:

  String dataAux="{\n";
 
  dataAux+="\"temperature_celsius\":";
  dataAux+=temperature;
  dataAux+=",\n";
 
 
  dataAux+="\"humidity_percentage\":";
  dataAux+=humidity;
  dataAux+=",\n";
 
 
  float pressure = 0.0;
  dataAux+="\"pressure_kpa\":";
  dataAux+=pressure;
  dataAux+="}";

Then, there is another line of code sending the entire dataAux variable to the Serial port, and therefore, as he uses Serial.println(dataAux), there is a last new linechar after the closing bracket.

So, what you suggest is to read the JSON object, line by line, save each line in a new variable and then, print it all to the stdout?

34

Ah, you can ignore my suggestion then. I was wondering if this was sending newline delimited JSON, which looks like this:

"{\"temperature_celsius\":3.9}\n"

i.e a newline only after the end of the whole JSON record. Scuh a stream can be read as text, with one complete JSON record per text line.

What you’ve got is not structured like that, and since it is flat (no nesting, and hopefully no field names containing a }) your approach is reading to the next closing } is the best approach.

I wasn’t suggesting anything for the sending (Arduino) side, just your receiving end.

Cheers,
Cameron

2 Likes