Help: json.loads() cannot parse valid json

Yeah, here is a real example I’m struggling with. It’s absolutely the same string :slight_smile:

import json
dummy_response = '{"overriding_parameters": {"jar_params": ["{\"aggregationType\":\"Type1\",\"startDate\":\"2022-05-10\",\"endDate\":\"2022-05-10\"}"]}}'
json.loads(r'{"overriding_parameters": {"jar_params": ["{\"aggregationType\":\"Type1\",\"startDate\":\"2022-05-10\",\"endDate\":\"2022-05-10\"}"]}}') # works fine - test it out
json.loads(dummy_response) # raises JSONDecodeError
print(dummy_response) # {"overriding_parameters": {"jar_params": ["{"aggregationType":"Type1","startDate":"2022-05-10","endDate":"2022-05-10"}"]}}
print(repr(dummy_response)) # '{"overriding_parameters": {"jar_params": ["{"aggregationType":"Type1","startDate":"2022-05-10","endDate":"2022-05-10"}"]}}'

Prints have different in the first single quote in repr.

Like I said, dummy_response (the contents of the variable after assignment) is not real JSON because you’re losing the backslashes. But you’re not really typing it in, I hope. Is that what the real response gives you?

In other words, the text is JSON, but the contents of the variable is not. r"" only applies to typed in data, not to data received from a function. What’s the real data received from the function? Presumably the backslashes are already lost by that point?

No, this is a real response from the server. This response is pushed to airflow's xcom and then I read it. The response code is 200, and the response text itself is what I’ve shared.

Can you run this:

response = kwargs['ti'].xcom_pull(task_ids='get_run_list')
print(repr(response))
json.loads(response) 

(With a real request, not with a string literal in the code.)

Does it work? What does it print?

But we haven’t seen that. It is significant exactly how the text is encoded in the response. Can you actually show the repr() of the response rather than from something typed in?