Downloading a csv file using requests, but getting the header and malformed csv

Python Help
1

I’m trying to download a csv file from a server that uses a basic authkey in the header.

On the webpage, there is just a text box for the Authkey and a button to download the csv file.

So far I can authenticate and print the data using.

url = 'https://example.com/stats/data/'
keyLogin = {"from":"2023-2-28T0:0:0Z","to":"2023-2-28T23:59:59Z","lists":"%%All%%","authKey":"pythonKey","timeOffset":0}

x = requests.post(url, json = keyLogin)

f = open('data.txt', 'wb')
f.write(x.text.encode('utf8'))
f.close()

The file generated by python, is saved as one continuous line, with no line breaks and also still has the JSON header followed by the data I need with slashes \ separating the data and the \r\n line returns

{"statusCode": 200, "headers": {"Content-Type": "text/csv; charset=utf-8-sig", "MIME-Type": "text/csv", "Content-Disposition": "attachment; filename=tooldata.csv.csv"}, "data": "\ufeff\"date\",\"connected.time\",\"lastupdate.time\",\"tool.number\",\ etc etc etc file string continues

This is the same data I can see under Chromes Dev Tools under [Network] > [Response] when i click the webpage download button.

Help! Looking for some pointers to investigate and keywords to search for.

2

Use the json module to extract the information you need to write to the file from the response.

You will parse the json in to python objects then you will need to walk that tree of objects to find your CSV data so that you can write it.

3

Thanks, I’ll research that and see how I get on.