Trouble with pandas and to_csv, getting [brackets] inside csv file

Python Help
1

Python 3.9 on Windows 10 Pro. I’m still new to Python. I’m trying to create my own program to practice.

I have to read several XML files and pick out data from them. Then for each XML file I have to construct one line for the CSV file. The real CSV file I’m writing is very odd and has a row of headers, a row of data, 2 blank lines, then unlimited rows of data. So I broke some tasks into functions in my main program which is not shown here. This example is the minimum required to show the problem.

Here’s the mini program and the problems I’m having.

# Created by xxx on 3/18/2024 10:51 AM
import inspect
import pandas
import warnings

csvlist = []
csvbody = []

# Functions/classes
def addsome():
    global csvbody
    tlist = ['One', 2, 3, 4]
    csvbody.append(tlist)
    tlist = [5,6,7,8]
    csvbody.append(tlist)

def writecsv(csvlist,csvfile):
    """Function writecsv:
    csvlist = a list of values in pandas dataframe format.
    csvfile = filename to write to ending in .csv.
    Ex: csvlist=[['Hdra', 'Hdrb', 'Hdrc'],
         ['Row2a', 'Row2b', 'Row2c'],
         ]
    """
    frame = inspect.currentframe()
    subname = frame.f_code.co_name + ": " # Get subroutine name for error messages.
    df = pandas.DataFrame(csvlist) # Make a dataframe from our csvlist.
    df.to_csv(csvfile, index=False, header=False, mode='w')  # This writes the file
    print(subname,"Wrote",csvfile)

# Main program
warnings.filterwarnings("ignore", category=DeprecationWarning)

hdr = ["Hdr1", "Hdr2", "Hdr3"]
csvlist.append(hdr)
addsome() # Add some rows to csvbody.
csvlist.append(csvbody)

"""
And the test.csv file looks wrong, I get:
Hdr1,Hdr2,Hdr3
"['One', 2, 3, 4]","[5, 6, 7, 8]",

Expected output for test.csv: 
Hdr1,Hdr2,Hdr3
'One', 2, 3, 4
5, 6, 7, 8

"""
writecsv(csvlist,'test.csv')

I’ve looked at 5-6 web pages already and tried several things to make this work but I must be missing something. Can someone help?

Thank you.

2

I could be wrong but I think your dataframe is wrongly shaped at the point of writing the CSV. You have csvbody which is a list of lists:

[['One', 2, 3, 4],[5,6,7,8]]

while csvlist becomes

[["Hdr1", "Hdr2", "Hdr3"]]

after appending hdr. So the csvlist.append(csvbody) would look like this:

[[['One', 2, 3, 4],[5,6,7,8]],["Hdr1", "Hdr2", "Hdr3"]]

If you try to extend your csvlist with csvbody it should work:

csvlist.extend(csvbody)

Because both csvlist and csvbody are list-valued at the same level.

3

You were right! Thank you!

The part of code I fixed:

hdr = ["Hdr1", "Hdr2", "Hdr3"]
csvlist.append(hdr)
addsome() # Add some rows to csvbody
csvlist.extend(csvbody) # <-- Fixed to use .extend()
1 Like