How to read a csv file with multiple header rows into pandas?

Question

I am not sure how to read the multiple rows. Do i have to do it without the rows and then do it manually in pandas? Or is there a way to read the whole csv file into pandas?

This is how the file looks Poverty data

Solution

You need parameter header=[0,1] in read_csv.

Comments

[Caribgirl]

I tried that then this error came up "Error tokenizing data. C error: Expected 1 fields in line 107, saw 3"

[jezrael]

It seems there are bad data, can you check line 107? Or is possible remove all problematic rows?

[jezrael]

If need remove problematic rows need error_bad_lines=False

[Trisa Biswas]

I tried doing this, but it is taking my first column as the index. How do I fix it?

[jezrael]

@TRISABISWAS - It is pain, mainly it depends of data, try index_col=[0]

[Trisa Biswas]

index_col=[0] and index_col=None both are giving me same result.

[Trisa Biswas]

I did a workaround by adding an index column in my excel sheet. Please let me know if there is any other efficient way to achieve this using pandas. Thank you.

[jezrael]

@TRISABISWAS - really hard way, because it depends of data, so find working solution is problem.... I have no another idea like index_col=[0], so sorry :(