pandas.errors.ParserError: Error could possibly be due to quotes being ignored when a multi-char delimiter is used

Question

I am getting a ParserError when I am trying to read a csv file using Pandas. Given below is the error and the data set that threw this error.

pandas.errors.ParserError: Expected 10 fields in line 8, saw 11. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.

Given below is the line 8 that has this error

10/29/18 10:20,85505306,    Scott,20181029102023-file.csv,  22.49,-12.18,CITY,,12:15.0,51:00.0,ABCD,9898,320,D231

I am reading the csv using the below command:

df.to_csv('file.csv'), index = False)

Sample output of the csv file:

File_Received_Time  Label1  City    FileName    Label2  Label3  State   Unnamed: 12 cTimestamp  dTimestamp  Label4  Label5  Label6  Label7  Label8
10/29/18 10:20  56776   Paris   file1.csv   29  29  IL      29-10-2018 04:11:11     COL06   620 398 516 451
10/29/18 10:20  46069   Hongkong    file2.csv   61  58  VA      29-10-2018 04:03:17 28-10-2018 05:58:00 COL06   576 645 349 374
10/29/18 10:20  47240   Sydney  file3.csv   43  42  IL      29-10-2018 04:12:46     COL06   534 2047    56831   372
10/29/18 10:20  47432   NewYork file4.csv   55  61  OH          28-10-2018 09:01:00 COL06   514 2354    640 633
10/29/18 10:20  41794   London  file5.csv   39  29          29-10-2018 04:12:46 28-10-2018 09:01:00 COL06   470 2354    56831   550
10/29/18 10:20  49643   LA  file6.csv   55  43  TX      29-10-2018 04:05:18     COL06   523 2301    53942   403
10/29/18 10:20  54700   Shangai file7.csv   37  29  AZ      29-10-2018 04:12:15 28-10-2018 12:51:00 COL06   569 2683    53642   538
10/29/18 10:20  37134   Singapore   file8.csv   53  62  AZ      29-10-2018 04:09:16     COL06   560 391 54541   542
10/29/18 10:20  51144   Taiwan  file9.csv   43  33  TX      29-10-2018 04:12:15     COL06   469 472 458 481

Solution 1

Seems the Python Engine is the problem try reading with pandas.read_csv and set delim_whitespace=True

Solution 2

You can solve the pandas.errors.ParserError: Error could possibly be due to quotes being ignored when a multi-char delimiter is used by using:

df = pd.read_csv("train.tsv",index_col = False,sep="\t")

Comments

[RunTheGauntlet]

Helped in my case.