parse date-time while reading 'csv' file with pandas

Question

I am trying to parse dates while I am​ reading my data from cvs file. The command that I use is

df  = pd.read_csv('/Users/n....', names=names, parse_dates=['date'])​ ) 

And it is working on my files generally. But I have couple of data sets which has variety in date formats. I mean it has date format is like that (09/20/15 09:59​ ) while it has another format in other lines is like that ( 2015-09-20 10:22:01.013​ ) in the same file. And the command that I wrote above doesn't work on these file. It is working when I delete (parse_dates=['date'])​, but that time I can't use date column as datetime format, it reads that column as integer . I would be appreciate anyone could answer that!

Solution 1

Pandas read_csv accepts date_parser argument which you can define your own date parsing function. So for example in your case you have 2 different datetime formats you can simply do:

import datetime

def date_parser(d):
    try:
        d = datetime.datetime.strptime("format 1")
    except ValueError:
        try:
            d = datetime.datetime.strptime("format 2")
        except:
            # both formats not match, do something about it
    return d

df = pd.read_csv('/Users/n....', 
                 names=names, 
                 parse_dates=['date1', 'date2']),
                 date_parser=date_parser) 

You can then parse those dates in different formats in those columns.

Solution 2

Like this:

df = pd.read_csv(file, names=names)
df['date'] = pd.to_datetime(df['date'])