Convert Pandas time series: UNIX epoch to datetime

Question

I'm tying to convert the following series of UNIX epochs to regular datetime objects:

>> val = pd.Series(["1440643875", "1440644191", "1440645638", "1440998720"])
>> val
0    1440643875
1    1440644191
2    1440645638
3    1440998720
Name: obj, dtype: object

There appears to be two ways of doing this. The first is:

>> pd.to_datetime(val, unit='s')
ValueError: year is out of range

And the second:

val.astype("datetime64[s]")
TypeError: Cannot parse "1445124547" as unit 's' using casting rule 'same_kind'

What seems to be the problem here?

I also tried checking these timestamps with the "Online Epoch Calculator" tools, and they give out reasonable answers..

Solution 1

The issue was that the elements were strings, and not ints. Apparently, pd.to_datetime() isn't smart enough to convert from strings to datetime.

My solution was this:

>> val.astype('int').astype("datetime64[s]")
0   2015-08-27 02:51:15
1   2015-08-27 02:56:31
2   2015-08-27 03:20:38
3   2015-08-31 05:25:20
dtype: datetime64[ns]

Solution 2

We can directly convert the epoch time to datetime. By default it will be in %Y-%m-%d %I:%M:%S format by using pd.to_datetime. By using dt.strftime complete column can be formatted in the required format.

from datetime import datetime as dt
import pandas as pd
input_data_df['timestamp']=pd.to_datetime(input_data_df['epoch'],unit='ms')
input_data_df['timestamp'] = input_data_df['timestamp'].dt.strftime('%d-%m-%Y %I:%M:%S')

Comments

[user1496984]

Try quoting the elements of the series s, as in s = pd.Series(["1440643875", ...], dtype=object)

[Anton Protopopov]

you are right, first you need to convert it to int, then use that methods

[user1496984]

Yes, that would work. I personally find the chained astype methods more descriptive and easier to grasp.