Pandas: Using Unix epoch timestamp as Datetime index
Solution 1
Convert them to datetime64[s]
:
np.array([1368431149, 1368431150]).astype('datetime64[s]')
# array([2013-05-13 07:45:49, 2013-05-13 07:45:50], dtype=datetime64[s])
Solution 2
You can also use pandas to_datetime:
df['datetime'] = pd.to_datetime(df["timestamp"], unit='s')
This method requires Pandas 0.18 or later.
Solution 3
You can also use Pandas DatetimeIndex like so
pd.DatetimeIndex(df['timestamp']*10**9)
the *10**9
puts it into the format it's expecting for such timestamps.
This is nice since it allows you to use functions such as .date()
or .tz_localize()
on the series.
![Nipun Batra](https://i.stack.imgur.com/66qdU.jpg?s=256&g=1)
Nipun Batra
Nipun Batra is an Assistant Professor in Computer Science at IIT Gandhinagar. He previously completed his postdoc from University of Virginia. He completed his PhD. from IIIT Delhi where he was a TCS PhD fellow. His group broadly works on machine learning/AI/sensors/IoT for computational sustainability problems like smart buildings and air quality. His work has been awarded several awards, including, the best PhD presentation at ACM Sensys, best demo at ACM Buildsys, and a best video nominee at ACM KDD.
Updated on June 16, 2022Comments
-
Nipun Batra about 2 years
My application involves dealing with data (contained in a CSV) which is of the following form:
Epoch (number of seconds since Jan 1, 1970), Value 1368431149,20.3 1368431150,21.4 ..
Currently i read the CSV using numpy loadtxt method (can easily use read_csv from Pandas). Currently for my series i am converting the timestamps field as follows:
timestamp_date=[datetime.datetime.fromtimestamp(timestamp_column[i]) for i in range(len(timestamp_column))]
I follow this by setting timestamp_date as the Datetime index for my DataFrame. I tried searching at several places to see if there is a quicker (inbuilt) way of using these Unix epoch timestamps, but could not find any. A lot of applications make use of such timestamp terminology.
- Is there an inbuilt method for handling such timestamp formats?
- If not, what is the recommended way of handling these formats?
-
Nipun Batra about 11 yearsWow! Did not know it could be so easy! The best part is that it retains the feel of a vectorized operation.