How can I make a python numpy arange of datetime

Question

I have some input data, with timestamps in the input file in the form of hours from the date time specified in the filename.

This is a bit useless, so I need to convert it to python datetime.datetime objects, and then put it in a numpy array. I could write a for loop, but I'd like to do something like:

numpy.arange(datetime.datetime(2000, 1,1), datetime.datetime(2000, 1,2), datetime.timedelta(hours=1))

which throws a TypeError.

Can this be done? I'm stuck with python 2.6 and numpy 1.6.1.

Solution 1

See NumPy Datetimes and Timedeltas. Basically, you can represent datetimes in NumPy using the numpy.datetime64 type, which permits you to do ranges of values.

For NumPy 1.6, which has a much less useful datetime64 type, you can use a suitable list comprehension to build the datetimes (see also Creating a range of dates in Python):

base = datetime.datetime(2000, 1, 1)
arr = numpy.array([base + datetime.timedelta(hours=i) for i in xrange(24)])

This produces

array([2000-01-01 00:00:00, 2000-01-01 01:00:00, 2000-01-01 02:00:00,
   2000-01-01 03:00:00, 2000-01-01 04:00:00, 2000-01-01 05:00:00,
   2000-01-01 06:00:00, 2000-01-01 07:00:00, 2000-01-01 08:00:00,
   2000-01-01 09:00:00, 2000-01-01 10:00:00, 2000-01-01 11:00:00,
   2000-01-01 12:00:00, 2000-01-01 13:00:00, 2000-01-01 14:00:00,
   2000-01-01 15:00:00, 2000-01-01 16:00:00, 2000-01-01 17:00:00,
   2000-01-01 18:00:00, 2000-01-01 19:00:00, 2000-01-01 20:00:00,
   2000-01-01 21:00:00, 2000-01-01 22:00:00, 2000-01-01 23:00:00], dtype=object)

Solution 2

from datetime import datetime, timedelta

t = np.arange(datetime(1985,7,1), datetime(2015,7,1), timedelta(days=1)).astype(datetime)

The key point here is to use astype(datetime), otherwise the result will be datetime64.

Solution 3

With modern NumPy you can do this:

np.arange(np.datetime64('2017-01-01'), np.datetime64('2017-01-08'))

And it gives you:

array(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04',
       '2017-01-05', '2017-01-06', '2017-01-07'], dtype='datetime64[D]')

Solution 4

As noted in another answer, for Numpy > 1.7, you can use Numpy's built-in datetime capability. The examples in the Numpy documentation don't include using np.arange with steps, so here's one:

timearray = np.arange('2000-01-01', '2000-01-02',np.timedelta64(1,'h'), dtype='datetime64')

Numpy sets the dtype of this result to datetime64[h]. You can set this explicitly to some smaller unit of time with dtype='datetime64[m]'.

In version 1.8.1 (and I expect earlier), trying to add an offset to that result array that is smaller than an hour will have no effect.

• timearray += np.timedelta64(10,'s') does not change timearray
• timearray2 = timearray + np.timedelta64(10,'s') will add 10 seconds to timearray and converts the dtype of timearray2 to datetime64[s]

Solution 5

Note that @nneonneo solution can be simplified in

result = first_date + np.arange(24) * datetime.timedelta(hours=1)

thanks to NumPy array manipulations. The result array has then a dtype=object.

For more complex ranges, you might be interested in the scikits.timeseries package (no longer maintained) or better, the pandas package that reimplemented most of the ideas of scikits.timeseries. Both packages support older versions of NumPy (1.5, 1.6...)

View more solutions

Comments

[Melanie]

If only I had numpy 1.7, this would be the answer. But it seems I have 1.6.1, so the example doesn't work.

[Melanie]

And is also compatible with the datetime I need to output. Thanks!

[Melanie]

Thanks - it looks like I should have been using pandas for the entire task. Next time :-)

[weefwefwqg3]

TypeError: 'module' object is not callable. Any idea? what to import first?

[weefwefwqg3]

Can I put step here? like: a week, 10 days, a month, or a year?

[John Zwinck]

@weefwefwqg3: Sure. Use np.timedelta64(...).

[jeromerg]

this is not a vectorized solution, so may be slow

[Right leg]

@weefwefwqg3 Your import is probably import datetime instead of from datetime import datetime, so datetime is not the class but the module itself.