Making matplotlib's date2num and num2date perfect inverses

Question

I'm trying to write a pair of functions, plottm and unixtm, which convert back and forth between normal unix time (seconds since 1970-01-01) and Matplotlib's date representation (days since the last day of -1BC or something, a float).

If plottm and unixtm were proper inverses then this code would print the same date/time twice:

import time, datetime
import matplotlib.dates as dt

# Convert a unix time u to plot time p, and vice versa
def plottm(u): return dt.date2num(datetime.datetime.fromtimestamp(u))
def unixtm(p): return time.mktime(dt.num2date(p).timetuple())

u = 1270000000
print datetime.datetime.fromtimestamp(u), "-->", \
      datetime.datetime.fromtimestamp(unixtm(plottm(u)))

Alas, it's off by an hour (which only happens for some timestamps, otherwise I'd insert an offset and be done with it).

Probably related: Problems with Localtime

UPDATE: Related question that isn't specific to Matplotlib: Convert a unixtime to a datetime object and back again (pair of time conversion functions that are inverses)

Solution 1

Based on @dreeves answer, a solution adapted to work with timezone aware datetimes:

import matplotlib.dates as dt

from calendar import timegm
from datetime import datetime

from pytz import utc


# Convert a unix time u to plot time p, and vice versa
def plottm(u):
    return dt.date2num(datetime.fromtimestamp(u, utc))

def unixtm(p):
    return timegm(dt.num2date(p, utc).utctimetuple())


u = 1270000000
print datetime.fromtimestamp(u, utc), "-->", \
      datetime.fromtimestamp(unixtm(plottm(u)), utc)

output (tested for several timezones):

2010-03-31 01:46:40+00:00 --> 2010-03-31 01:46:40+00:00

Solution 2

There are matplotlib.dates.epoch2num()/num2epoch functions that do exactly that:

from datetime import datetime, timedelta
import matplotlib.dates as mpl_dt

matplotlib_epoch = datetime(1, 1, 1)  # utc
posix_epoch = datetime(1970, 1, 1)  # utc
DAY = 86400  # seconds


def plottm(u):
    """posix timestamp -> plot time"""
    td = (datetime.utcfromtimestamp(u) - matplotlib_epoch)
    return td.days + 1 + (1000000 * td.seconds + td.microseconds) / 1e6 / DAY


def unixtm(p):
    """plot time -> posix timestamp"""
    td = timedelta(days=p-1)
    return (matplotlib_epoch + td - posix_epoch).total_seconds()


def main():
    f = datetime.utcfromtimestamp
    u = 1270000000.1234567890
    print(f(u))
    print(mpl_dt.epoch2num(u))
    print(plottm(u))
    print(f(mpl_dt.num2epoch(mpl_dt.epoch2num(u))))
    print(f(mpl_dt.num2epoch(plottm(u))))
    print(f(unixtm(mpl_dt.epoch2num(u))))
    print(f(unixtm(plottm(u))))

    assert abs(mpl_dt.epoch2num(u) - plottm(u)) < 1e-5

    p = 86401.234567890 / DAY
    print(f(mpl_dt.num2epoch(p)))
    print(f(unixtm(p)))
    assert abs(mpl_dt.num2epoch(p) - unixtm(p)) < 1e-5

main()

Output

2010-03-31 01:46:40.123457
733862.074076
733862.074076
2010-03-31 01:46:40.123453
2010-03-31 01:46:40.123453
2010-03-31 01:46:40.123453
2010-03-31 01:46:40.123453
0001-01-01 00:00:01.234566
0001-01-01 00:00:01.234566

Solution 3

Thanks to F.J.'s answer to a similar question, I believe the following may be the best way to deal with this:

import datetime, calendar
import matplotlib.dates as dt

def plottm(u): return dt.date2num(datetime.datetime.utcfromtimestamp(u))
def unixtm(p): return calendar.timegm(dt.num2date(p).timetuple())

Comments

[dreeves]

When I run this my output is 2010-03-31 01:46:40+00:00 --> 2010-03-31 06:46:40+00:00 (1270000000 --> 1270018000)

[Pedro Romano]

The problem is that my time zone is currently UTC, and I should have tested with different localtimes... Testing now.

[Pedro Romano]

Can you try the corrected example? It tested it with several local timezones so an it should be correct now.

[dreeves]

Thanks so much, Pedro! In the meantime I asked a more general version of this question which led to what I think is a better approach than messing with tzlocal. I just posted an answer that's now working for me.

[Pedro Romano]

Definitely better solution: +1 from me! :) I was sure there was a UTC equivalent for time.mktime but for the life of me, I couldn't remember where (and it was staring me in the face a few lines above in the time.gmtime documentation)! One should always work internally in UTC and to/from local time should only happen at user interface.

[dreeves]

Thanks J.F.! Do you see any reason to prefer this solution over the one I posted (based, confusingly, on F.J.'s answer)?

[jfs]

num2date depends on tz that can be configured to be any timezone, timegm expect utc time so num2date should always return time in utc. I don't know what timezone date2num expects (if it is not always utc; the code might fail).

[jfs]

@dreeves: yes. Your code might be incorrect. I've left appropriate comment on your answer. epoch2num/num2epoch should be used unless you have specific reasons not to use them. Also you could test times around datetime.min, datetime.max to see whether the precision is lost or does it work at all.