Date time conversion in R

Question

I know similar things have been discussed but here's a few things I haven't found answers yet.

1: Date to POSIXct

> Sys.timezone()
[1] "EST"

> as.POSIXct("2011-01-30")
[1] "2011-01-30 EST"

> as.POSIXct(as.Date("2011-01-30"))
[1] "2011-01-29 19:00:00 EST"

> as.POSIXct(as.Date("2011-01-30"), tz="EST")
[1] "2011-01-29 19:00:00 EST"

> as.POSIXct(as.Date("2011-01-30"), tz="GMT")
[1] "2011-01-29 19:00:00 EST"

> as.POSIXct(as.Date("2011-01-30"), tz="America/New_York")
[1] "2011-01-29 19:00:00 EST"

> as.POSIXct(as.Date("2011-01-30"), tz="")
[1] "2011-01-29 19:00:00 EST"

Can anybody tell me why, if the input is a Date, I cannot get midnight January 30 even though I specified the correct time zone? While if the input is character string then I'm OK.

2: Date/Character conversion.

Both are horribly slow. Then I found that to convert a character string to a Date it's actually a lot faster by converting it to POSIXlt first:

> d3.str = "2011-03-02 23:59:00";
> N=10000
> system.time(for(i in 1:N) r5.dt = as.Date(d3.str))
   user  system elapsed 
   1.25    0.00    1.24 
> system.time(for(i in 1:N) r6.dt = as.Date(strptime(d3.str, format="%Y-%m-%d")))
   user  system elapsed 
   0.37    0.00    0.38 
> r5.dt
[1] "2011-03-02"
> r5.dt==r6.dt
[1] TRUE

I'm now confused. Is there any faster, more elegant way to convert a character to a Date? And a Date to a character instead as.character which is also quite slow?

Thank you in advance!

Solution 1

As for your second question, a lot will probably be clearer if you compare the code for as.Date.character and as.Date.POSIXlt. One of them has to do a lot more checking of stuff since you haven't specified a format.

As for general speed, you aren't vectorizing your conversions. as.Date and strptime and most other Date/Time conversion functions accept vectors of dates and DateTimes. It will be considerably faster than way.

Solution 2

Very early in the as.POSIXct help page we read: "Dates without times are treated as being at midnight UTC." Your expectation that the 'tz' argument will modify this is simply mistaken. You can get the original date if you specify the output to be in UCT:

 strftime(as.POSIXct(as.Date("2011-01-30")), format="%Y-%m-%d %H:%M:%S", tz="UCT")
#[1] "2011-01-30 00:00:00"

If you really just want a date then you should use as.Date.

Solution 3

The tz argument of as.POSIXct is only used with strings, and (silently) ignored otherwise. You may want to look at packages designed to help manipulate dates, e.g., the with_tz function in the lubridate package.

Comments

[sunt]

Hey Gavin, thanks for editing! It looks much better now.

[IRTFM]

Leap-seconds are added at the turn of the year and DST changes have never occurred in January, so I think you should strike your last sentence.

[sunt]

I just figured out similar functions like as.character.Date, as.POSIXlt.POSIXct etc., and generally specifying the format speeds up things a lot. In my situation I have to use a loop. This is great stuff, thank everybody for your help!

[Dirk Eddelbuettel]

+1 -- and if you want midnight in your time zone, use ISODatetime(2012,1,11,0,0,0) (or any other date).

[IRTFM]

Thanks for that. It would be good to note that srtftime creates a 'character' result, while ISOdatetime (note that the "d" is not capped) returns class 'POSIXct', so it may be more desirable.