behavior of na.action = na.exclude using lmer in lme4 1.0-5 inconsistent with lm and with older versions of lme4

Question

Consider a simple example:

> library(lme4)
Loading required package: lattice
Loading required package: Matrix
> set.seed(1)
> df <- data.frame(x = c(rnorm(7), NA), y = rep(c("A", "B"), 4))
> length(fitted(lm(data = df, x ~ y, na.action = na.exclude)))
[1] 8

This behaves as I would expect. Although there is no fitted value for the 8th observation, because x is NA for that row, the fitted values are "padded" with NA so that they are the same length as the number of rows in the input data frame df, which is very handy. But calling na.action = na.exclude no longer has the same effect in lme4.

> length(fitted(lmer(data = df, x ~ (1 | y), na.action = na.exclude)))
[1] 7

I am fairly certain that, in older versions of lme4, the length would be 8, with the last value being NA, just as it is with lm().

How can I get lmer to behave in the same way as lm --- padding the fitted vector with NAs (in the appropriate locations) so that it is the same length as the number of rows in the input data frame?

More generally, is there something that I am missing about what na.action and na.exclude are doing and/or intending to do here?

My goal is to get fitted(), when applied to an lmer model object, to return a vector with the same length as the number of rows as the input data frame (and in the same order!) and with NAs in the "appropriate" locations. Is there a simple way to do this?

Thanks,

Dave Kane

UPDATE: Thanks to Ben Bolker for the work around! Although predict() is not the same as fitted(), using it does solve my main problem.

> sessionInfo()
R version 3.0.2 (2013-09-25)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] lme4_1.0-5      Matrix_1.1-0    lattice_0.20-23

loaded via a namespace (and not attached):
[1] grid_3.0.2    MASS_7.3-29   minqa_1.2.1   nlme_3.1-111  splines_3.0.2 tools_3.0.2  

Solution

As a quick workaround, you can use predict (with no additional arguments) rather than fitted:

length(predict(lmer(data = df, x ~ (1 | y), na.action = na.exclude)))
## 8

(this is with 1.1-1 rather than 1.0-5 but I think the results will be the same).

Comments

[Ben Bolker]

I will look into this. It might be more suited as an lme4 "issue" posted at github.com/lme4/lme4/issues (feel free to post there as well). Possibly too narrow for SO.

[Ben Bolker]

this should be fixed in 1.1-1, may take a little longer to propagate to the "release" branch (1.0-6)