R count NA by group

Question

Could someone please explain why I get different answers using the aggregate function to count missing values by group? Also, is there a better way to count missing values by group using a native R function?

DF <- data.frame(YEAR=c(2000,2000,2000,2001,2001,2001,2001,2002,2002,2002), X=c(1,NA,3,NA,NA,NA,7,8,9,10))
DF

aggregate(X ~ YEAR, data=DF, function(x) { sum(is.na(x)) })
with(DF, aggregate(X, list(YEAR), function(x) { sum(is.na(x)) }))

aggregate(X ~ YEAR, data=DF, function(x) { sum(! is.na(x)) })
with(DF, aggregate(X, list(YEAR), function(x) { sum(! is.na(x)) }))

Solution

The help page at ?aggregate points out that the formula method has an argument na.action which is set by default to na.omit.

na.action: a function which indicates what should happen when the data contain NA values. The default is to ignore missing values in the given variables.

Change that argument to NULL or na.pass instead to get the results you are probably expecting:

# aggregate(X ~ YEAR, data=DF, function(x) {sum(is.na(x))}, na.action = na.pass)
aggregate(X ~ YEAR, data=DF, function(x) {sum(is.na(x))}, na.action = NULL)
#   YEAR X
# 1 2000 1
# 2 2001 3
# 3 2002 0

Comments

[Jenks]

This doesn't answer how to count NA's during an aggregate or group by