R cor.test : "not enough finite observations"

Question

I'm currently trying to create an R function computing the corr.test correlation of a specified column with all the numeric columns of a dataframe. Here's my code :

#function returning only numeric columns
only_num <- function(dataframe)
{
  nums <- sapply(dataframe, is.numeric)
  dataframe[ , nums]
}

#function returning a one-variable function computing the cor.test correlation of the variable
#with the specified column

function_generator <- function(column)
  {
    function(x)
    {
      cor.test(x, column, na.action = na.omit)
    } 
  }

data_analysis <- function(dataframe, column)
  {
  DF <- only_num(dataframe)

  fonction_corr <- function_generator(column)

  sapply(DF, fonction_corr)

  }

data_analysis(40, 6, m, DF$Morphine)

When I call "data_analysis" at the last line, I get the following error :

"Error in cor.test.default(x, column, na.action=na.omit) : not enough finite observations"

What could it mean? What should I change? I'm kind of stuck...

Thanks.

Clément

Solution

"Not enough finite obervations" is an error returned by cor.test under certain circumstances. If you take a look a the cor.test.default source code, you'll see :

OK <- complete.cases(x, y)
x <- x[OK]
y <- y[OK]
n <- length(x)

cor.test removes NA values from you vectors [...]

if (method = "pearson") {
    if (n < 3L) 
        stop("not enough finite obervations")

[...]

else {
    if (n<2)
        stop("not enough finite obervations")

If your vectors do not contain enough non-NA values (less than 3), the function will return the error.

Make all of the columns in your dataframe contain enough non-NA values before you use cor.test.

I hope this will be useful.

Comments

[Clément F]

I've realised it works on a "classic" data set. However, my personnal data frame contains A LOT of NA values... And that's the reason why cor.test won't work : cor.test returns an error and stops if they are too numerous.