Prevent unlist to drop NULL values

list r null

26,913

Solution 1

The issue here is that you can't have NULL in the middle of a vector. For example:

> c(1,NULL,3)
[1] 1 3

You can have NA in the middle though. You could could convert it to character and then back to numeric, which automatically converts the NULL values to NA (with a warning):

> b <- as.numeric(as.character(a))
Warning message:
NAs introduced by coercion

then put the names back in, because they've been dropped by the previous operation:

> names(b) <- names(a)
> b
p1 p2 p1 p2 p1 p2 p1 p2 
2  5  3  4 NA NA  4  5 `

Solution 2

In this case (one level depth list) this should works too:

a[sapply(a, is.null)] <- NA
unlist(a)
# p1 p2 p1 p2 p1 p2 p1 p2 
#  2  5  3  4 NA NA  4  5

Solution 3

If you are dealing with a long complex JSON with several levels you should give this a try:

I extracted game log data from nba.com/stats web site. The problem is, some players have a NULL value for 3 point free throws (mostly centers) and jsonlite::fromJSON seems to handle NULL values very well:

#### Player game logs URL: one record per player per game played ####
gameLogsURL <- paste("http://stats.nba.com/stats/leaguegamelog?Counter=1000&Direction=DESC&LeagueID=00&PlayerOrTeam=P&Season=2016-17&SeasonType=Regular+Season&Sorter=PTS")

#### Import game logs data from JSON ####
# use jsonlite::fromJSON to handle NULL values
gameLogsData <- jsonlite::fromJSON(gameLogsURL, simplifyDataFrame = TRUE)
# Save into a data frame and add column names
gameLogs <- data.frame(gameLogsData$resultSets$rowSet)
colnames(gameLogs) <- gameLogsData$resultSets$headers[[1]]

26,913

Author by

nico

I am a lecturer at the University of Edinburgh. I am interested in understanding how cells decode and interpret signals from their environment to generate meaningful outputs. I mostly use neuroendocrine systems (the ones that produce hormones in your body) to study this question. I am also very interested in all that relates to data analysis.

Updated on July 05, 2022

Comments

nico almost 2 years
I have a vector of lists and I use unlist on them. Some of the elements in the vectors are NULL and unlist seems to be dropping them.

How can I prevent this?

Here's a simple (non) working example showing this unwanted feature of unlist
```
a = c(list("p1"=2, "p2"=5), 
      list("p1"=3, "p2"=4), 
      list("p1"=NULL, "p2"=NULL), 
      list("p1"=4, "p2"=5))
unlist(a)
 p1 p2 p1 p2 p1 p2 
 2  5  3  4  4  5 
```
nico almost 14 years

thanks for the answer. Obviously I do not define the list by hand, it is returned by a function. Anyway changing the NULLs to NA before unlist seemed to do the trick.
Marek almost 14 years

@nico If it's your function then you might consider rewriting it to return NA instead of NULL. take a look on help pages to NA and NULL to see differences between this two objects.
nico almost 14 years

@Marek: No, it actually is a list returned by applying coef on a list of objects returned by nls. Some of these objects are NULL and coef(NULL) returns NULL...
cylondude over 8 years

On 3.2.2, it looks like as.numeric(as.character(NULL)) returns numeric(0). A new approach might be to use lapply(b, function(x) ifelse(is.null(x), NA, x))
Ferroao about 6 years

a[sapply(a, function(x) length(x)==0L)] <- NA
nelliott over 5 years

For the approach suggested by @cylondude, substitute sapply for lapply (or use lapply with simplify = TRUE) to get a vector instead of a list.
torenunez over 4 years

Thanks @Ferroao, that worked for me, but is.null did not.