Cleaning up factor levels (collapsing multiple levels/labels)

Question

What is the most effective (ie efficient / appropriate) way to clean up a factor containing multiple levels that need to be collapsed? That is, how to combine two or more factor levels into one.

Here's an example where the two levels "Yes" and "Y" should be collapsed to "Yes", and "No" and "N" collapsed to "No":

## Given: 
x <- c("Y", "Y", "Yes", "N", "No", "H")   # The 'H' should be treated as NA

## expectedOutput
[1] Yes  Yes  Yes  No   No   <NA>
Levels: Yes No  # <~~ NOTICE ONLY **TWO** LEVELS

---

One option is of course to clean the strings before hand using sub and friends.

Another method, is to allow duplicate label, then drop them

## Duplicate levels ==> "Warning: deprecated"
x.f <- factor(x, levels=c("Y", "Yes", "No", "N"), labels=c("Yes", "Yes", "No", "No"))

## the above line can be wrapped in either of the next two lines
factor(x.f)      
droplevels(x.f) 

However, is there a more effective way?

---

While I know that the levels and labels arguments should be vectors, I experimented with lists and named lists and named vectors to see what happens Needless to say, none of the following got me any closer to my goal.

  factor(x, levels=list(c("Yes", "Y"), c("No", "N")), labels=c("Yes", "No"))
  factor(x, levels=c("Yes", "No"), labels=list(c("Yes", "Y"), c("No", "N")))

  factor(x, levels=c("Y", "Yes", "No", "N"), labels=c(Y="Yes", Yes="Yes", No="No", N="No"))
  factor(x, levels=c("Y", "Yes", "No", "N"), labels=c(Yes="Y", Yes="Yes", No="No", No="N"))
  factor(x, levels=c("Yes", "No"), labels=c(Y="Yes", Yes="Yes", No="No", N="No"))

Solution 1

UPDATE 2: See Uwe's answer which shows the new "tidyverse" way of doing this, which is quickly becoming the standard.

UPDATE 1: Duplicated labels (but not levels!) are now indeed allowed (per my comment above); see Tim's answer.

ORIGINAL ANSWER, BUT STILL USEFUL AND OF INTEREST: There is a little known option to pass a named list to the levels function, for exactly this purpose. The names of the list should be the desired names of the levels and the elements should be the current names that should be renamed. Some (including the OP, see Ricardo's comment to Tim's answer) prefer this for ease of reading.

x <- c("Y", "Y", "Yes", "N", "No", "H", NA)
x <- factor(x)
levels(x) <- list("Yes"=c("Y", "Yes"), "No"=c("N", "No"))
x
## [1] Yes  Yes  Yes  No   No   <NA>  <NA>
## Levels: Yes No

As mentioned in the levels documentation; also see the examples there.

value: For the 'factor' method, a vector of character strings with length at least the number of levels of 'x', or a named list specifying how to rename the levels.

This can also be done in one line, as Marek does here: https://stackoverflow.com/a/10432263/210673; the levels<- sorcery is explained here https://stackoverflow.com/a/10491881/210673.

> `levels<-`(factor(x), list(Yes=c("Y", "Yes"), No=c("N", "No")))
[1] Yes  Yes  Yes  No   No   <NA>
Levels: Yes No

Solution 2

As the question is titled Cleaning up factor levels (collapsing multiple levels/labels), the forcats package should be mentioned here as well, for the sake of completeness. forcats appeared on CRAN in August 2016.

There are several convenience functions available for cleaning up factor levels:

x <- c("Y", "Y", "Yes", "N", "No", "H") 

library(forcats)

Collapse factor levels into manually defined groups

fct_collapse(x, Yes = c("Y", "Yes"), No = c("N", "No"), NULL = "H")
#[1] Yes  Yes  Yes  No   No   <NA>
#Levels: No Yes

Change factor levels by hand

fct_recode(x, Yes = "Y", Yes = "Yes", No = "N", No = "No", NULL = "H")
#[1] Yes  Yes  Yes  No   No   <NA>
#Levels: No Yes

Automatically relabel factor levels, collapse as necessary

fun <- function(z) {
  z[z == "Y"] <- "Yes"
  z[z == "N"] <- "No"
  z[!(z %in% c("Yes", "No"))] <- NA
  z
}
fct_relabel(factor(x), fun)
#[1] Yes  Yes  Yes  No   No   <NA>
#Levels: No Yes

Note that fct_relabel() works with factor levels, so it expects a factor as first argument. The two other functions, fct_collapse() and fct_recode(), accept also a character vector which is an undocumented feature.

Reorder factor levels by first appearance

The expected output given by the OP is

[1] Yes  Yes  Yes  No   No   <NA>
Levels: Yes No

Here the levels are ordered as they appear in x which is different from the default (?factor: The levels of a factor are by default sorted).

To be in line with the expected output, this can be achieved by using fct_inorder() before collapsing the levels:

fct_collapse(fct_inorder(x), Yes = c("Y", "Yes"), No = c("N", "No"), NULL = "H")
fct_recode(fct_inorder(x), Yes = "Y", Yes = "Yes", No = "N", No = "No", NULL = "H")

Both return the expected output with levels in the same order, now.

Solution 3

Perhaps a named vector as a key might be of use:

> factor(unname(c(Y = "Yes", Yes = "Yes", N = "No", No = "No", H = NA)[x]))
[1] Yes  Yes  Yes  No   No   <NA>
Levels: No Yes

This looks very similar to your last attempt... but this one works :-)

Solution 4

Since R 3.5.0 (2018-04-23) you can do this in one clear and simple line:

x = c("Y", "Y", "Yes", "N", "No", "H") # The 'H' should be treated as NA

tmp = factor(x, levels= c("Y", "Yes", "N", "No"), labels= c("Yes", "Yes", "No", "No"))
tmp
# [1] Yes  Yes  Yes  No   No   <NA>
# Levels: Yes No

1 line, maps multiple values to the same level, sets NA for missing levels" – h/t @Aaron

Solution 5

Another way is to make a table containing the mapping:

# stacking the list from Aaron's answer
fmap = stack(list(Yes = c("Y", "Yes"), No = c("N", "No")))

fmap$ind[ match(x, fmap$values) ]
# [1] Yes  Yes  Yes  No   No   <NA>
# Levels: No Yes

# or...

library(data.table)
setDT(fmap)[x, on=.(values), ind ]
# [1] Yes  Yes  Yes  No   No   <NA>
# Levels: No Yes

I prefer this way, since it leaves behind an easily inspected object summarizing the map; and the data.table code looks just like any other join in that syntax.

---

Of course, if you don't want an object like fmap summarizing the change, it can be a "one-liner":

library(data.table)
setDT(stack(list(Yes = c("Y", "Yes"), No = c("N", "No"))))[x, on=.(values), ind ]
# [1] Yes  Yes  Yes  No   No   <NA>
# Levels: No Yes

View more solutions

Comments

[Simon O'Hanlon]

+1 more robust and I would imagine a lot safer than my attempt.

[Ricardo Saporta]

Thanks Aaron, I like this approach in that it at least avoids the warnings associated with droplevles(factor(x, ...)) but I remain curious as to any more direct methods. eg: If it were possible to use levels=<a named list> right in the factor(.) call)

[Aaron left Stack Overflow]

Agree that it's odd this can't be done within factor; I don't know of a more direct way, except for using something like Ananda's solution or perhaps something with match.

[Ricardo Saporta]

Thanks Ananda. This a great idea. and for my applications, I can probably do away with unname ... this just might take the cake

[asnr]

This also works for ordered and the collapsed levels are ordered as they are supplied, for example a = ordered(c(1, 2, 3)); levels(a) = list("3" = 3, "1,2" = c(1, 2)) yields the ordering Levels: 3 < 1,2.

[Frank]

Another example: franknarf1.github.io/r-tutorial/_book/tables.html#dt-recode

[Frank]

Revisiting years later... this will drop levels that do not show up, which might not be desirable, e.g., with x="N" only the "No" level will show up in the result.

[A5C1D2H2I1M1N2O1R2T1]

@Frank, isn't this easily resolved by adding explicit levels to the factor step?

[A5C1D2H2I1M1N2O1R2T1]

@Frank, this question also led to Factor from "SOfun" :-)

[Frank]

Ah cool stuff :) Yeah, adding explicit levels works, though you'd have to type the list a second time, save the list somewhere or do some pipery or functioning like c(Y = "Yes", Yes = "Yes", N = "No", No = "No", H = NA) %>% { factor(unname(.[x]), levels = unique(.)) } eh.

[Uwe]

@frank Even more cool stuff with the added benefit that it orders the levels as in the expected out: Yes, No.

[Ricardo Saporta]

helpful update, but the named list is friendlier to anyone who needs to read the code