In R, how to make the variables inside a function available to the lower level function inside this function?(with, attach, environment)

Question

Update 2 @G. Grothendieck posted two approaches. The second one is changing the function environment inside a function. This solves my problem of too many coding replicates. I am not sure if this is a good method to pass through the CRAN check when making my scripts into a package. I will update again when I have some conclusions.

Update

I am trying to pass a lot of input argument variables to f2 and do not want to index every variable inside the function as env$c, env$d, env$calls, that is why I tried to use with in f5 and f6(a modified f2). However, assign does not work with with inside the {}, moving assign outside with will do the job but in my real case I have a few assigns inside the with expressions which I do not know how to move them out of the with function easily.

Here is an example:

## In the <environment: R_GlobalEnv>
a <- 1
b <- 2
f1 <- function(){
    c <- 3
d <- 4
f2 <- function(P){
    assign("calls", calls+1, inherits=TRUE)
    print(calls)
    return(P+c+d)
 }
calls <- 0
v <- vector()
for(i in 1:10){
    v[i] <- f2(P=0)
    c <- c+1
    d <- d+1
  }
 return(v)
}
f1()

Function f2 is inside f1, when f2 is called, it looks for variables calls,c,d in the environment environment(f1). This is what I wanted.

However, when I want to use f2 also in the other functions, I will define this function in the Global environment instead, call it f4.

f4 <- function(P){
  assign("calls", calls+1, inherits=TRUE)
  print(calls)
  return(P+c+d)
}

This won't work, because it will look for calls,c,d in the Global environment instead of inside a function where the function is called. For example:

f3 <- function(){
  c <- 3
  d <- 4
  calls <- 0
  v <- vector()
  for(i in 1:10){
    v[i] <- f4(P=0) ## or replace here with f5(P=0)
    c <- c+1
    d <- d+1
  }
  return(v)
}
f3()

The safe way should be define calls,c,d in the input arguments of f4 and then pass these parameters into f4. However, in my case, there are too many variables to be passed into this function f4 and it would be better that I can pass it as an environment and tell f4 do not look in the Global environment(environment(f4)), only look inside the environment when f3 is called.

The way I solve it now is to use the environment as a list and use the with function.

f5 <- function(P,liste){
  with(liste,{
     assign("calls", calls+1, inherits=TRUE)
     print(calls)
     return(P+c+d)
     }
  )
}
f3 <- function(){
  c <- 3
  d <- 4
  calls <- 0
  v <- vector()
  for(i in 1:10){
    v[i] <- f5(P=0,as.list(environment())) ## or replace here with f5(P=0)
    c <- c+1
    d <- d+1
  }
  return(v)
}
f3()

However, now assign("calls", calls+1, inherits=TRUE) does not work as it should be since assign does not modify the original object. The variable calls is connected to an optimization function where the objective function is f5. That is the reason I use assign instead of passing calls as an input arguments. Using attach is also not clear to me. Here is my way to correct the assign issue:

f7 <- function(P,calls,liste){
  ##calls <<- calls+1
  ##browser()
  assign("calls", calls+1, inherits=TRUE,envir = sys.frame(-1))
  print(calls)
  with(liste,{
    print(paste('with the listed envrionment, calls=',calls))
    return(P+c+d)
  }
  )
}
########
##################
f8 <- function(){
  c <- 3
  d <- 4
  calls <- 0
  v <- vector()
  for(i in 1:10){
    ##browser()
    ##v[i] <- f4(P=0) ## or replace here with f5(P=0)
    v[i] <- f7(P=0,calls,liste=as.list(environment()))
    c <- c+1
    d <- d+1
  }
  f7(P=0,calls,liste=as.list(environment()))
  print(paste('final call number',calls))
  return(v)
}
f8()

I am not sure how this should be done in R. Am I on the right direction, especially when passing through the CRAN check? Anyone has some hints on this?

Solution 1

(1) Pass caller's environment. You can explicitly pass the parent environment and index into it. Try this:

f2a <- function(P, env = parent.frame()) {
    env$calls <- env$calls + 1
    print(env$calls)
    return(P + env$c + env$d)
}

a <- 1
b <- 2
# same as f1 except f2 removed and call to f2 replaced with call to f2a
f1a <- function(){
    c <- 3
    d <- 4
    calls <- 0
    v <- vector()
    for(i in 1:10){
        v[i] <- f2a(P=0)
        c <- c+1
        d <- d+1
      }
     return(v)
}
f1a()

(2) Reset called function's environment We can reset the environment of f2b in f1b as shown here:

f2b <- function(P) {
    calls <<- calls + 1
    print(calls)
    return(P + c + d)
}

a <- 1
b <- 2
# same as f1 except f2 removed, call to f2 replaced with call to f2b
#  and line marked ## at the beginning is new
f1b <- function(){
    environment(f2b) <- environment() ##
    c <- 3
    d <- 4
    calls <- 0
    v <- vector()
    for(i in 1:10){
        v[i] <- f2b(P=0)
        c <- c+1
        d <- d+1
      }
     return(v)
}
f1b()

(3) Macro using eval.parent(substitute(...)) Yet another approach is to define a macro-like construct which effectively injects the body of f2c inline into f1c1. Here f2c is the same as f2b except for the calls <- calls + 1 line (no <<- needed) and the wrapping of the entire body in eval.parent(substitute({...})). f1c is the same as f1a except the call to f2a is replaced with a call to f2c .

f2c <- function(P) eval.parent(substitute({
    calls <- calls + 1
    print(calls)
    return(P + c + d)
}))

a <- 1
b <- 2
f1c <- function(){
    c <- 3
    d <- 4
    calls <- 0
    v <- vector()
    for(i in 1:10){
        v[i] <- f2c(P=0)
        c <- c+1
        d <- d+1
      }
     return(v)
}
f1c()

(4) defmacro This is almost the same as the the last solution except it uses defmacro in the gtools package to define the macro rather than doing it ourself. (Also see the Rcmdr package for another defmacro version.) Because of the way defmacro works we must also pass calls but since it's a macro and not a function this just tells it to substitute calls in and is not the same as passing calls to a function.

library(gtools)

f2d <- defmacro(P, calls, expr = {
    calls <- calls + 1
    print(calls)
    return(P + c + d)
})

a <- 1
b <- 2
f1d <- function(){
    c <- 3
    d <- 4
    calls <- 0
    v <- vector()
    for(i in 1:10){
        v[i] <- f2d(P=0, calls)
        c <- c+1
        d <- d+1
      }
     return(v)
}
f1d()

Solution 2

In general, I would say that any variable that is needed inside a function should be passed on through its arguments. In addition, if its value is needed later you pass it back from the function. Not doing this can quite quickly lead to strange results, e.g. what if there are multiple functions defining a variable x, which one should be used. If the amount of variables is larger, you create a custom data structure for it, e.g. putting them into a named list.

Solution 3

One could also use a function that redefines other functions in the specified environment.

test_var <- "global"

get_test_var <- function(){
  return(test_var)
}

some_function <- function(){
  test_var <- "local"
  return(get_test_var()) 

}

some_function() # Returns "global". Not what we want here...

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

some_function2 <- function(){
  test_var <- "local"
  # define function locally
  get_test_var2 <- function(){
    return(test_var)
  }
  return(get_test_var2()) 
}

some_function2() # Returns "local", but 'get_test_var2' can't be used in other places.

# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

add_function_to_envir <- function(my_function_name, to_envir) {
  script_text <- capture.output(eval(parse(text = my_function_name)))
  script_text[1] <- paste0(my_function_name, " <- ", script_text[1])
  eval(parse(text = script_text), envir = to_envir)
}

some_function3 <- function(){
  test_var <- "local"
  add_function_to_envir("get_test_var", environment()) 
  return(get_test_var()) 
}

some_function3() # Returns "local" and we can use 'get_test_var' from anywhere.

Here add_function_to_envir(my_function_name, to_envir) captures the script of the function, parses and reevaluates it in the new environment.

Note: the name of the function for my_function_name needs to be in quotes.

Comments

[Zhenglei]

I agree with you @Paul in general . I am trying to make my codes an R package but cannot pass through the CRAN check easily with many warnings like global variable bindings. There is many repeated code because the f2 is defined inside a function and I want to use it in another new function. I realize that copy paste it is not a good option and can cause problems in later steps. I also want to reduce the effort because I don't want to change the already existing scrips too much. That's why I try to pass the environment instead of define a new data structure.

[Zhenglei]

This gives the result I want but I did not write clearly my request. In fact, I want to avoid to index all the variables passing into f2, i.e, not writing env$, that is why I tried to use with. I want to be able to modify the outside calls inside f2 but not changing the other variables in the current environment. I will update the question.

[Zhenglei]

It looks like just what I need. I will test it with my real case and update later. Thank you very much for the help!