Error: C stack usage is too close to the limit
Solution 1
The stack size is an operating system parameter, adjustable per-process (see setrlimit(2)
). You can't adjust it from within R as far as I can tell, but you can adjust it from the shell before starting R, with the ulimit
command. It works like this:
$ ulimit -s # print default
8192
$ R --slave -e 'Cstack_info()["size"]'
size
8388608
8388608 = 1024 * 8192; R is printing the same value as ulimit -s
, but in bytes instead of kilobytes.
$ ulimit -s 16384 # enlarge stack limit to 16 megs
$ R --slave -e 'Cstack_info()["size"]'
size
16777216
To make a permanent adjustment to this setting, add the ulimit
command to your shell startup file, so it's executed every time you log in. I can't give more specific directions than that, because it depends on exactly which shell you have and stuff. I also don't know how to do it for logging into a graphical environment (which will be relevant if you're not running R inside a terminal window).
Solution 2
I suspect that, regardless of stack limit, you'll end up with recursions that are too deep. For instance, with lambda = Inf, f(1) leads to an immediate recursion, indefinitely. The depth of the recursion seems to be a random walk, with some probability r of going deeper, 1 - r of finishing the current recursion. By the time you've hit the stack limit, you've made a large number of steps 'deeper'. This implies that r > 1 / 2, and the very large majority of time you'll just continue to recurse.
Also, it seems like it is almost possible to derive an analytic or at least numerical solution even in the face of infinite recursion. One can define p as the probability that f(1) == 1, write implicit expressions for the 'child' states after a single iteration, and equate these with p, and solve. p can then be used as the chance of success in a single draw from a binomial distribution.
Solution 3
This error is not due to memory it is due to recursion. A function is calling itself. This isn't always obvious from examining the definition of only one function. To illustrate the point, here is a minimal example of 2 functions that call each other:
change_to_factor <- function(x){
x <- change_to_character(x)
as.factor(x)
}
change_to_character <- function(x){
x <- change_to_factor(x)
as.character(x)
}
change_to_character("1")
Error: C stack usage 7971600 is too close to the limit
The functions will continue to call each other recursively and will theoretically never complete, even if you increase the limit it will still be exceeded. It is only checks within your system that prevent this from occurring indefinitely and consuming all of the compute resources of your machine. You need to alter the functions to ensure that they won't indefinitely call itself (or each other) recursively.
Solution 4
This happened to me for a completely different reason. I accidentally created a superlong string while combining two columns:
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, col = "_"))
instead of
output_table_subset = mutate(big_data_frame,
combined_table = paste0(first_part, second_part, sep = "_"))
Took me for ever to figure it out as I never expected the paste to have caused the problem.
Solution 5
I encountered the same problem of receiving the "C stack usage is too close to the limit" error (albeit for another application than the one stated by user2045093 above). I tried zwol's proposal but it didn't work out.
To my own surprise, I could solve the problem by installing the newest version of R for OS X (currently: version 3.2.3) as well as the newest version of R Studio for OS X (currently: 0.99.840), since I am working with R Studio.
Hopefully, this may be of some help to you as well.
Related videos on Youtube
user2045093
Updated on February 14, 2022Comments
-
user2045093 about 2 years
I'm attempting to run some fairly deep recursive code in R and it keeps giving me this error:
Error: C stack usage is too close to the limit
My output from
CStack_info()
is:Cstack_info() size current direction eval_depth 67108864 8120 1 2
I have plenty of memory on my machine, I'm just trying to figure out how I can increase the CStack for R.
EDIT: Someone asked for a reproducible example. Here's some basic sample code that causes the problem. Running f(1,1) a few times you'll get the error. Note that I've already set --max-ppsize = 500000 and options(expressions=500000) so if you don't set those you might get an error about one of those two things instead. As you can see, the recursion can go pretty deep here and I've got no idea how to get it to work consistently. Thanks.
f <- function(root=1,lambda=1) { x <- c(0,1); prob <- c(1/(lambda+1),lambda/(lambda+1)); repeat { if(root == 0) { break; } else { child <- sample(x,2,replace=TRUE,prob); if(child[1] == 0 && child[2] == 0) { break; } if(child[1] == 1) { child[1] <- f(root=child[1],lambda); } if(child[2] == 1 && child[1] == 0) { child[2] <- f(root=child[2],lambda); } } if(child[1] == 0 && child[2] == 0) { break; } if(child[1] == 1 || child[2] == 1) { root <- sample(x,1,replace=TRUE,prob); } } return(root) }
-
mnel about 11 yearsThis question suggests perhaps
options(expressions = somethinglarge)
-
zwol about 11 years@mnel The expression nesting depth, the pointer protection stack, and the C stack are three separate (but related) things.
-
user2045093 about 11 yearsThanks so much for your prompt response, Zack. I think that your answer may be for a Linux OS though? I'm currently running Windows 7 64 bit, does that change things at all? Thanks again for any help.
-
zwol about 11 yearsMy answer should be valid for any Unix variant (of which Linux and OSX are the most common nowadays) but ... yeah, I have no idea what the Windows equivalent is.
-
Martin Morgan about 11 yearsGoogling the error message shows that in the past this has usually been an error in user code, so you should probably reduce your problem to a simple reproducible example and post that here.
-
Robert Casey about 9 yearsI'm not sure there is an error in the code at all. This is simply a case of probabilities that could in theory end up with infinite recursion. f(1,1) is basically flipping a coin. It could keep coming up heads forever. For a condition where the level of recursion is unknown and unbounded, you are better off coming up with something more iterative, using memoization of prior sample() results to inform future operations. Then the only thing you risk is running out of vector memory, or disk, depending on where you are storing your backlog of results. Recursion can be expensive and brittle.
-
Lazarus Thurston about 5 yearsThe same error comes in a simple
googlesheets::gs_ls()
. There is no recursion as I am running it on the command line of rstuido. I use a mac. How is this harmless googlesheetls
command giving an error?Error: C stack usage 7970624 is too close to the limit
seems strange. I could not search any answer till now. This is the closest thread to the problem but the marked answer does not give any fix. -
gaut almost 2 yearshad the same error with shiny, trying to
runApp
a file which also included arunApp
statement. removing this duplicated statement fixed the issue
-
-
Paul Hiemstra about 11 years...or just set it to
unlimited
. -
Kamil S Jaron almost 9 yearshere is actually hidden correct answer - make sure you don't get that deep in recusion...
-
krlmlr almost 8 yearsThe
RAppArmor
package offers an interface tosetrlimit(2)
. This functionality may become available in theulimit
package at some point. -
CoderGuy123 almost 7 yearsThis function no longer exists in the RAppArmor package. Any ideas where it went?
-
Richard DiSalvo over 6 yearsSame here, but I was doing a summarize. I had it like this:
summarize( states = paste0(state,collapse=', ') )
. When I should have done something like:summarize( states = paste0(sort(unique(state)),collapse=', ') )
. Goal was to get a comma separated list of unique states available for each subgroup. -
S.Perera about 6 yearsWhat is the fix for Windows?
-
zwol about 6 years@Shana I haven't the faintest idea. Ask a new question and specifically mention+tag Windows and hopefully someone who does know will answer.
-
murphy1310 almost 6 yearsI switched to a higher version of R. It worked once, but the error reappeared and is consistent now. Help!
-
Tom Kelly over 5 yearsChanging the limit will not resolve this. A recursive function will simply continue to run until the higher limit is reached.
-
Code42 about 5 yearsDo you have to set it every time you log in/every time the machine is rebooted?
-
zwol about 5 years@Dr_Hope Yes. You can put the
ulimit
command in your.profile
, or there may be a file in/etc
that can be edited to change the limit system-wide. -
stevec almost 5 yearsThis setting seems to have reset when I rebooted. Is there an option to permanently change it?
-
Good Will over 4 yearsIn my case, the error is caused by sourcing the same R script multiple times (i.e. in multiple R scripts) in my project.
-
cineS. over 2 yearsRefusing a recursion narrows field of solving problems by computer. I better advise to use so called terminators in each recursively called function. The role of a terminator is to conditionally stop further recursive calling, The best way is to count how deep in recursion you are and stop it as soon as you reach given limit (before system error occurs).