nodejs out of memory

Question

I came across a curious issue today. This may be an easy answer for others, but it has me stumped. Why does the code below cause a memory error?

var cur = 167772160;
var bcast = 184549375;
var addresses = [];
while (cur <= bcast){
  cur += 1;
  addresses.push(cur);
}
addresses.length 
addresses // memory goes from a few megs to over a gig in seconds when trying to print this

I get one of these two errors...the first when i run this code in node's interpreter and the latter when i run it through nodeunit:

FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory

FATAL ERROR: JS Allocation failed - process out of memory

Solution 1

It happens when I try to access the array. But getting the length does not.

> var cur = 167772160;
> var bcast = 184549375;
> var addresses = [];
> while (cur <= bcast){
...   cur += 1;
...   addresses.push(cur);
... }
16777216
> addresses.length 
16777216
> addresses
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory

Here's another SO question, memory limit in Node.js (and chrome V8) that relates to issue with memory usage.

Solution 2

You can increase the default limits by passing --max-old-space-size=<value> which is in MB.

The example will allow node's heap use up to 4GB (4096 megabytes) of memory:

node --max-old-space-size=4096 app

Solution 3

I don't get a memory allocation error when I run your script. How much RAM is on your system?

Edit Ok with the author's updated notes, I can replicate it.

Node is trying to convert your entire array to a string. The array is 16777216 elements long. Each element contains a number at least 9 digits long. Converting that to a string 150,994,994 characters long. Its just a huge operation that is exceeding the memory capabilities of node.