Node.js heap out of memory

Question

Today I ran my script for filesystem indexing to refresh RAID files index and after 4h it crashed with following error:

[md5:]  241613/241627 97.5%  
[md5:]  241614/241627 97.5%  
[md5:]  241625/241627 98.1%
Creating missing list... (79570 files missing)
Creating new files list... (241627 new files)

<--- Last few GCs --->

11629672 ms: Mark-sweep 1174.6 (1426.5) -> 1172.4 (1418.3) MB, 659.9 / 0 ms [allocation failure] [GC in old space requested].
11630371 ms: Mark-sweep 1172.4 (1418.3) -> 1172.4 (1411.3) MB, 698.9 / 0 ms [allocation failure] [GC in old space requested].
11631105 ms: Mark-sweep 1172.4 (1411.3) -> 1172.4 (1389.3) MB, 733.5 / 0 ms [last resort gc].
11631778 ms: Mark-sweep 1172.4 (1389.3) -> 1172.4 (1368.3) MB, 673.6 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3d1d329c9e59 <JS Object>
1: SparseJoinWithSeparatorJS(aka SparseJoinWithSeparatorJS) [native array.js:~84] [pc=0x3629ef689ad0] (this=0x3d1d32904189 <undefined>,w=0x2b690ce91071 <JS Array[241627]>,L=241627,M=0x3d1d329b4a11 <JS Function ConvertToString (SharedFunctionInfo 0x3d1d3294ef79)>,N=0x7c953bf4d49 <String[4]\: ,\n  >)
2: Join(aka Join) [native array.js:143] [pc=0x3629ef616696] (this=0x3d1d32904189 <undefin...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/bin/node]
 2: 0xe2c5fc [/usr/bin/node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/bin/node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/bin/node]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/bin/node]
 6: v8::internal::Runtime_SparseJoinWithSeparator(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
 7: 0x3629ef50961b

Server is equipped with 16gb RAM and 24gb SSD swap. I highly doubt my script exceeded 36gb of memory. At least it shouldn't

Script creates index of files stored as Array of Objects with files metadata (modification dates, permissions, etc, no big data)

Here's full script code: http://pastebin.com/mjaD76c3

I've already experiend weird node issues in the past with this script what forced me eg. split index into multiple files as node was glitching when working on such big files as String. Is there any way to improve nodejs memory management with huge datasets?

Solution 1

If I remember correctly, there is a strict standard limit for the memory usage in V8 of around 1.7 GB, if you do not increase it manually.

In one of our products we followed this solution in our deploy script:

 node --max-old-space-size=4096 yourFile.js

There would also be a new space command but as I read here: a-tour-of-v8-garbage-collection the new space only collects the newly created short-term data and the old space contains all referenced data structures which should be in your case the best option.

Solution 2

If you want to increase the memory usage of the node globally - not only single script, you can export environment variable, like this:
export NODE_OPTIONS=--max_old_space_size=4096

Then you do not need to play with files when running builds like npm run build.

Solution 3

Just in case anyone runs into this in an environment where they cannot set node properties directly (in my case a build tool):

NODE_OPTIONS="--max-old-space-size=4096" node ...

You can set the node options using an environment variable if you cannot pass them on the command line.

Solution 4

Here are some flag values to add some additional info on how to allow more memory when you start up your node server.

1GB - 8GB

#increase to 1gb
node --max-old-space-size=1024 index.js

#increase to 2gb
node --max-old-space-size=2048 index.js 

#increase to 3gb
node --max-old-space-size=3072 index.js

#increase to 4gb
node --max-old-space-size=4096 index.js

#increase to 5gb
node --max-old-space-size=5120 index.js

#increase to 6gb
node --max-old-space-size=6144 index.js

#increase to 7gb
node --max-old-space-size=7168 index.js

#increase to 8gb 
node --max-old-space-size=8192 index.js 

Solution 5

I just faced same problem with my EC2 instance t2.micro which has 1 GB memory.

I resolved the problem by creating swap file using this url and set following environment variable.

export NODE_OPTIONS=--max_old_space_size=4096

Finally the problem has gone.

I hope that would be helpful for future.

View more solutions

Comments

[Felix]

In the same way this config is for nodejs independently on the framework.@Simer

[Vikram]

I am developing with angular 4 and getting same issue, what should be yourFile.js file for angular app ?

[Felix]

@VikramSingh are you using ng serve or do you distribute the result of ng build the /dist folder by another webserver like express? But if your Angular project is using more than the standard 1.7GB Memory than you'll maybe have an architectural problem in your application? It looks like that you are using the development env with nmp start maybe this is a solution for it github.com/mgechev/angular-seed/issues/2063

[Vikram]

I am using ng build with angular cli and aot ( dist folder )

[ZachB]

"best to specify both syntaxes --max-old-space-size and --max_old_space_size" -- you do not need to do this, they are synonyms. From nodejs.org/api/cli.html#cli_options: "All options, including V8 options, allow words to be separated by both dashes (-) or underscores (_)."

[crashbus]

max-executable-size was removed and ends in an error when it is used: github.com/nodejs/node/issues/13341

[Techdive]

which file put put in yourFile.js ?? Nodemon file or NodeJS file ?

[Leo]

Cheers worked for me, it may be necessary to sudo npm -g install increase-memory-limit --unsafe-perm

[Keselme]

Can you please explain what you mean, when you say " ...set the node options using an environment variable.. "?

[Rob Evans]

@Keselme An environment variable is a variable that has been set on the server that all processes can read the data from. Open an SSH terminal to your server and type: MY_VAR=hello then type: echo $MY_VAR. You will see that it prints "hello" in the terminal. You've just set an environment variable and read it back.

[Basit]

index,js file @Techdive which you use to start server

[Harry Moreno]

can one keep increasing it in powers of 2? should one set it larger than system memory? if not what's a good system memory to max-old-space-size ratio?

[Nicholas Porter]

@HarryMoreno You can actually put in any number value you like. Doesn't have to be in power of 2. Not sure about the ratio though. It's only a max limit, it wont be using all the memory. I would just set it as high as you need then scale back if needed.

[Harry Moreno]

I'll give system ram - 1gb a try. Assuming this vm is only for running this node app.

[Rahal Kanishka]

worked on Ubuntu 18.04, just added the export command to my bashrc file

[Gigimoi]

@HarryMoreno A good system memory to max-old-space-size ratio depends entirely on what else is running on your machine. You can increase it in powers of two - or you can use any number. You can set it larger than system memory - but you will hit swap issues.

[Sergey Pleshakov]

I had to cut it to --max-old-space-size=8192 --optimize-for-size --max_old_space_size=8192 --optimize_for_size and it worked

[Muhammad Rosyid]

node --max-old-space-size=1024 index.js #increase to 1gb node --max-old-space-size=2048 index.js #increase to 2gb node --max-old-space-size=3072 index.js #increase to 3gb node --max-old-space-size=4096 index.js #increase to 4gb node --max-old-space-size=5120 index.js #increase to 5gb node --max-old-space-size=6144 index.js #increase to 6gb node --max-old-space-size=7168 index.js #increase to 7gb node --max-old-space-size=8192 index.js #increase to 8gb

[kant312]

Hello Angela and welcome to SO! Could you maybe specify the exact version of Node.js you updated to for future readers? Thanks!

[Admin]

I upgraded to Latest LTS Version: 12.18.0

[Tropicalrambler]

As an added convenience, add to bash_profile or zsh profile.

[Henrique Van Klaveren]

Thanks to share. I had the same problem, and your tip works fine for me.

[Bassam Helal]

Didn't think this would work but it actually did! So for anyone reading this and their Node version is old, try upgrading to the latest version ( I went from a 10 version to 14.8), likely it will fix this issue for you. Many thanks

[So Js]

Would launch.json be the same a package.json?

[Charles Stover]

No, launch.json is a configuration file specifically for running an application from VS Code.

[Anubisoft]

Thanks. I'm now able to "yarn build" Strapi on my $5/mo Linode nanode instance after I created a 2GB swap file and added an "ENV NODE_OPTIONS=--max_old_space_size=1024" to my Dockerfile. Not sure the swap step was needed in my case but it can't hurt.

[withoutOne]

4k is not enough. developer was keep on 4k as static. good solution from developer. Also when I explore the npm page, I couldnt saw the info about change the limit value. Actually, There is a solution but didnt worked.

[Ciaran Gallagher]

Where is this the launch.json file for VS Code?

[Constantinos]

Thanks, you're an absolute life-saver. optimize-for-size has finally allowed my builds to succeed.

[Andre Mesquita]

Great! Works for me at Azure Standard_B1s

[Mekel Ilyasa]

If the problem still persists the only thing to do is buy more RAM, large scale project usually consume a lot of heap memory

[Ayesh Weerasinghe]

I'm getting the same error even after setting NODE_OPTIONS to 4096 or more. When I run the command npm run build, I see some processes running with commands like usr/bin/node --max_old_space_size=2048. What could be the reason?

[Maksim Luzik]

@AyeshWeerasinghe probably libraries or scripts that you are using or running have a hardcoded max_old_space_size parameter which overrides the exported env variable.

[dawsnap]

Now in 2021 there must be better alternatives as the lib is marked as deprecated: npmjs.com/package/increase-memory-limit

[Ossip]

exactly, with small machines you may need to make it smaller rather than bigger, e.g. export NODE_OPTIONS=--max_old_space_size=1024

[Vikas Acharya]

After running this i type npm run dev and it stopped there. it is not showing any progress not giving any error after this line > webpack-dev-server --config ./webpack.dev.config.js. It was playing statue so, project cannot be run.

[Michal Štefanec]

this saved me from getting 'Segmentation fault'. Thank you so much!

[Venryx]

The strace tip works, though note that if you're using Docker with the Node Alpine base image, you will have to install strace yourself.

[Venryx]

Also note that strace produces a ton of output lines, which will obstruct your ability to see the regular log lines (unless you have some sort of filtering of the output). Any way to cut down on this noise to only show events relevant to std::bad_alloc errors?

[stuisme]

This helped fix an issue in github actions with the build running out of memory. "build": "react-scripts --expose-gc --max_old_space_size=4096 build",