Should I check in folder "node_modules" to Git when creating a Node.js app on Heroku?

Question

I followed the basic getting started instructions for Node.js on Heroku here:

https://devcenter.heroku.com/categories/nodejs

These instruction don't tell you to create a .gitignore node_modules, and therefore imply that folder node_modules should be checked in to Git. When I included node_modules in Git repository, my getting started application ran correctly.

When I followed the more advanced example at:

• Building a Real-time, Polyglot Application with Node.js, Ruby, MongoDB and Socket.IO
• https://github.com/mongolab/tractorpush-server (source)

It instructed me to add folder node_modules to file .gitignore. So I removed folder node_modules from Git, added it to file .gitignore, and then redeployed. This time the deployed failed like so:

-----> Heroku receiving push
-----> Node.js app detected
-----> Resolving engine versions
       Using Node.js version: 0.8.2
       Using npm version: 1.0.106
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
       Error: npm doesn't work with node v0.8.2
       Required: [email protected] || 0.5 || 0.6
           at /tmp/node-npm-5iGk/bin/npm-cli.js:57:23
           at Object.<anonymous> (/tmp/node-npm-5iGk/bin/npm-cli.js:77:3)
           at Module._compile (module.js:449:26)
           at Object.Module._extensions..js (module.js:467:10)
           at Module.load (module.js:356:32)
           at Function.Module._load (module.js:312:12)
           at Module.require (module.js:362:17)
           at require (module.js:378:17)
           at Object.<anonymous> (/tmp/node-npm-5iGk/cli.js:2:1)
           at Module._compile (module.js:449:26)
       Error: npm doesn't work with node v0.8.2
       Required: [email protected] || 0.5 || 0.6
           at /tmp/node-npm-5iGk/bin/npm-cli.js:57:23
           at Object.<anonymous> (/tmp/node-npm-5iGk/bin/npm-cli.js:77:3)
           at Module._compile (module.js:449:26)
           at Object.Module._extensions..js (module.js:467:10)
           at Module.load (module.js:356:32)
           at Function.Module._load (module.js:312:12)
           at Module.require (module.js:362:17)
           at require (module.js:378:17)
           at Object.<anonymous> (/tmp/node-npm-5iGk/cli.js:2:1)
           at Module._compile (module.js:449:26)
       Dependencies installed
-----> Discovering process types
       Procfile declares types -> mongod, redis, web
-----> Compiled slug size is 5.0MB
-----> Launching... done, v9

Running "heroku ps" confirms the crash. OK, no problem, so I rolled back the change, added folder node_module back to the Git repository and removed it from file .gitignore. However, even after reverting, I still get the same error message on deploy, but now the application is running correctly again. Running "heroku ps" tells me the application is running.

What's the right way to do this? Include folder node_modules or not? And why would I still be getting the error message when I rollback? My guess is the Git repository is in a bad state on the Heroku side.

Solution 1

Second Update

The FAQ is not available anymore.

From the documentation of shrinkwrap:

If you wish to lock down the specific bytes included in a package, for example to have 100% confidence in being able to reproduce a deployment or build, then you ought to check your dependencies into source control, or pursue some other mechanism that can verify contents rather than versions.

Shannon and Steven mentioned this before but I think it should be part of the accepted answer.

---

Update

The source listed for the below recommendation has been updated. They are no longer recommending the node_modules folder be committed.

Usually, no. Allow npm to resolve dependencies for your packages.

For packages you deploy, such as websites and apps, you should use npm shrinkwrap to lock down your full dependency tree:

https://docs.npmjs.com/cli/shrinkwrap

---

Original Post

For reference, npm FAQ answers your question clearly:

Check node_modules into git for things you deploy, such as websites and apps. Do not check node_modules into git for libraries and modules intended to be reused. Use npm to manage dependencies in your dev environment, but not in your deployment scripts.

and for some good rationale for this, read Mikeal Rogers' post on this.

---

Source: https://docs.npmjs.com/misc/faq#should-i-check-my-node-modules-folder-into-git

Solution 2

My biggest concern with not checking folder node_modules into Git is that 10 years down the road, when your production application is still in use, npm may not be around. Or npm might become corrupted; or the maintainers might decide to remove the library that you rely on from their repository; or the version you use might be trimmed out.

This can be mitigated with repository managers like Maven, because you can always use your own local Nexus (Sonatype) or Artifactory to maintain a mirror with the packages that you use. As far as I understand, such a system doesn't exist for npm. The same goes for client-side library managers like Bower and Jam.js.

If you've committed the files to your own Git repository, then you can update them when you like, and you have the comfort of repeatable builds and the knowledge that your application won't break because of some third-party action.

Solution 3

You should not include folder node_modules in your .gitignore file (or rather you should include folder node_modules in your source deployed to Heroku).

If folder node_modules:

• exists then npm install will use those vendored libraries and will rebuild any binary dependencies with npm rebuild.
• doesn't exist then npm install will have to fetch all dependencies itself which adds time to the slug compile step.

See the Node.js buildpack source for these exact steps.

However, the original error looks to be an incompatibility between the versions of npm and Node.js. It is a good idea to always explicitly set the engines section of your packages.json file according to this guide to avoid these types of situations:

{
  "name": "myapp",
  "version": "0.0.1",
  "engines": {
    "node": "0.8.x",
    "npm":  "1.1.x"
  }
}

This will ensure development/production parity and reduce the likelihood of such situations in the future.

Solution 4

I was going to leave this after this comment: Should I check in folder "node_modules" to Git when creating a Node.js app on Heroku?

But Stack Overflow was formatting it weirdly.

If you don't have identical machines and are checking in node_modules, do a .gitignore on the native extensions. Our .gitignore looks like:

# Ignore native extensions in the node_modules folder (things changed by npm rebuild)
node_modules/**/*.node
node_modules/**/*.o
node_modules/**/*.a
node_modules/**/*.mk
node_modules/**/*.gypi
node_modules/**/*.target
node_modules/**/.deps/
node_modules/**/build/Makefile
node_modules/**/**/build/Makefile

Test this by first checking everything in, and then have another developer do the following:

rm -rf node_modules
git checkout -- node_modules
npm rebuild
git status

Ensure that no files changed.

Solution 5

I believe that npm install should not run in a production environment. There are several things that can go wrong - npm outage, download of newer dependencies (shrinkwrap seems to have solved this) are two of them.

On the other hand, folder node_modules should not be committed to Git. Apart from their big size, commits including them can become distracting.

The best solutions would be this: npm install should run in a CI environment that is similar to the production environment. All tests will run and a zipped release file will be created that will include all dependencies.

View more solutions