Best workflow using node.js npm and git

Question

I am about to do a large project with node.js and currently try sort a few things out.

In earlier node projects I had an extra folder for all node modules I used. This folder was ignored by git and I managed version and updates via git submodules, which was not easy (no dependencies, updating to new version was not always fun.)

What I am looking for is:

npm install packagename
npm dump_modules_into_file

So everyone else who is involved in this project could do:

npm install_or_update_modules_from_file

I don not want to have node_modules tracked by my git repository. Basically I want something similar to how symonfy2 handles it bundles.

P.S.: I know about npm submodule packagename, but this command is not very helpful because it does not install dependencies and it does not update the modules.

P.S.2: I ready about the package.json, but this also has some flaws. (No parameters and you have to update module versions by hand.)

Solution 1

package.json will accomplish what you're looking for. In your comment about passing the --mongodb:native flag, that flag is an argument to the npm command and does work when using a package.json in your own project. The mongodb package has an "install script" which looks for that flag in the node processing environment. If that flag is present, then it spawns another process for the build. So, if you have mongodb as a dependency in your package.json

{
    "name": "MyProject"
  , "description": "Test"
  , "version": "0.0.1"
  , "dependencies": {
        "mongodb": "*"
    }
}

Running npm install --mongodb:native will work.

With regards to "updating it by hand" - it's really only the first time that it might take a while, and I'm sure you could write a script to generate it if there are a lot of dependencies. However, it sounds like you have a fairly large team, and if that is the case, then automating the updates to package.json will turn really ugly (think new developers, experimental features, etc.) Having accountability for broken builds in this part of the development cycle isn't necessarily a bad idea.

References:

• https://github.com/christkv/node-mongodb-native/blob/master/install.js
• http://package.json.nodejitsu.com/ (see "scripts" section)

EDIT: and as Nick mentioned, adding the 'node_modules' directory to .gitignore will prevent any of those files from being checked into your repo

Solution 2

Afaik, the only ways to do package management are what you've described although I'm unsure as to what you're uninterested about wrt package.json.

If you want tight control over the versions of the modules your using you can explicitly specify the version number. You can also use the >=X.X.X approach as well to automatically grab the latest (above a threshold) which is sometimes fine for development purposes.

This allows your teammates to do:

npm install .

Which will install all the dependencies listed inside the package.json file. These will install to ./node_modules but you can .gitignore that as you noted.

Solution 3

In order to fully test the package configuration and ensure that modules behave as they will on the final deployment I now use the following procedure. It completely avoids the need to hack your node_module directories or require() code so when you go to deploy it just works.

For internal projects or pre-release to github you also might want to set "private": true in your package.json so npm will refuse to publish it.

1. Create a project directory under git version control and add all your node modules as subdirectories. Subdirectory names must match their package names. If you're working with github, you will want to create a separate git repo for each module directory. They can be git submodules in your project repo. Add node_module to your .gitignore files.

2. Install a tool like npm-server and run it in the project directory. Then set npm registry to localhost so now npm will talk to your local npm server to fetch packages. Any it finds as subdirectories it will send. Any it does not find it will proxy to registry.npmjs.org . $ npm set registry http://localhost:6070/ $ cd ~/projects $ npm-server

3. Start a new shell and create a separate sandbox directory $ mkdir sandbox $ cd sandbox

4. Install your app using your local registry server. Clear local npm cache and reinstall your app. I do this on one line so it's easy to redo via the shell. You might want to script it.

   $ npm cache clear; sleep 3; npm uninstall -g app; sleep 3; npm install -g app

5. Test your app:

   $ app ....

6. Deregister local npm registry when you're done installing:

   $ npm set registry http://registry.npmjs.org:80/

7. Once you've completed testing you can publish your app and retest the deployment with npm-server stopped.

$ cd ~/projects $ npm publish app

---

Rather than register & deregister the sever, you can just use the localhost server for a one off install: $ npm --registry=http://localhost:6070/ install app

---

I'm in the process of writing a forked version of npm-server so you just do: $ npmsvr on // Registers local registry server $ npmsvr start // Start local registry server $ npmsvr off // Deregisters local registry server

Comments

[TheHippo]

Thanks for the tip with parameters passed to npm in general! I will include the npm command in a bash script or a Makefile and we are good to go...

[Tony O'Hagan]

I've just published npmsvr to npm registry to make this task easier. npmjs.org/package/npmsvr

[Tony O'Hagan]

Note that this version is just a tiny front end to npm and npm-server so you still need to install npm-server