How do I manage relative path aliasing in multiple grunt-browserify bundles?

Question

This is tad long but I'll need the code example to illustrate my confusion. After which I am interested to the answer for the following:

1. How do I use require('module') instead of require('../../src/module') or require('./module')?
2. How do I reuse ./index.js in spec/specs.js without duplicating work? (And preventing src/app.js from running as it's an entry module).

I've started several browser based projects already and love browserify and grunt. But each project dies at the same point in my development/learning curve. Once I add testing to the mix and have to manage two browserify bundles (app.js and spec/specs.js) the whole system falls apart. I'll explain:

I use grunt-browserify and set my initial directory:

.
├── Gruntfile.js
├── index.js  (generated via grunt-browserify)      [1]
├── lib
│   ├── jquery
│   │   └── jquery.js                               [2]
│   └── jquery-ui
│       └── jquery-ui.js                            [3]
├── spec
│   ├── specs.js  (generated via grunt-browserify)  [4]
│   └── src
│       ├── spec_helper.js  (entry)
│       └── module_spec.js  (entry)
└── src
    ├── app.js  (entry)
    └── module.js

1. Uses one entry file (src/app.js) and does a code walk to bundle all required modules.
2. Uses browserify-shim to alias jquery.
3. Is just aliased to jquery-ui without a shim (required after you var $ = require('jquery')).
4. Uses all helper and spec files in spec/src as entry modules.

I'll step through my config:

browserify: {
  dist: {
    files: {
      'index.js': ['src/app.js']
    }
  }
}

// in app.js
var MyModule = require('./module'); // <-- relative path required?!

Happy

Now add jquery:

browserify: {
  options: {
    shim: {
      jquery: {
        path: 'lib/jquery/jquery.js',
        exports: '$'
      }
    },
    noParse: ['lib/**/*.js'],
    alias: [
      'lib/jquery-ui/jquery-ui.js:jquery-ui'
    ]
  },
  dist: {
    files: {
      'index.js': ['src/app.js']
    }
  }
}

// in app.js
var $ = require('jquery');
require('jquery-ui');
var MyModule = require('./module');

Happy

Now add specs:

options: {
  shim: {
    jquery: {
      path: 'lib/jquery/jquery.js',
      exports: '$'
    }
  },
  noParse: ['lib/**/*.js'],
  alias: [
    'lib/jquery-ui/jquery-ui.js:jquery-ui'
  ]
},
dist: {
  files: {
    'app.js': 'src/app.js'
  }
},
spec: {
  files: {
    'spec/specs.js': ['spec/src/**/*helper.js', 'spec/src/**/*spec.js']
  }
}

// in app.js
var $ = require('jquery');
require('jquery-ui');
var MyModule = require('./module');

// in spec/src/module_spec.js
describe("MyModule", function() {
  var MyModule = require('../../src/module'); // <-- This looks like butt!!!
});

Sad

To summarize: How do I...

1. Use require('module') instead of require('../../src/module') or require('./module')?
2. reuse ./index.js in spec/specs.js without duplicating work? (And preventing src/app.js from running as it's an entry module).

Solution 1

These answers depend on how the rest of your project is setup, but maybe it's a good starting point. Also, you will need to use current v2 beta of grunt-browserify for this to actually work (npm install grunt-browserify@2).

1.

You can use aliasMapping to create some dynamic aliases for your modules. Just for clarity, lets move all your modules to src/modules/. Then, aliasMapping configuration could be something like this:

options: {
  aliasMappings: {
    cwd: 'src',
    src: ['modules/**/*.js']
  }
}

Lets suppose you have a module in src/modules/magic/stuff.js, then you can require it like this, regardless of where the .js file that's doing the require is located:

var magicStuff = require('modules/magic/stuff.js');

2.

Not sure about this one. Your project structure shows a spec/index.js, but you mention spec/specs.js. Are they supposed to be the same file?

Anyways, what duplicate work are you talking about? Because ./index.js has a different entry file than spec/index.js. If you are looking for a way to include ./index.js in specs/, then maybe you can copy it before running the tests instead of building it from scratch.

Solution 2

Simple answer:

The simplest is to use the paths option of browserify. I use it for some months with great success. I have even made a starter kit that uses this feature: https://github.com/stample/gulp-browserify-react-phonegap-starter

var b = browserify('./app', {paths: ['./node_modules','./src/js']});

paths - require.paths array to use if nothing is found on the normal node_modules recursive walk

If you have a file in src/js/modulePath/myModule.js this won't let you write require("myModule") everywhere, but rather require("modulePath/myModule"), from any of your other source files.

Deprecated option?

It does not seem so!

The Browserify module resolution algorithm mirrors the resolution algorithm in NodeJS. The paths option of Browserify is thus the mirror of NODE_PATH env variable behavior for NodeJS. The Browserify author (substack) claims in this SO topic that the NODE_PATH option is deprecated in NodeJS and thus it is also deprecated in Browserify and could be removed in next versions.

I do not agree with this claim.

See the NODE_PATH documentation. It is not mentionned that the option is deprecated. However there is still an interesting mention that does in the direction of substack's claim:

You are highly encouraged to place your dependencies locally in node_modules folders. They will be loaded faster, and more reliably.

And this question has been posted in 2012 on the mailing list.

Oliver Leics: is NODE_PATH deprecated? 
Ben Noordhuis (ex core NodeJS contributor): No. Why do you ask? 

And if something is not removed in NodeJS resolution algorithm, I don't think it will be removed anytime soon from Browserify :)

Conclusion

You can either use paths option or put your code in node_modules like the official documentation and Browserify author recommends.

Personally, I don't like the idea to put my own code in node_modules as I simply keep this whole folder out of my source control. I use the paths option for some months now and did not have any problem at all, and my build speed is pretty good.

The substack's solution of putting a symlink inside node_modules could be convenient but unfortunately we have developers working with Windows here...

I think there's however a case where you don't want to use the paths option: when you are developping a library published on an NPM repository that will be required by other apps. You really don't want these library clients to have to setup special build config just because you wanted to avoid relative path hell in your lib.

Another possible option is to use remapify

Solution 3

All the answers here about aliases and opts.paths/$NODE_PATH are not great because that approach is a deprecated part of the module system in node and browserify, so it could stop working at any time.

You should learn how the node_modules algorithm works so you can effectively organize your code in a way that plays well with nested node_modules directories.

There is a section in the browserify handbook that covers avoiding ../../../../../../.. relative path issues. It can be summarized as:

• Put your internal modular code in node_modules/ or node_modules/app so that you can require('yourmodule') or require('app/yourmodule') depending on which you prefer.
• You can use a symlink if you're developing for non-windows platforms and that is what you prefer.

Don't use opts.path/$NODE_PATH. It makes your project:

• implicitly depend on a non-obvious configuration or environment setting
• harder to make work in both node and the browser
• vulnerable to changes in the module system since array paths are deprecated in node and browserify

Solution 4

I think the absolute best way to go is, as Sebastien Lorber notes, with setting the path in your calling of browserify through a pipe.

But with the latest version of browserify, (as of this moment, that is [email protected] ) the path variable stores the only paths that Browserify will use for its process. So setting the paths variable will exclude say... your global folders for node, as far as I can tell. As a result, you'll need a Gulp task that looks sort of like this:

gulp.task('reactBuild', function() {
  return gulp.src(newThemeJSX)
    .pipe(browserify({
        debug: true,
        extensions: ['.jsx', '.js', '.json'],
        transform: [reactify],
        paths: ['../base_folder/node_modules', '/usr/lib/node_modules']
    }))
    .pipe(gulp.dest(newThemeBuilt))
    .on('error', function(error) {
        console.log(error);
    });
});

View more solutions

Comments

[Sukima]

spec/index.js vs spec/specs,js is a typo. Edited question. Thank you for answering. Seems like the truth is the original (older version) wasn't designed to do this setup. I think the better question is is how should a project like this be organized Many tutorials don't bundle production scripts and testing scripts. Add something like testem and it no wonder people get confused. We need a meta guideline for integrating all these packages together harmoniously.

[Joni Bekenstein]

Probably the root problem is the diversity of the JS ecosystem. It is still young. Maybe some day duplicated packages/utilities will stop to coexist, and one of them will become the default resource for solving X thing. Right now, when it comes to JS, you have at least 2/3 different libraries doing the same thing. So it's specially hard for something like a build system to be totally agnostic and work well for all the possible combinations of libraries.

[eightyfive]

Is this aliasMappings option specific to grunt-browserify or is it a wrapper over some browserify hidden option? I'm trying to get the same behavior with opts.basedir option, but with no luck..

[David Kaneda]

Just came across this thread, and this option is way better for the current browserify/grunt-browserify. For a sample of how it looks in grunt-browserify (verified), see here: cl.ly/VXyx (notice paths is under browserify options)

[Sebastien Lorber]

thanks for your insights. Do you know why opts.path has been deprecated? I don't like using deprecated solutions but I however think this deprecated option is much more convenient than putting code in node_modules. Do you know why it has been deprecated? Any github issue? I can't find any

[Sebastien Lorber]

By the way, on the official documentation (nodejs.org/api/modules.html) we can read: In order to make modules available to the node REPL, it might be useful to also add the /usr/lib/node_modules folder to the $NODE_PATH environment variable. Since the module lookups using node_modules folders are all relative, and based on the real path of the files making the calls to require(), the packages themselves can be anywhere. Please tell us where you've seen it's deprecated becauuse the official documentation doesn't reflect this.

[Sebastien Lorber]

Ok so I found some reference to what you are saying here: nodejs.org/api/all.html#all_loading_from_the_global_folders . These are mostly for historic reasons. You are highly encouraged to place your dependencies locally in node_modules folders. They will be loaded faster, and more reliably. It is a recommendation but but it actually doesn't say the feature is deprecated and may be removed. It is not deprecated also according to this ML discussion: groups.google.com/forum/#!topic/nodejs/EENwg8GAJd8

[pavloskkr]

I find it really weird to put your application code in node_modules. :S

[JMM]

Putting the application code in node_modules breaks transforms that are configured programatically via the API. See the concept of "top-level" files. That concept, and behavior of transforms, are poorly explained in the documentation. See this thread for some discussion.

[Cory]

Has anyone else seen an issue when using paths with Browserify 8.1.1? I've been seeing "cannot find module" messages and reverted back to 8.0.1.

[aristidesfl]

Using node_modules seems to be one of the simplest solutions. The big problem with it is that many people have configured git and editors to ignore everything inside node_modules. Even if you disregard collaborator's configurations, it is currently impossible in Sublime Text to ignore only the root node_modules folder. Also your git root might not match your node_modules for all projects, making a global ~/.gitignore unreliable. The best would be if an additional folder called local_modules was supported in the same way node_modules is.

[airtonix]

@substack moving your project code into node_modules is a hilarously bad idea which will just end up causing issues latter on.. oh remember not to rm -rf ./node_modules