webpack dynamic module loader by require

Question

OK, i have searched high and low but cannot reliably deterrmine if this is or is not possible with webpack.

https://github.com/webpack/webpack/tree/master/examples/require.context Appears to indicate that one can pass a string to a function and it load a module...

But my attempt is just not working: webpack.config.js

'use strict';
let webpack     = require('webpack'),
    jsonLoader  = require("json-loader"),
    path        = require("path"),
    fs          = require('fs'),
    nodeModules = {};

fs.readdirSync('node_modules')
    .filter(function(x) {
        return ['.bin'].indexOf(x) === -1;
    })
    .forEach(function(mod) {
        nodeModules[mod] = 'commonjs ' + mod;
    });


let PATHS = {
    app: __dirname + '/src'
};

module.exports = {
    context: PATHS.app,
    entry: {
        app: PATHS.app+'/server.js'
    },
    target: 'node',
    output: {
        path: PATHS.app,
        filename: '../build/server.js'
    },
    externals: nodeModules,
    performance: {
        hints: "warning"
    },
    plugins: [
        jsonLoader
    ],
    resolve: {
        modules: [
            './node_modules',
            path.resolve(__dirname),
            path.resolve(__dirname + "/src"),
            path.resolve('./config')
        ]
    },
    node: {
        fs: "empty"
    }
};

The server.js

let _ = require('lodash');
let modules = [ "modules/test" ];

require( 'modules/test' )();

_.map( modules, function( module ){
    require( module );
});

The module in modules/ named test.js

module.exports = () => {
    console.log('hello world');
};

But the result is always the same... the pm2 logs just say hello world for the static require... but for the dynamic load of the same module

Error: Cannot find module "."

All i want to be able to do is loop through an array of paths to modules and load then...

Solution 1

You cannot use a variable as argument to require. Webpack needs to know what files to bundle at compile time. As it does no program flow analysis, it can't know what you pass to the function. In that case it might be obvious, but this could go as far as using user input to decide what module to require, and there is no way webpack can possibly know which modules to include at compile time, so webpack does not allow it.

The example you posted is a bit different. You could use require with a concatenated string. For example:

require(`./src/${moduleName}/test`);

Which modules does webpack need to include in the bundle? The variable moduleName could be anything, so the exact module is not known at compile time. Instead it includes all modules that could possibly match the above expression. Assuming the following directory structure:

src
├─ one
│   └─ test.js
├─ two
│   ├─ subdir
│   │   └─ test.js
│   └─ test.js
└─ three
    └─ test.js

All of these test.js files will be included in the bundle, because moduleName could be one or something nested like two/subdir.

For more details see require with expression of the official docs.

You cannot loop through an array and import every module of the array, with the above exception by concatenating a string, but that has the effect of including all possible modules and should generally be avoided.

Solution 2

I ran into this problem in an electron environment. My use case was being able to require dynamically created files in an IDE like application. I wanted to use the electron require, which is basically a NodeJS Common module loader. After some back and forth I landed on a solution that uses webpack's noParse module configuration.

First create a module that that will be ignored by webpack's parser:

// file: native-require.js
// webpack replaces calls to `require()` from within a bundle. This module
// is not parsed by webpack and exports the real `require`
// NOTE: since the module is unparsed, do not use es6 exports
module.exports = require

In my webpack config, under module, instruct the bundler not to parse this module:

{
  module: {
    noParse: /\/native-require.js$/,
  }
}

Lastly, in any bundle where you want to access the original require:

import nativeRequire from './native-require'
const someModule = nativeRequire('/some/module.js') // dynamic imports

Solution 3

A bit late....but... since you are bundling to target: 'node', there is a workaround to dynamic requiring modules, and bypassing the "the effect of including all possible modules".

The solution is lifted from:

Using dynamic require on node targets WITHOUT resolve or bundle the target module · Issue #4175 · webpack/webpack

Quoted from that comment:

const requireFunc = typeof __webpack_require__ === "function" ? __non_webpack_require__ : require;
const foo = requireFunc(moduleName);

Bundles to:

const requireFunc = true ? require : require;
const foo = requireFunc(moduleName);

Comments

[Stephan-v]

If only things were this clearly explained in the Webpack documentation.

[MoSheikh]

Thank you so much, I appreciate you clearing that up, and especially the detailed description of the underlying cause. Do you have any suggestions for how I should be loading my files for display in terms of general practices? I'm very new to React programming and any serious programming in general, so a rough idea on what kind of design patterns would be suitable would be great, so I could research them further. Thank you.

[sunny-lan]

Is there a way to get webpack to bundle every single module in package.json? (of course it would not all be loaded onto the clients browser unless one of them is needed)

[BillyRayCyrus]

This solution worked well for me, with a couple of tweaks: 1. To access, I had to use: import nativeRequire = require('./native-require'). It was complaining that the module did not have a default export. 2. To export the global require, in native-require.js I had to use: module.exports = require. This is because global.require was undefined for me (Node 10.13, webpack 4.29)

[xlm]

Additional doco webpack.js.org/api/module-methods/…

[niranjan_harpale]

Basically starting path can NOT be dynamic!! for example this is still valid: require('./src/' + moduleName ); . Please read the attached require with expression doc.

[Alexey Romanov]

Can you just use __non_webpack_require__?