Dynamic loading of external modules in webpack fails

Question

I am trying to set up the following architecture: a core React application that gets built with some basic functionality, and the ability to load additional React components at runtime. These additional React components can be loaded on-demand, and they are not available at build time for the core application (so they cannot be included in the bundles for the core application, and must be built separately). After researching for some time, I came across Webpack Externals, which seemed like a good fit. I am now building my modules separately using the following webpack.config.js:

const path = require('path');
const fs = require('fs');

process.env.BABEL_ENV = 'production';
process.env.NODE_ENV = 'production';

const appDirectory = fs.realpathSync(process.cwd());
const resolveApp = relativePath => path.resolve(appDirectory, relativePath);

module.exports = {
  entry: './src/MyModule.jsx',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'MyModule.js',
    library: 'MyModule',
    libraryTarget: 'umd'
  },
   externals: {
    "react": "react",
    "semantic-ui-react": "semantic-ui-react"
   },
   module: {
    rules: [
        {
            test: /\.(js|jsx|mjs)$/,
            include: resolveApp('src'),
            loader: require.resolve('babel-loader'),
            options: {              
              compact: true,
            },
        }
    ]
  },
  resolve: {
    extensions: ['.wasm', '.mjs', '.js', '.json', '.jsx']
  }
};

Took a look at the generated MyModule.js file, and it looks correct to me.

Now, in my core app, I am importing the module as follows:

let myComponent = React.lazy(() => import(componentName + '.js'));

where componentName is the variable that matches the name of my module, in this case, "MyModule" The name is not known at build time, and the file is not present in the src folder at build time. To avoid errors from webpack when building this code with an unknown import, I have added the following to my webpack.config.js for the core project:

module.exports = {
    externals: function (context, request, callback/*(err, result)*/) {
        if (request === './MyModule.js') {
            callback(null, "umd " + request);
        } else {
            callback();
        }
    }
}

I have confirmed that the function in externals gets called during the build, and the if condition is matched for this module. The build succeeds, and I am able to run my core application.

Then, to test dynamic loading, I drop MyModule.js into the static/js folder where the bundles for my core app live, then I navigate to the page in my core app that requests MyModule via let myComponent = React.lazy(() => import(componentName + '.js'));

I see a runtime error in the console on the import line,

TypeError: undefined is not a function
    at Array.map (<anonymous>)
    at webpackAsyncContext 

My guess is it's failing to find the module. I don't understand where it is looking for the module, or how to get more information to debug.

Solution

Turns out that I was making a couple of incorrect assumptions about webpack and dynamic loading.

I was having issues with two things - the kind of module I was loading, and the way that I was loading it.

1. Dynamic importing is not yet a standard ES feature - it is due to be standardized in ES 2020. This dynamic import will only return a module if the module object you are attempting to load is an ES6 module (aka something that contains an 'export ModuleName'). If you attempt to load something packed up as a CommonJS module, AMD, UMD, the import will succeed, but you will get an empty object. Webpack does not appear to support creating bundles in ES6 format - it can create a variety of module types, and in my config file above, I was actually creating UMD modules (configured via libraryTarget setting).

2. I had issues with the import statement itself because I was using it within an app bundled by Webpack. Webpack reinterprets the standard ES import statement. Within a standard webpack config (including the one you get from CRA), webpack uses this statement as a split point for bundles, so even modules that are dynamically imported are expected to be there at webpack build time (and the build process will fail if they are not available). I had tried to use webpack externals to tell webpack to load the modules dynamically, which allowed the build to succeed without the modules being there. However, the app still used Webpack's import function instead of the standard JS import function at runtime. I confirmed this by attempting to run import('modulename') from the browser console and getting a different result than my app, which was bundled with webpack.

To solve problem #2, you can tell Webpack to not reinterpret the ES dynamic import by adding some annotation to the import statement.

import(/*webpackIgnore: true*/ 'path/to/module.js');

This will both prevent Webpack from attempting to find and bundle the dynamically imported module at build time, and attempting to import it at runtime. This will make behavior in the app match behavior in the browser console.

Problem #1 was a bit more difficult to solve. As I mentioned above, importing a non-ES6 module will return an empty object (if you await the promise or use .then()). However, as it turns out, the file itself does load and the code gets executed. You can export the module in the "window" format using Webpack, and then load it as follows.

await import(/*webpackIgnore: true*/`path/to/module.js`);
let myModule = window['module'].default;

Another potential solution that avoids using the window object is building the module using a system capable of producing ES6 modules (so, not Webpack). I ended up using Rollup to create an ES6 module that pulled all dependencies into a single file, and ran the output through Babel. This produced a module that loaded successfully via a dynamic ES import. The following was my rollup.config.js (note that I included all external node modules needed in my module - this bloated the module size but is a requirement for my specific application - yours will likely differ and you will need to configure rollup to exclude the modules)

// node-resolve will resolve all the node dependencies
import commonjs from 'rollup-plugin-commonjs';
import resolve from 'rollup-plugin-node-resolve';
import babel from 'rollup-plugin-babel';
import replace from 'rollup-plugin-replace';

export default {
  input: 'src/myModule.jsx',
  output: {
    file: 'dist/bundle.js',
    format: 'esm'
  },
  plugins: [
    resolve(),
    babel({
      exclude: 'node_modules/**'
    }),
    commonjs({
      include: 'node_modules/**',      
      namedExports: {
        'node_modules/react/index.js': ['Children', 'Component', 'PropTypes',   'PureComponent', 'React', 'createElement', 'createRef', 'isValidElement', 'cloneElement', 'Fragment'],
        'node_modules/react-dom/index.js': ['render', 'createElement', 'findDOMNode', 'createPortal'],
        'node_modules/react-is/index.js': ['isForwardRef']
      }
    }),
    replace({
      'process.env.NODE_ENV': JSON.stringify( 'production' )
    })
  ]
}

Comments

[Alies]

Thanks for such detailed answer! It seems like /*webpackIgnore: true*/ doesn't work with webpack v3 that I can't upgrade. Do you know any other options?

[SlyCaptainFlint]

Are you confident that Webpack 3 uses dynamic import statements for code splitting? If it does not, you can just leave the import statement alone. If yes, I suspect you will need to further investigate webpack externals (see my original question - that was the first thing I tried).

[treznik]

Thank you sir for paving the way! I was going through the same thing this evening :).

[Harsh]

Webpack now produce ES6 modules. See webpack.js.org/configuration/output/#outputlibrarytarget for details See gist.github.com/hargup/5f3c307111fc1d925d640deb8deb8807 for sample webpack config.

[Hyster]

Were you able to make the dynamic react components and the react app use the same React library in runtime?