How to correctly build NestJS app for production with node_modules dependencies in bundle?

Question

After nest build or nest build --webpack dist folder does not contain all required modules and I got Error: Cannot find module '@nestjs/core' when trying to run node main.js.

I could not find any clear instructions on https://docs.nestjs.com/ on how to correctly build app for production, so maybe I missed something?

Solution 1

Out of the box, nest cli does not support including the node_modules dependencies into the dist bundle.

---

However, there are some community examples of custom webpack configs that include the dependencies in the bundle, e.g. bundled-nest. As described in this issue, it is necessary to include the webpack.IgnorePlugin to whitelist unused dynamic libraries.

Solution 2

bundle-nest has been archived/discontinued:

We've concluded that it is not recommended to bundle NestJS, or actually, NodeJS web servers in general. This is archived for historical reference during the period of time when the community was attempting to tree-shake, and bundle NestJS apps. Refer to @kamilmysliwiec comment for details:

In many real-world scenarios (depending on what libraries are being used), you should not bundle Node.js applications (not only NestJS applications) with all dependencies (external packages located in the node_modules folder). Although this may make your docker images smaller (due to tree-shaking), somewhat reduce the memory consumption, slightly increase the bootstrap time (which is particularly useful in the serverless environments), it won't work in combination with many popular libraries commonly used in the ecosystem. For instance, if you try to build NestJS (or just express) application with MongoDB, you will see the following error in your console:

Error: Cannot find module './drivers/node-mongodb-native/connection' at webpackEmptyContext

Why? Because mongoose depends on mongodb which depends on kerberos (C++) and node-gyp.

Well, about mongo, you can make some exceptions (leave some modules in node_modules), can you? It's not like it's all or nothing. But still, I'm not sure you want to follow this path. I've just succeeded with bundling a nestjs application. It was a proof of concept, I'm not sure if it'll go into production. And it was hard, I might have broken something in the process, but at first glance it works. The most complex part was adminjs. It has rollup and babel as dependencies. And in the app code they unconditionally call watch for some reason (UDP noop in production). Anyways, if you'd like to follow this path you should be ready to debug/inspect your packages' code. And you might need to add workarounds as new packages are added to the project. But it all depends on your dependencies, it may be easier than in my case. For a freshly created nestjs + mysql app it was relatively simple.

The config I ended up with (it overrides the nestjs defaults):

webpack.config.js (webpack-5.58.2, @nestjs/cli-8.1.4):

const path = require('path');
const MakeOptionalPlugin = require('./make-optional-plugin');
module.exports = (defaultOptions, webpack) => {
    return {
        externals: {},  // make it not exclude `node_modules`
                        // https://github.com/nestjs/nest-cli/blob/v7.0.1/lib/compiler/defaults/webpack-defaults.ts#L24
        resolve: {
            ...defaultOptions.resolve,
            extensions: [...defaultOptions.resolve.extensions, '.json'], // some packages require json files
                                                                         // https://unpkg.com/browse/[email protected]/core-js-compat/data.js
                                                                         // https://unpkg.com/browse/[email protected]/data.json
            alias: {
                // an issue with rollup plugins
                // https://github.com/webpack/enhanced-resolve/issues/319
                '@rollup/plugin-json': '/app/node_modules/@rollup/plugin-json/dist/index.js',
                '@rollup/plugin-replace': '/app/node_modules/@rollup/plugin-replace/dist/rollup-plugin-replace.cjs.js',
                '@rollup/plugin-commonjs': '/app/node_modules/@rollup/plugin-commonjs/dist/index.js',
            },
        },
        module: {
            ...defaultOptions.module,
            rules: [
                ...defaultOptions.module.rules,

                // a context dependency
                // https://github.com/RobinBuschmann/sequelize-typescript/blob/v2.1.1/src/sequelize/sequelize/sequelize-service.ts#L51
                {test: path.resolve('node_modules/sequelize-typescript/dist/sequelize/sequelize/sequelize-service.js'),
                use: [
                    {loader: path.resolve('rewrite-require-loader.js'),
                    options: {
                        search: 'fullPath',
                        context: {
                            directory: path.resolve('src'),
                            useSubdirectories: true,
                            regExp: '/\\.entity\\.ts$/',
                            transform: ".replace('/app/src', '.').replace(/$/, '.ts')",
                        },
                    }},
                ]},

                // adminjs resolves some files using stack (relative to the requiring module)
                // and actually it needs them in the filesystem at runtime
                // so you need to leave node_modules/@adminjs/upload
                // I failed to find a workaround
                // it bundles them to `$prj_root/.adminjs` using `rollup`, probably on production too
                // https://github.com/SoftwareBrothers/adminjs-upload/blob/v2.0.1/src/features/upload-file/upload-file.feature.ts#L92-L100
                {test: path.resolve('node_modules/@adminjs/upload/build/features/upload-file/upload-file.feature.js'),
                use: [
                    {loader: path.resolve('rewrite-code-loader.js'),
                    options: {
                        replacements: [
                            {search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/edit'\)/,
                            replace: "adminjs_1.default.bundle('/app/node_modules/@adminjs/upload/src/features/upload-file/components/edit')"},

                            {search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/list'\)/,
                            replace: "adminjs_1.default.bundle('/app/node_modules/@adminjs/upload/src/features/upload-file/components/list')"},

                            {search: /adminjs_1\.default\.bundle\('\.\.\/\.\.\/\.\.\/src\/features\/upload-file\/components\/show'\)/,
                            replace: "adminjs_1.default.bundle('/app/node_modules/@adminjs/upload/src/features/upload-file/components/show')"},
                        ],
                    }},
                ]},

                // not sure what babel does here
                // I made it return standardizedName
                // https://github.com/babel/babel/blob/v7.16.4/packages/babel-core/src/config/files/plugins.ts#L100
                {test: path.resolve('node_modules/@babel/core/lib/config/files/plugins.js'),
                use: [
                    {loader: path.resolve('rewrite-code-loader.js'),
                    options: {
                        replacements: [
                            {search: /const standardizedName = [^;]+;/,
                            replace: match => `${match} return standardizedName;`},
                        ],
                    }},
                ]},

                // a context dependency
                // https://github.com/babel/babel/blob/v7.16.4/packages/babel-core/src/config/files/module-types.ts#L51
                {test: path.resolve('node_modules/@babel/core/lib/config/files/module-types.js'),
                use: [
                    {loader: path.resolve('rewrite-require-loader.js'),
                    options: {
                        search: 'filepath',
                        context: {
                            directory: path.resolve('node_modules/@babel'),
                            useSubdirectories: true,
                            regExp: '/(preset-env\\/lib\\/index\\.js|preset-react\\/lib\\/index\\.js|preset-typescript\\/lib\\/index\\.js)$/',
                            transform: ".replace('./node_modules/@babel', '.')",
                        },
                    }},
                ]},
            ],
        },
        plugins: [
            ...defaultOptions.plugins,
            // some optional dependencies, like this:
            // https://github.com/nestjs/nest/blob/master/packages/core/nest-application.ts#L45-L52
            // `webpack` detects optional dependencies when they are in try/catch
            // https://github.com/webpack/webpack/blob/main/lib/dependencies/CommonJsImportsParserPlugin.js#L152
            new MakeOptionalPlugin([
                '@nestjs/websockets/socket-module',
                '@nestjs/microservices/microservices-module',
                'class-transformer/storage',
                'fastify-swagger',
                'pg-native',
            ]),
        ],

        // to have have module names in the bundle, not some numbers
        // although numbers are sometimes useful
        // not really needed
        optimization: {
            moduleIds: 'named',
        }
    };
};

make-optional-plugin.js:

class MakeOptionalPlugin {
    constructor(deps) {
        this.deps = deps;
    }

    apply(compiler) {
        compiler.hooks.compilation.tap('HelloCompilationPlugin', compilation => {
            compilation.hooks.succeedModule.tap(
                'MakeOptionalPlugin', (module) => {
                    module.dependencies.forEach(d => {
                        this.deps.forEach(d2 => {
                            if (d.request == d2)
                                d.optional = true;
                        });
                    });
                }
            );
        });
    }
}

module.exports = MakeOptionalPlugin;

rewrite-require-loader.js:

// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#escaping
function escapeRegExp(string) {
  return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}

function processFile(source, search, replace) {
    const re = `require\\(${escapeRegExp(search)}\\)`;
    return source.replace(
        new RegExp(re, 'g'),
        `require(${replace})`);
}

function processFileContext(source, search, context) {
    const re = `require\\(${escapeRegExp(search)}\\)`;
    const _d = JSON.stringify(context.directory);
    const _us = JSON.stringify(context.useSubdirectories);
    const _re = context.regExp;
    const _t = context.transform || '';
    const r = source.replace(
        new RegExp(re, 'g'),
        match => `require.context(${_d}, ${_us}, ${_re})(${search}${_t})`);
    return r;
}

module.exports = function(source) {
    const options = this.getOptions();
    return options.context
        ? processFileContext(source, options.search, options.context)
        : processFile(source, options.search, options.replace);
};

rewrite-code-loader.js:

function processFile(source, search, replace) {
    return source.replace(search, replace);
}

module.exports = function(source) {
    const options = this.getOptions();
    return options.replacements.reduce(
        (prv, cur) => {
            return prv.replace(cur.search, cur.replace);
        },
        source);
};

The supposed way to build the app is:

$ nest build --webpack

I didn't bother with source maps, since the target is nodejs.

It's not a config you can just copy-paste, you should figure out what's needed for your project yourself.

One more trick here, but well, you probably won't need it.

UPD adminjs seems to come with prebuilt bundles, so this config may be significantly simpler.