How can I run multiple npm scripts in parallel?

Question

In my package.json I have these two scripts:

  "scripts": {
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server",
  }

I have to run these 2 scripts in parallel everytime I start developing in Node.js. The first thing I thought of was adding a third script like this:

"dev": "npm run start-watch && npm run wp-server"

... but that will wait for start-watch to finish before running wp-server.

How can I run these in parallel? Please keep in mind that I need to see the output of these commands. Also, if your solution involves a build tool, I'd rather use gulp instead of grunt because I already use it in another project.

Solution 1

Use a package called concurrently.

npm i concurrently --save-dev

Then setup your npm run dev task as so:

"dev": "concurrently --kill-others \"npm run start-watch\" \"npm run wp-server\""

Solution 2

If you're using an UNIX-like environment, just use & as the separator:

"dev": "npm run start-watch & npm run wp-server"

Otherwise if you're interested on a cross-platform solution, you could use npm-run-all module:

"dev": "npm-run-all --parallel start-watch wp-server"

Solution 3

From windows cmd you can use start:

"dev": "start npm run start-watch && start npm run wp-server"

Every command launched this way starts in its own window.

Solution 4

You should use npm-run-all (or concurrently, parallelshell), because it has more control over starting and killing commands. The operators &, | are bad ideas because you'll need to manually stop it after all tests are finished.

This is an example for protractor testing through npm:

scripts: {
  "webdriver-start": "./node_modules/protractor/bin/webdriver-manager update && ./node_modules/protractor/bin/webdriver-manager start",
  "protractor": "./node_modules/protractor/bin/protractor ./tests/protractor.conf.js",
  "http-server": "./node_modules/http-server/bin/http-server -a localhost -p 8000",
  "test": "npm-run-all -p -r webdriver-start http-server protractor"
}

-p = Run commands in parallel.

-r = Kill all commands when one of them finishes with an exit code of zero.

Running npm run test will start Selenium driver, start http server (to serve you files) and run protractor tests. Once all tests are finished, it will close the http server and the selenium driver.

Solution 5

You can use one & for parallel run script

"dev": "npm run start-watch & npm run wp-server"

Reference link

View more solutions

Comments

[raine]

node ./node_modules/concurrently/src/main.js is not needed. concurrent will work just fine in scripts because the module installs a bin to ./node_modules/.bin/concurrent

[Patrick Burtchaell]

You can also use `npm bin`/concurrent.

[Stijn de Witt]

No, it's not better because it does not work on all platforms.

[Stijn de Witt]

Nope, as & operator is not supported on Windows.

[james_womack]

@StijndeWitt my post says "If you're not on Windows...". 0% of the folks I work with, at one of the largest tech companies in the world, run Node on Windows. So clearly my post is still valuable to many developers.

[Stijn de Witt]

It's kind of a circular way of reasoning isn't it though? If you write your npm scripts like this you will not be able to use Windows because it won't work. So no one uses Windows, so it doesn't matter that it does not work... You end up with platform dependent software. Now if the thing that needs to be done is very hard to do cross-platform, than that might be a good trade-off to make. But this problem right here is very easy to do with standard npm scripts such as concurrently and parallelshell.

[Stijn de Witt]

There is also parallelshell. I actually recommend that one as concurrently uses multiple streams that mess with console output (colors may go weird, cursor gone) whereas parallelshell doesn't have that issue.

[james_womack]

@StijndeWitt None of my reasoning was circular. I made a statement of fact sans reasoning. We're posting techniques common to Node developers, many of whom build & deploy on Linux servers. Yes, it should work on Windows if it's a userland script, but the majority of npm scripts are for development and deployment—mostly on *nix machines. Regarding the modules you mentioned a) it's an enormous stretch to call concurrently and parallelshell "standard" (~1500 downloads a day is far from standard in NPMland) and b) if you need additional software for a parallel process, you might as well use Gulp.

[james_womack]

@StijndeWitt I appreciate being made aware of those modules though—thank you

[Stijn de Witt]

@You're welcome and sorry if I came off as harsh. My nope was directed at the statement that bash shell scripts are the best bet. You see I am one of those poor Windows devs for who all those scripts break so for me it's a recurring annoyance to have to find workarounds / fixes for them. I feel that as NodeJS / NPM is multi-platform we should strive for our packages to be so as well... But it all depends on the situation of course.

[Stijn de Witt]

What I meant with circular reasoning was that if we write our script to only work on *nix, it's should be no surprise that almost all devs run Node on *nix. But we can get Windows devs to join the party if we tweak our scripts to work on Windows as well.

[james_womack]

Re: circular reasoning, gotcha

[Kimmo]

The bugs in concurrently mentioned by @StijndeWitt have now been fixed in 2.0.0 release. You can use --raw mode to preserve colors in output.

[Ashley Coolman]

I did not know that. What platforms doesn't it work on? @Corey - update your answer with the warning on inter-op and I'll upvote you

[congusbongus]

@rane "concurrent" command is deprecated, use "concurrently" instead.

[Kamil Tomšík]

I do this - from time to time when I "ctrl-c" npm, the command keeps hanging on in background... Any ideas?

[Trevor]

& works on Windows, but it works differently. On OSX, it will run both commands concurrently, but on Windows, it will run the first command, and after the first command exists, it will run the second command.

[TetraDev]

Perfect solution! I love that it launches the new window. Great for VS2015 package.json needs

[jtzero]

@StijndeWitt parallelshell has been deprecated in favor of npm-run-all github.com/keithamus/…

[Stijn de Witt]

@jtzero That's a great development! Thanks for pointing it out.

[snakeoil]

I always thought it was && but this doesn't work and I didn't know you could use & and I had not considered | all of which are subtly different..

[j2L4e]

a && b starts b after a finished successfully, but nodemon never stops without errors, so that can't work. a & b starts a, moves it to the background and starts b right away. Win! a | b pipes the stdout of a to the stdin of b which requires both running simultaneously. Although this might seem to have the desired effect, you shouldn't use it here.

[asenovm]

I wonder how this works properly for running the tests, though. Whilst webdriver-start and http-server can run in parallel, the protractor task should only run after the first two.

[olefrank]

I'm trying to do npm dev but it's not recognized (like npm start or npm test). So I have to do npm run dev. Why is that?

[ngryman]

@KamilTomšík & is a really bad idea as it detaches the process. It means that npm will not be the parent process anymore. You'll end up with a zombie npm run start-watch that won't be killed with ctrl-c.

[ngryman]

No it's not as it detaches the process, you won't be able to kill it in a simple fashion.

[Nick McCurdy]

@olefrank Because custom npm scripts need to be run with the run or run-script commands. Commands like npm test are aliases for recommended script names, but you can run your custom script with npm run dev.

[Ruslan Prokopchuk]

Just add wait to mitigate problem with hanging processes: "dev": "npm run start-watch & npm run wp-server & wait"

[musicin3d]

@ngryman That's what I expected too. However, I tried this and it does kill all three processes (dev, start-watch, and wp-server) when you hit Ctrl+C.

[Benny Neugebauer]

This does not work if you have watcher tasks because && waits for the first command to finish before starting the second command and a watcher task will never finish.

[r3wt]

@asenovm for order dependant tasks, why not just use gulp and gulp-sync?

[Addison]

@BennyNeugebauer The commands are preceded with the "start" command which opens up a new command line for each of the commands. I was confused at first as well because I thought "using the && operator will not work". This solution is very simple and requires no additional packages/work from the developer.

[zhekaus]

This is wrong. Command will be run sequentially. On Windows you have to use a plugin in order to run commands simultaneously.

[Damjan Pavlica]

--kill-others killed my code without reason. I just removed it and it works.

[Andrew T Finnell]

There has to be a better way for us to manage Javascript build/run scripts. Everything for this platform seems tacked together. quotes with escaped quotes and npm builds to call other 'npm run' builds.. This is getting pretty painful.

[Frank Nocke]

@musicin3d I am making the opposite experience right now: All but the last process in the &-chain keeps working…

[magikMaker]

Exactly, it's simple and elegant, no need for dependencies or other magic.

[binki]

It’s not a zombie. But & on unix prevents the command from responding to C-c/C-z and also prevents its return code from propagating in case of a failure.

[binki]

Isn’t this Windows-specific?

[airtonix]

it also now means i have to use windows work on your projects.

[Joshua Pinter]

If you're running npm run commands, you case use a nifty shorthand and do concurrently "npm:server" "npm:watch" "npm:electron". Clean and prefixes the output correctly in the stdout. Perfection.

[Vicary]

Down voting this because daemonize is not the optimal way to run multiple commands.

[cojack]

@Corey it's not better solution, but it's the best solution, who cares about windows fuzzy users.

[Jorge Fuentes González]

@Ginzburg Because don't works the same for all platforms, like you can see in other answers.

[frank]

This is a good solution without any further dependency of the external libs

[Jeb50]

Like your solution. Also have UI (node app) and API (Angular in a subfolder src, guess is cd src/ng serve), only the first part works. For example node app& cd src& ng serve.

[Benjamin]

I'm working with Webstorm using Windows 10's default cmd and single & works fine

[Benison Sam]

Will this also work in Windows? Sorry, I am pretty new to node and I don't know how to verify this!

[Rong.l]

I think npm-run-all is a better solution if you are developing a public library or some situations like for example running babel watch and webpack watch simultaneously and keeping logs of both shown in the console.

[HarshitMadhav]

Where to add this dev I am adding this in script but it still not allowing me to run two "npm-start"

[Franva]

@BenisonSam I tried on my Windows pc, it doesn't run the 2nd command even with single "&"

[nick]

I have updated my script, i thought that was working , I posted my update above

[vaughan]

@AndrewTFinnell There are other solutions involving external config files such as github.com/pawelgalazka/tasksfile or before that grunt and gulp or you could build your own cli or use your chosen framework's cli. Tools like concurrently win because they are simple and don't obfuscate the commands that are being run - you don't have to look in another file, you don't have a large api surface layer that can contain bugs. Also by simply running npm run you can see exactly which commands are being run. You can also use concurrently programmatically...

[Anja]

what is -p and -r for on npm?

[Martin Braun]

The same answer got posted 4 years ago and has less upvotes than this. Also it's discussed enough why this approach is a bad idea, already. Uhm, why this has so many upvotes, again?

[Martin Braun]

I use npm-run-all and I get the output of both processes in the terminal.

[Alexandre]

Right, I think they did some update on the way to handle output, I lately use npm-run-all and seem to work pretty well so far.

[Mick]

@MartinBraun quick and easy

[Crupeng]

No it does not.

[Brian Burns]

Another useful option is -l or --print-labels - it prints the task name as a prefix on each line of output, so you can tell them apart. Nicely colored also.

[newbreedofgeek]

tnx - This is the answer - all the other mentioned solutions are overkill

[Shawn]

"&": runs concurrently "&&": runs in sequence

[Martin Braun]

@Mick He literally copied Corey's answer that is four years older. If you want to support a quick and easy solution, upvote his and downvote this answer.

[Freax]

I got this error. Where from start command. Error: spawn start ENOENT at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19) at onErrorNT (internal/child_process.js:467:16) at processTicksAndRejections (internal/process/task_queues.js:82:21)

[Mosia Thabo]

@Freax Check this out: stackoverflow.com/questions/57054403/…

[Admin]

two notes: 1) should an error occur in the full command, & will cause the backgrounded process to become orphaned, not zombied. 2) adding & wait on the end backgrounds the last command, preventing it from accepting keyboard input sincewait is now in the foreground. this may or may not be a problem for you. (it will be a problem if you are running a fancy interactive program like jest --watch as the last command.)

[B Bulfin]

@zhekaus: I actually just tried it. Windows 10. No plugins. Multiple simultaneous watchers, each in a window.

[Harshit Gupta]

Double || didn't seem to work on my Windows 10 PowerShell, however, a single | seems to do fine even on PowerShell. I tried it with just two commands and could only see the output of 2nd part and not 1st one.

[Entity Black]

@HarshitGupta || might not be implemented in Windows Ppowershell. Apparently it was introduced in PowerShell [Core] 7.0 but might not be backported into Windows Powershell. Sadly my solution isn't bulletproof.

[Firez]

Great this works, this works in Windows as well.