Call async/await functions in parallel

Question

As far as I understand, in ES7/ES2016 putting multiple await's in code will work similar to chaining .then() with promises, meaning that they will execute one after the other rather than in parallel. So, for example, we have this code:

await someCall();
await anotherCall();

Do I understand it correctly that anotherCall() will be called only when someCall() is completed? What is the most elegant way of calling them in parallel?

I want to use it in Node, so maybe there's a solution with async library?

EDIT: I'm not satisfied with the solution provided in this question: Slowdown due to non-parallel awaiting of promises in async generators, because it uses generators and I'm asking about a more general use case.

Solution 1

You can await on Promise.all():

await Promise.all([someCall(), anotherCall()]);

To store the results:

let [someResult, anotherResult] = await Promise.all([someCall(), anotherCall()]);

Note that Promise.all fails fast, which means that as soon as one of the promises supplied to it rejects, then the entire thing rejects.

const happy = (v, ms) => new Promise((resolve) => setTimeout(() => resolve(v), ms))
const sad = (v, ms) => new Promise((_, reject) => setTimeout(() => reject(v), ms))

Promise.all([happy('happy', 100), sad('sad', 50)])
  .then(console.log).catch(console.log) // 'sad'

If, instead, you want to wait for all the promises to either fulfill or reject, then you can use Promise.allSettled. Note that Internet Explorer does not natively support this method.

const happy = (v, ms) => new Promise((resolve) => setTimeout(() => resolve(v), ms))
const sad = (v, ms) => new Promise((_, reject) => setTimeout(() => reject(v), ms))

Promise.allSettled([happy('happy', 100), sad('sad', 50)])
  .then(console.log) // [{ "status":"fulfilled", "value":"happy" }, { "status":"rejected", "reason":"sad" }]

Note: If you use Promise.all actions that managed to finish before rejection happen are not rolled back, so you may need to take care of such situation. For example if you have 5 actions, 4 quick, 1 slow and slow rejects. Those 4 actions may be already executed so you may need to roll back. In such situation consider using Promise.allSettled while it will provide exact detail which action failed and which not.

Solution 2

TL;DR

Use Promise.all for the parallel function calls, the answer behaviors not correctly when the error occurs.

---

First, execute all the asynchronous calls at once and obtain all the Promise objects. Second, use await on the Promise objects. This way, while you wait for the first Promise to resolve the other asynchronous calls are still progressing. Overall, you will only wait for as long as the slowest asynchronous call. For example:

// Begin first call and store promise without waiting
const someResult = someCall();

// Begin second call and store promise without waiting
const anotherResult = anotherCall();

// Now we await for both results, whose async processes have already been started
const finalResult = [await someResult, await anotherResult];

// At this point all calls have been resolved
// Now when accessing someResult| anotherResult,
// you will have a value instead of a promise

JSbin example: http://jsbin.com/xerifanima/edit?js,console

Caveat: It doesn't matter if the await calls are on the same line or on different lines, so long as the first await call happens after all of the asynchronous calls. See JohnnyHK's comment.

---

Update: this answer has a different timing in error handling according to the @bergi's answer, it does NOT throw out the error as the error occurs but after all the promises are executed. I compare the result with @jonny's tip: [result1, result2] = Promise.all([async1(), async2()]), check the following code snippet

const correctAsync500ms = () => {
  return new Promise(resolve => {
    setTimeout(resolve, 500, 'correct500msResult');
  });
};

const correctAsync100ms = () => {
  return new Promise(resolve => {
    setTimeout(resolve, 100, 'correct100msResult');
  });
};

const rejectAsync100ms = () => {
  return new Promise((resolve, reject) => {
    setTimeout(reject, 100, 'reject100msError');
  });
};

const asyncInArray = async (fun1, fun2) => {
  const label = 'test async functions in array';
  try {
    console.time(label);
    const p1 = fun1();
    const p2 = fun2();
    const result = [await p1, await p2];
    console.timeEnd(label);
  } catch (e) {
    console.error('error is', e);
    console.timeEnd(label);
  }
};

const asyncInPromiseAll = async (fun1, fun2) => {
  const label = 'test async functions with Promise.all';
  try {
    console.time(label);
    let [value1, value2] = await Promise.all([fun1(), fun2()]);
    console.timeEnd(label);
  } catch (e) {
    console.error('error is', e);
    console.timeEnd(label);
  }
};

(async () => {
  console.group('async functions without error');
  console.log('async functions without error: start')
  await asyncInArray(correctAsync500ms, correctAsync100ms);
  await asyncInPromiseAll(correctAsync500ms, correctAsync100ms);
  console.groupEnd();

  console.group('async functions with error');
  console.log('async functions with error: start')
  await asyncInArray(correctAsync500ms, rejectAsync100ms);
  await asyncInPromiseAll(correctAsync500ms, rejectAsync100ms);
  console.groupEnd();
})();

Solution 3

Update:

The original answer makes it difficult (and in some cases impossible) to correctly handle promise rejections. The correct solution is to use Promise.all:

const [someResult, anotherResult] = await Promise.all([someCall(), anotherCall()]);

Original answer:

Just make sure you call both functions before you await either one:

// Call both functions
const somePromise = someCall();
const anotherPromise = anotherCall();

// Await both promises    
const someResult = await somePromise;
const anotherResult = await anotherPromise;

Solution 4

There is another way without Promise.all() to do it in parallel:

First, we have 2 functions to print numbers:

function printNumber1() {
   return new Promise((resolve,reject) => {
      setTimeout(() => {
      console.log("Number1 is done");
      resolve(10);
      },1000);
   });
}

function printNumber2() {
   return new Promise((resolve,reject) => {
      setTimeout(() => {
      console.log("Number2 is done");
      resolve(20);
      },500);
   });
}

This is sequential:

async function oneByOne() {
   const number1 = await printNumber1();
   const number2 = await printNumber2();
} 
//Output: Number1 is done, Number2 is done

This is parallel:

async function inParallel() {
   const promise1 = printNumber1();
   const promise2 = printNumber2();
   const number1 = await promise1;
   const number2 = await promise2;
}
//Output: Number2 is done, Number1 is done

Solution 5

I've created a gist testing some different ways of resolving promises, with results. It may be helpful to see the options that work.

Edit: Gist content as per Jin Lee's comment

// Simple gist to test parallel promise resolution when using async / await

function promiseWait(time) {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
        resolve(true);
    }, time);
});
}


async function test() {
    return [
    await promiseWait(1000),
    await promiseWait(5000),
    await promiseWait(9000),
    await promiseWait(3000),
    ]
}

async function test2() {
    return {
        'aa': await promiseWait(1000),
        'bb': await promiseWait(5000),
        'cc': await promiseWait(9000),
        'dd': await promiseWait(3000),
    }
}

async function test3() {
    return await {
        'aa': promiseWait(1000),
        'bb': promiseWait(5000),
        'cc': promiseWait(9000),
        'dd': promiseWait(3000),
    }
}

async function test4() {
    const p1 =  promiseWait(1000);
    const p2 =  promiseWait(5000);
    const p3 =  promiseWait(9000);
    const p4 =  promiseWait(3000);
    return {
        'aa': await p1,
        'bb': await p2,
        'cc': await p3,
        'dd': await p4,
    };
}

async function test5() {
    return await Promise.all([
                             await promiseWait(1000),
                             await promiseWait(5000),
                             await promiseWait(9000),
                             await promiseWait(3000),
                             ]);
}

async function test6() {
    return await Promise.all([
                             promiseWait(1000),
                             promiseWait(5000),
                             promiseWait(9000),
                             promiseWait(3000),
                             ]);
}

async function test7() {
    const p1 =  promiseWait(1000);
    const p2 =  promiseWait(5000);
    const p3 =  promiseWait(9000);
    return {
        'aa': await p1,
        'bb': await p2,
        'cc': await p3,
        'dd': await promiseWait(3000),
    };
}

let start = Date.now();

test().then((res) => {
    console.log('Test Done, elapsed', (Date.now() - start) / 1000, res);

    start = Date.now();
    test2().then((res) => {
        console.log('Test2 Done, elapsed', (Date.now() - start) / 1000, res);

        start = Date.now();
        test3().then((res) => {
            console.log('Test3 Done, elapsed', (Date.now() - start) / 1000, res);

            start = Date.now();
            test4().then((res) => {
                console.log('Test4 Done, elapsed', (Date.now() - start) / 1000, res);

                start = Date.now();
                test5().then((res) => {
                    console.log('Test5 Done, elapsed', (Date.now() - start) / 1000, res);

                    start = Date.now();
                    test6().then((res) => {
                        console.log('Test6 Done, elapsed', (Date.now() - start) / 1000, res);
                    });

                    start = Date.now();
                    test7().then((res) => {
                        console.log('Test7 Done, elapsed', (Date.now() - start) / 1000, res);
                    });
                });
            });

        });
    });

});
/*
Test Done, elapsed 18.006 [ true, true, true, true ]
Test2 Done, elapsed 18.009 { aa: true, bb: true, cc: true, dd: true }
Test3 Done, elapsed 0 { aa: Promise { <pending> },
  bb: Promise { <pending> },
  cc: Promise { <pending> },
  dd: Promise { <pending> } }
Test4 Done, elapsed 9 { aa: true, bb: true, cc: true, dd: true }
Test5 Done, elapsed 18.008 [ true, true, true, true ]
Test6 Done, elapsed 9.003 [ true, true, true, true ]
Test7 Done, elapsed 12.007 { aa: true, bb: true, cc: true, dd: true }
*/

View more solutions

Comments

[zerkms]

"generator-like syntax:" --- where is it defined?

[zerkms]

@Bergi yep, that's why I asked :-) It was rhetorical :-)

[NoNameProvided]

Clean but be aware the fast-fail behaviour of Promise.all. If any of the functions throws an error, Promise.all will reject

[JohnnyQ]

@NoNameProvided so what is the alternative to Promise.all in this case?

[NoNameProvided]

You can handle partial results nicely with async/await, see stackoverflow.com/a/42158854/2019689

[jawj]

This looks like a much nicer option to me than Promise.all — and with destructuring assignment you can even do [someResult, anotherResult] = [await someResult, await anotherResult] if you change const to let.

[Andru]

But this still executes the await statements serially, right? That is, execution pauses until the first await resolves, then moves onto the second. Promise.all executes in parallel.

[Haven]

@Andru No, it operates in parallel. I add a jsbin example, you can check there.

[CKGrafico]

I've updated the example with destructuring jsbin.com/hutaqehepo/edit?js,console

[Benja]

yay, it looks like object destructuring works too! jsbin.com/midefamodu/edit?js,console

[Quang Hoàng]

I tested it, your solution is still slow.

[Haven]

Since you declare the function outside/before the condition test, and called them. Try wrapping them in else block.

[Henok T]

This won't work. Each await is executed sequentially. I've verified it in node console.

[JohnnyHK]

This answer is misleading as the fact that both awaits are done in the same line is irrelevant. What matters is that the two async calls are made before either is awaited.

[Haven]

@JohnnyHK Actually it cannot be run in parallel while they are still in a single node application, this is same as Promise.all to pretend to run the two async call simultaneously.

[Hoang Le Anh Tu]

@Haven: I mean when you separate the moments you call functions vs await can lead to unexpected results, for example: async HTTP requests.

[Jeff Fischer]

I would add a comments to the code because there's nothing saying that the next dev is going to understand what you're doing OOB.

[Ben Winding]

@Haven this solution is not the same as Promise.all. If each request is a network call, await someResult will need to be resolved before await anotherResult is even started. Conversely, in Promise.all the two await calls can be started before either one is resolved.

[jonny]

Pro tip: use array destructuring in order to initialise an arbitrary number of results from Promise.all(), like: [result1, result2] = Promise.all([async1(), async2()]);

[Anand N]

The answer is misleading. The jsbin code appears to be executing promises in parallel, but they are not. When you create promise with new operator, the constructor is called in sync. That's the reason we see start call starts & second call starts immediately.

[Anand N]

Adding to my above comment. The output will still be the same, even if we remove the line const finalResult = [await someResult, await anotherResult]

[Lawrence Cheuk]

@Anand N It missed a return statement "resolve('ok')" inside both setTimeout function that's why it is still waiting and never return to main.

[Lawrence Cheuk]

Although the question is how to call function in parallel, this answer do give a very good example of how to await both promise to resolve before execute the remaining code. they are not started in parallel but do run concurrently. Thx

[Anand N]

@LawrenceCheuk Yes, Now it's correct :) They are not started Parallel and the Array awaits sequentially Of course the processing has started for all of them (almost at the same time), before the array

[Anand N]

Check the below link where it is clearly mentioned that call await in sequence is not replacement for promise.all developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…

[habsq]

There's a catch to this. If you put the whole statement within a try-catch block and the second promise rejects before the first one is done, the exception would be unhandled. Promise.all is much safer.

[Kaleo Brandt]

@habsq I'm not seeing that behavior... jsbin.com/hawamihixa/edit?js,console Seems to work fine.

[habsq]

@KaleoBrandt check your browser console instead of the jsbin one. This is what I see: Uncaught (in promise) Error: Whoops! at Promise.then

[Gershom Maes]

I feel like this is certainly the most pure answer

[Shekhar Kumar]

This will not make two parallel calls, instead once first await call is done, then next await is executed. Thats the normal behaviour of await, it just waits. Better way would be const [someResp, anotherResp] = await Promise.all([someResult(), anotherResult()]); It will make these two calls in parallel and then move on to next line when both responses have come.

[VDog]

As others have stated, this is not parallel calls, but sequential.

[theUtherSide]

@jonny Is this subject to fail-fast? Also, does one still need to = await Promise.all?

[jonny]

@theUtherSide You're absolutely right - I neglected to include the await.

[Szczepan Hołyszewski]

Nope, parallelization isn't happening at all here. The for loop sequentially awaits each promise and adds the result to the array.

[WSBT]

This answer is much more clear than Haven's. It's clear that the function calls will return promise objects, and await will then resolve them into actual values.

[akraines]

Tests 4 and 6 in the gist returned the expected results. See stackoverflow.com/a/42158854/5683904 by NoNameProvided who explains the difference between the options.

[Qwerty]

@ShekharKumar That is not true. results = [await firstPromise, await secondPromise, await thirdPromise]. In this example, results is assigned after all awaits have resolved and they are being processed since the moment of first execution, before they are being awaited in the array.

[Big Money]

@jonny Not to be pedantic, but you'll also need to declare the variables with either let, const, or var as in: let [result1, result2] = await Promise.all([async1(), async2()]);

[Bergi]

This appears to work at a cursory glance, but has horrible problems with unhandled rejections. Do not use this!

[Bergi]

This appears to work at a cursory glance, but has horrible problems with unhandled rejections. Do not use this!

[mikep]

Never use example like in fn asyncInArray() because in negative scenario try/catch will not catch errors from all async parallel tasks and you will end with unhandled rejections warning. The only one correct way for async parallel tasks is to use Promise.all() so asyncInPromiseAll() in this answer.

[Marc_L]

@madox2, does it make any difference if I say "return await Promise.all(...) or just "await Promise.all()"?

[madox2]

@user2774480 in the first case your function will return promise with the result of Promise.all, in second it will return promise but without any result

[G.T.]

I understand this seems not working for people. So I tested in node.js and browser. The test is passed in node.js (v10, v11), firefox, it it does not work in browser chrome. The test case is in gist.github.com/fredyang/ea736a7b8293edf7a1a25c39c7d2fbbf

[Szczepan Hołyszewski]

I refuse to believe this. There is nothing in the standard that says different iterations of a for loop can be automagically parallelized; this is not how javascript works. The way the loop code is written, it means this: "await one item (the await expr), THEN push result to temp, THEN take next item (next iteration of the for loop). The "awaiting" for each item is completely confined to a single iteration of the loop. If tests show that there is parallelization, it must be because the transpiler is doing something nonstandard or is flat out buggy.

[G.T.]

@SzczepanHołyszewski Your confidence of disbieving without running the test case inspire me to do some rename refactory and extra comments. All code are plain old ES6, no transpiling is required.

[Marc_L]

hello @madox2, why is this? I thought await will resolve (or reject) the promise automatically and then give the result of resolved back (or rejected). Or do I explicitely have to return the value, because otherwise, the value is there, but not used in any way? Can u clarify again when do I need to use "return await Promise.all(...)", when do I need to return it? when I use this Promise.all code in another piece of code? what would the return statement do?

[Vadorequest]

Note that the "Updated answer" is a good way of doing it (using Promise.all), unlike the "Original answer", so don't take @Bergi words too strongly. It's indeed the best answer here IMHO.

[sudo soul]

If you want a one-liner for Promise.all to fail gracefully, use const results = await Promise.all(promises.map(promise => promise.catch(error => error))); where promises is an array of promises. Since technically everything is resolving now, to check if something failed, iterate through results and check if item is an instanceof Error. (assuming u reject with an Error).

[Theo]

Welcome to Stack Overflow. While your code may provide the answer to the question, please add context around it so others will have some idea what it does and why it is there.

[html_programmer]

@madox2What I'm missing here is that the callback will wait until all promises have resolved, which may not be what you want. E.g. Long and short running process, maybe you don't want the short process callback to execute after the long process has finished.

[Jonathan Potter]

@Bergi You're right, thanks for pointing that out! I've updated the answer with a better solution.

[Jonathan Sudiaman]

Not sure why this is downvoted so heavily. It's essentially the same answer that @user2883596 gave.

[Matt]

bluebirds .map, .filter and .mapSeries also help with concurrency if you don't want to introduce the async modules callback api

[oldwizard]

This is dangerous, promise2 might reject before promise1 is resolved. If that happens you cannot catch the error from promise1. Either use the sequential pattern in this answer, or use Promise.all([printNumber1(), printNumber2()])

[Jin Lee]

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review

[SkarXa]

@JinLee Per your suggestion I've added the content, didn't add it originally because it looked like a lot of content

[Jin Lee]

@SkarXa SO will like your answer even more now. :) And your code is not that long. Don't worry. Thank you!

[Hart Simha]

couldn't you handle the error where the async functions are called? To me, this appears to make it easier to add a .catch to each thing individually then the Promise.all answers above

[Carmine Tambascia]

To me seems that the same concept of @asyncronous and "parallel" don't fit together completely at all. I mean as long as OP intend to run async code in parallel then this answer is correct, but not considering that the result of those promises will be anyway async, as state here "A pending Promise in all other cases. This returned promise is then resolved/rejected asynchronously (as soon as the stack is empty) when all the promises in the given iterable have resolved, or if any of the promises reject. developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/…

[Carmine Tambascia]

On top as if as soon as one of those Promises passed rejected, then the result also will reject even if all the others would resolve and return the result. For this reason is not really accurate use the term of "Parallel" intending also the result but only limitate that to when they are called