using async correctly (with forEach)

12,119

First, there is no async.forEach, but there is the async.each function. The async.parallel function will execute all tasks simoultaneously and then execute the callback if it was defined.

In your source, the callback function will be executed as soon as the async.parallel is executed, not after all of it's functions have returned. I suggest you read the documentation.

If you want to execute the callback after all the parallel functions' callbacks have been called, you should pass the forEach's callback to the async.parallel function as a second parameter, after the array of functions. Also, you should probably pass the callbacks to both databaseCallA and databaseCallB thus making them call their callbacks when they finish so they aren't executed prematurely.

async.each(array, function(elem, callback) {
    async.parallel([
        function(cb) {
            databaseCallA(cb);
            // if you have called the cb here, the `async.parallel` function would "think" that the databaseCallA has finished, even though it may not have started yet.
        },
        function(cb) {
            databaseCallB(cb):
        } 
    ], callback);
});

You should modify your database call functions to accept the callback argument and make them call the callback after they have done their work.

The point of async calls is to free the resources for other jobs while waiting for your job to finish - your code will continue it's execution, but you can use the callbacks or some kind of event-based notifications to notify others your async job has finished.

EDIT

To execute something after all parallel calls have finished:

async.each(array, function(elem, callback) {
    async.parallel([
        function(cb) {
            // I use the setTimeout function to intentionally
            // delay the response
            setTimeout(function() {
                // the first cb parameter is error, the second is result 
                // in case there was no error
                cb(undefined, 'a');
            }, 500);
        },
        function(cb) {
            setTimeout(function() {
                cb(undefined, 'b');
            }, 300);
        },
        function(cb) {
            setTimeout(function() {
                cb(undefined, 'c');
            }, 800);
        }
    ], function(err, results) {
        // this will be executed only after all three cb's are executed. In
        // this case, after about 800ms. The results variable will be an 
        // array containing the resuts passed to each cb. In this case it
        // will  be ['a', 'b', 'c'].

        // you could call the main callback here
        callback();
    });
}, function(err) {
    // this callback will be executed either after everything was done,
    // or if an error has occurred.
    if (err) {
        handleError(err);
        return;
    }

    // this should be executed after all `each` and `parallel` calls
    // have finished. In this case, it should also be after about 800ms
    // from the start, since everything was executed in parallel
    console.log('finished');

    // now you are sure everything was done, do something afterwards
    after();
});

// this will be executed immediately so don't put anything here
// that depends on the outcome of the async calls above
console.log('test');
Share:
12,119
gadu
Author by

gadu

I code. And math. Hard

Updated on June 04, 2022

Comments

  • gadu
    gadu over 1 year

    So I have a nice chunk of nested async code running together and it all seems to be okay, except for when I get to the end of it. my last function in a series block is a forEach: that then goes into a async.parallel

    Managed to track down the following chunk which does not run in order:

    async.forEach(array, function(elem, callback) {
          async.parallel([
              function(callback) {
                  database-call-A(elem, unction(err, data)  {
                        if(err){
                            console.log("error on first parallel");
                            callback({err: false}); // will break out
                        } else {
                            elem.c = data;
                            callback();
                        }
                  });
              },
              function(callback) {
                   database-call-B(elem, function(err, data)  {
                        if(err){
                            console.log("error on first parallel");
                            callback({err: false}); // will break out
                        } else {
                            elem.c = data;
                            callback();
                        }
                  });
              } 
           ]); // end async.parallel 
    
           // if forEach needs a callback after every iteration (which I think it does?)
           console.log("PRINTS N TIMES - ONCE FOR EVERY ITERATION");
           callback(); // both parallel functions have run, call back forEach
    
     }); // end forEach
    
     console.log("Donions - prints when finished");
    

    When I tested this code by throwing print statements everywhere, I noticed that "PRINTS N TIMES ..." ran N times, then I got "Donions .." and THEN my do something(); and other stuff(); started getting called in my async.parallel.

    Why is my forEach iterating through without waiting for the callbacks from async.parallel?