How to write a node.js function that waits for an event to fire before 'returning'?

Question

I have a node application that is not a web application - it completes a series of asynchronous tasks before returning 1. Immediately before returning, the results of the program are printed to the console.

How do I make sure all the asynchronous work is completed before returning? I was able to achieve something similar to this in a web application by making sure all tasks we completed before calling res.end(), but I haven't any equivalent for a final 'event' to call before letting a script return.

See below for my (broken) function currently, attempting to wait until callStack is empty. I just discovered that this is a kind of nonsensical approach because node waits for processHub to complete before entering any of the asynchronous functions called in processObjWithRef.

function processHub(hubFileContents){
    var callStack = [];
    var myNewObj = {};
    processObjWithRef(samplePayload, myNewObj, callStack);
    while(callStack.length>0){
        //do nothing
    }
    return 1
}

Note: I have tried many times previously to achieve this kind of behavior with libraries like async (see my related question at How can I make this call to request in nodejs synchronous?) so please take the answer and comments there into account before suggesting any answers based on 'just use asynch'.

Solution 1

The problem is with your design of the function. You want to return a synchronous result from a list of tasks that are executed asynchronously.

You should implement your function with an extra parameter that will be the callback where you would put the result (in this case, 1) for some consumer to do something with it.

Also you need to have a callback parameter in your inner function, otherwise you won't know when it ends. If this last thing is not possible, then you should do some kind of polling (using setInterval perhaps) to test when the callStack array is populated.

Remember, in Javascript you should never ever do a busy wait. That will lock your program entirely as it runs on a single process.

Solution 2

You cannot wait for an asynchronous event before returning--that's the definition of asynchronous! Trying to force Node into this programming style will only cause you pain. A naive example would be to check periodically to see if callstack is empty.

var callstack = [...];

function processHub(contents) {
  doSomethingAsync(..., callstack);
}

// check every second to see if callstack is empty
var interval = setInterval(function() {
  if (callstack.length == 0) {
    clearInterval(interval);
    doSomething()
  }
}, 1000);

Instead, the usual way to do async stuff in Node is to implement a callback to your function.

function processHub(hubFileContents, callback){
  var callStack = [];
  var myNewObj = {};
  processObjWithRef(samplePayload, myNewObj, callStack, function() {
    if (callStack.length == 0) {
      callback(some_results);
    }
  });
}

If you really want to return something, check out promises; they are guaranteed to emit an event either immediately or at some point in the future when they are resolved.

function processHub(hubFileContents){
  var callStack = [];
  var myNewObj = {};
  var promise = new Promise();

  // assuming processObjWithRef takes a callback
  processObjWithRef(samplePayload, myNewObj, callStack, function() {
    if (callStack.length == 0) {
      promise.resolve(some_results);
    }
  });

  return promise;
}

processHubPromise = processHub(...);
processHubPromise.then(function(result) {
  // do something with 'result' when complete
});

Solution 3

deasync is desinged to address your problem exactly. Just replace

while(callStack.length>0){
    //do nothing
}

with

require('deasync').loopWhile(function(){return callStack.length>0;});

Solution 4

The problem is that node.js is single-threaded, which means that if one function runs, nothing else runs (event-loop) until that function has returned. So you can not block a function to make it return after async stuff is done.

You could, for example, set up a counter variable that counts started async tasks and decrement that counter using a callback function (that gets called after the task has finished) from your async code.

Solution 5

Node.js runs on A SINGLE threaded event loop and leverages asynchronous calls for doing various things, like I/O operations.

if you need to wait for a number of asynchronous operations to finish before executing additional code you can try using Async -

Node.js Async Tutorial

View more solutions