setTimeout in Node.js loop

Question

I'm a bit confused as to how setTimeout works. I'm trying to have a setTimeout in a loop, so that the loop iterations are, say, 1s apart. Each loop iteration makes an HTTP request and it seems like the server on the other end can't handle that many requests in such a short time span.

for (var i = 1; i<=2000 && ok; i++) {
    var options = {
        host:'www.host.com',
        path:'/path/'+i
    };

    setTimeout(makeRequest(options, i), 1000);
};

Why does this not work and how can I achieve this?

Thank you

Solution 1

You need something like this

var counter = 5;

function makeRequst(options, i) {
    // do your request here
}

function myFunction() {
    alert(counter);

    // create options object here
    //var options = {
    //    host:'www.host.com',
    //    path:'/path/'+counter
    //};
    //makeRequest(options, counter);

    counter--;
    if (counter > 0) {
        setTimeout(myFunction, 1000);    
    }
}

See also this fiddle

At the point of the alert(count); you can do your call to the server. Note that the counter works opposite (counting down). I updated with some comments where to do your thing.

Solution 2

setTimeout is non blocking, it is asynchronous. You give it a callback and when the delay is over, your callback is called.

Here are some implementations:

Using recursion

You can use a recursive call in the setTimeout callback.

function waitAndDo(times) {
  if(times < 1) {
    return;
  }

  setTimeout(function() {

    // Do something here
    console.log('Doing a request');

    waitAndDo(times-1);
  }, 1000);
}

Here is how to use your function:

waitAndDo(2000); // Do it 2000 times

About stack overflow errors: setTimeout clear the call stack (see this question) so you don't have to worry about stack overflow on setTimeout recursive calls.

Using generators (io.js, ES6)

If you are already using io.js (the "next" Node.js that uses ES6) you can solve your problem without recursion with an elegant solution:

function* waitAndDo(times) {
  for(var i=0; i<times; i++) {

    // Sleep
    yield function(callback) {
      setTimeout(callback, 1000);
    }    

    // Do something here
    console.log('Doing a request');
  }
}

Here is how to use your function (with co):

var co = require('co');

co(function* () {
  yield waitAndDo(10);
});

BTW: This is really using a loop ;)

Generator functions documentation.

Solution 3

Right now you're scheduling all of your requests to happen at the same time, just a second after the script runs. You'll need to do something like the following:

var numRequests = 2000,
    cur = 1;

function scheduleRequest() {
    if (cur > numRequests) return;

    makeRequest({
        host: 'www.host.com',
        path: '/path/' + cur
    }, cur);

    cur++;
    setTimeout(scheduleRequest, 1000)
}

Note that each subsequent request is only scheduled after the current one completes.

Solution 4

I might be late at the party but here is another (more readable) solution without the need to omit for loop.

What your code does is creating 2000 (actually 1999) setTimeout objects that will call the makeRequest function after 1 second from now. See, none of them knows about the existence of the other setTimeouts.

If you want them 1 sec apart from each other, you are responsible for creating them so.

This can be achieve by using your counter (in this case i) and the timeout delay.

for (var i = 1; i<=2000 && ok; i++) {
    var options = {
        host:'www.host.com',
        path:'/path/'+i
    };

    setTimeout(makeRequest(options, i), i * 1000); //Note i * 1000
};

The first timeout object will be set for 1 second from now and the second one will be set for 2 seconds from now and so on; Meaning 1 second apart from each other.

Solution 5

I'm surprised that no one has mentioned this above, but it sounds like you need setInterval not setTimeout.

vat poller = setInterval(makeRequestFunc, 3000)

The code above will make a request every 3 seconds. Since you saved the object to the variable poller, you can stop polling by clearing the object like so:

cleanInterval(poller)

View more solutions

Comments

[bart s]

Welcome. If your request to the server is not time critical, you even better use a construction where a new timer is only started after the previous server call has finished. Then only one server call will be active even if the call takes more than 1 second (the setTimeout timer value)

[glasspill]

that's what I thought as well

[Hard Tacos]

@jmar777 I realize that this is an old post, however I am having a similar issue. Do you mind answering why the scheduleRequest function doesn't pass cur or numRequests?

[Yves M.]

WARNIG: Your solution can lead to a RangeError: Maximum call stack size exceeded. Please see this solution that uses setImmediate. BTW OP is using Node.js.. (there is no alert in Node.js).

[Mike Tunnicliffe]

@Yves M. : The setTimeout() should be sufficient. There is no recursive call to myFunction in this solution.

[bart s]

If I look at this API document the note tells me that setImmediate might not become standard, althoug it existis in node.js

[Yves M.]

@MikeTunnicliffe myFunction calls setTimeout which calls myFunction again.. and so on. This is called recursion. Try your code with 30,000 times (and 1ms to be quick)

[Mike Tunnicliffe]

@Yves M : myFunction passes a reference to itself (a named function) to setTimeout, it does not call itself. If the invocation were setTimeout(myFunction(), 1000); you would be correct.

[Yves M.]

@barts This is a Node.js question... setImmediate exists, the documentation is here. If you want to use it browser side, you can still use this implementation of setImmediate.

[Yves M.]

@MikeTunnicliffe It doesn't matter how long is the snake.. if the snake eats it's own tail, it's recursion :) Please test your code (display the call stack at each myFunction call and you will se the call stack growing)

[Yves M.]

@MikeTunnicliffe Of course setTimeout calls the function you give to it. Otherwise it's complete non sense. From the Mozilla doc: Calls a function or executes a code snippet after a specified delay

[Mike Tunnicliffe]

@Yves M.: I agree, setTimeout calls the function you give it, but in Node it does so on a different iteration of the event loop and so myFunction doesn't in fact call itself. For example, doing a straight recursive call fails for me after 20 929 calls. Using setTimeout it's cleared 500 000 calls with no sign of failing.

[Yves M.]

@MikeTunnicliffe Yes sorry my bad :D setTimeout does clear the call stack.. I'm sorry. Read this: stackoverflow.com/questions/8058612/… :D Your code does not stack overflow... :)

[bart s]

Nice discussion here. Please leave the comments here for future reference. other people might benefit from it