Http request inside a loop

Question

I'm having some troubles in making a HTTP request inside a loop.

Let me explain what I have....

I make an http GET to retrieve some values and then I need to make another HTTP GET for each couple of values that I've just taken from the first request.

The two calls are ok, and if I cut the for cycle and try to run the whole request chain it works perfectly, but just one time since I removed the cycle. How can I make it work?

Here's the code:

request({
url: "some_url",
    method: "GET",
    json:true,
    headers:[{'content-type': 'application/json'}]
    }, function (error, response, body){
            if(!error & response.statusCode === 200){           
                for(var i=0;i<body.length;i++){
                    for(var j=i;j<body.length-1;j++){
                        //if(i === body.length-1) return;
                        src = body[i].name;
                        dest = body[j+1].name;
                        console.log("sorgente ",sorg);

                        request({
                            url: "https://maps.googleapis.com/maps/api/distancematrix/json?origins="+src+"&destinations="+dest,
                            method: "POST",
                            json:true,
                            headers:[{'content-type': 'application/json'}]
                        }, function (error, response, body){
                            if(!error & response.statusCode === 200){
                                console.log("TIME ",body.rows[0].elements[0].duration.text);
                                return;
                            }else{
                                console.log("google API failed!: ");
                                return;
                            }
                        }); 

                    }
                }

            }else{
                console.log("/google_api failed!: ");
                return;
            }
 });

I hope I was clear with the question.

Solution

The issue here is one of Javascript scoping, and typos.

First, you hardcoded the array indexes of body[0] and body[1]. It looks like you meant for them to be the loop variables.

Second, an outline of your scoping problem, in simplified pseudo-Javascript:

var requestList = [...];

for(var i = 0; i < requestList.length; i++){
    var current = requestList[i];

    // make a request based on current or the data
    // in current. 
    request(..., function(result){
        // do something with the current variable
        current.foo = result.bar;
    });
}

All web requests are asynchronous. There used to be a way to force them to run synchronously, but it is deprecated in most major browsers. This means that the request is made, and may get a response, outside of your actual code, and then calls some sort of callback--in this case, the anonymous inner function function(result){...}.

What this means is that that for loop continues to execute and loop while the request is being made, meaning if the request isn't fast enough, current will update and be different when the request comes back, and so will the for loop variables.

---

The solution I've run into for this sort of thing is function scoping out the inner request in the for loop.

Instead of making the new request directly inside the for loop, you move that out to its own function:

var requestList = [...];

for(var i = 0; i < requestList.length; i++){
    var current = requestList[i];

    GetMyResourceData(current);
}

function GetMyResourceData(current){
    request(..., function(result){
        // do something with the current variable
        current.foo = result.bar;
    });
}

Every time the GetMyResourceData function is called, a new scope is created for that function, so the current variable in that function is held when you reach the callback.

So, that's what I'd recommend you do for your code. Move the second request outside of the for loop into its own scope.

Comments

[Mirko]

Yes it was an error of copy and paste,obviously the variables are body[i] and body[j+1],I edit in the question,thanks :)

[Mirko]

I'v tried as you suggested but it seems like it does not see the for loop,the program execute the line just before the for and then it stops. I can't figure out why.

[Mirko]

Edit: Solved! For some unknown reason I wrote bad the for loop,the incremental part was as second argument instead of the last one.I feel stupid right now,thanks for the help!

[AbstractSage]

I find caolan's async module very useful while invoking lots of async functions in parallel

[Aeolingamenfel]

Just saw your comments, glad to know I could help! Let me know if you have any other questions.

[jdstein1]

Great answer! Worked for me as well!

[Pierrick Martellière]

Nice, saved me hours of search.