Using nodejs async and request module

Question

I'm trying to use async and request module together but i don't understand how the callbacks get passed. My code is

var fetch = function(file, cb) {
    return request(file, cb);
};

async.map(['file1', 'file2', 'file3'], fetch, function(err, resp, body) {
    // is this function passed as an argument to _fetch_ 
    // or is it excecuted as a callback at the end of all the request?
    // if so how do i pass a callback to the _fetch_ function
    if(!err) console.log(body);
});

I'm trying to fetch 3 files in order and concatenate the results. My head is stuck in callbacks I tryed and the different combinations I could think of. Google wasn't much help.

Solution 1

Request is asynchronous function, it does not return something, when its job is done, it calls back. From request examples, you should do something like:

var fetch = function(file,cb){
     request.get(file, function(err,response,body){
           if ( err){
                 cb(err);
           } else {
                 cb(null, body); // First param indicates error, null=> no error
           }
     });
}
async.map(["file1", "file2", "file3"], fetch, function(err, results){
    if ( err){
       // either file1, file2 or file3 has raised an error, so you should not use results and handle the error
    } else {
       // results[0] -> "file1" body
       // results[1] -> "file2" body
       // results[2] -> "file3" body
    }
});

Solution 2

In your example, the fetch function will be called three times, once for each of the file names in the array passed as the first parameter to async.map. A second, callback parameter will also be passed into fetch, but that callback is provided by the async framework and you must call it when your fetch function has completed its work, providing its results to that callback as the second parameter. The callback you provide as the third parameter to async.map will be called when all three of the fetch calls have called the callback provided to them.

See https://github.com/caolan/async#map

So to answer your specific question in the code, the callback function you provide is executed as a callback at then end of all requests. If you need to pass a callback to fetch you'd do something like this:

async.map([['file1', 'file2', 'file3'], function(value, callback) {
    fetch(value, <your result processing callback goes here>);
}, ...

Comments

[andrei]

code working and very easy to understand what I did wrong now :) thanks

[Catfish]

Your link to the examples does not show any callbacks. All they do is log to the console.