How can I make this call to request in nodejs synchronous?

Question

I have a function in my nodejs application called get_source_at. It takes a uri as an argument and its purpose is to return the source code from that uri. My problem is that I don't know how to make the function synchronously call request, rather than giving it that callback function. I want control flow to halt for the few seconds it takes to load the uri. How can I achieve this?

function get_source_at(uri){
    var source;
    request({ uri:uri}, function (error, response, body) {
        console.log(body);
    });
    return source;
}

Also, I've read about 'events' and how node is 'evented' and I should respect that in writing my code. I'm happy to do that, but I have to have a way to make sure I have the source code from a uri before continuing the control flow of my application - so if that's not by making the function synchronous, how can it be done?

Solution 1

You should avoid synchronous requests. If you want something like synchronous control flow, you can use async.

async.waterfall([
    function(callback){
        data = get_source_at(uri);
        callback(null, data);
    },
    function(data,callback){
        process(data, callback);
    },
], function (err,result) {
    console.log(result)
});

The process is promised to be run after get_source_at returns.

Solution 2

You can with deasync:

function get_source_at(uri){
    var source;
    request({ uri:uri}, function (error, response, body) {
        source = body;
        console.log(body);
    });
    while(source === undefined) {
      require('deasync').runLoopOnce();
    }
    return source;
}

Solution 3

This is better way of using deasync.

var request = require("request")
var deasync = require("deasync")

var getHtml = deasync(function (url, cb) {
   var userAgent = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36"}
   request({
      url: url, 
      headers: userAgent
   },
   function (err, resp, body) {
      if (err) { cb(err, null) }
      cb(null, body)
   })
})

var title = /<title>(.*?)<\/title>/

var myTitle = getHtml("http://www.yahoo.com").match(title)[1]
console.log(myTitle)

Please refer to documentation of deasync, you will find that you can use
desync(function (n params, cb) {})
to make the function where cb should come back with (err, data). So fs.readFile() like functions can be easily wrapped with deasync function. But for functions like request which don't come back with cb(err, data). You can make you own function (named or anonymous) with a custom cb(err, data) callback format just as I have done in the above code. This way you can force almost any async function perform like sync by waiting for callback cb(err, data) to come back on a different javascript layer (as the documentation says). Also make sure that you have covered all ways to get out from the function which you are wrapping with deasync with cb(err, data) callbacks, otherwise your program will block.

Hope, it helps someone out there!

Update:
Don't use this way of doing synchronous requests. Use Async/Await for writting promises based synchronous looking code. You can use request-promise-native npm module to avoid wrapping requests module with promises yourself.

Solution 4

Having a simple blocking function is a great boon for interactive development! The sync function (defined below) can synchronize any promise, cutting down dramatically on the amount of syntax needed to play with an API and learn it. For example, here's how to use it with the puppeteer library for headless Chrome:

var browser = sync(puppeteer.connect({ browserWSEndpoint: "ws://some-endpoint"}));
var pages = sync(browser.pages())
pages.length
1
var page = pages[0]
sync(page.goto('https://duckduckgo.com', {waitUntil: 'networkidle2'}))
sync(page.pdf({path: 'webpage.pdf', format: 'A4'}))

The best part is, each one of these lines can be tweaked until it does what you want, without having to re-run or re-type all of the previous lines each time you want to test it. This works because you have direct access to the browser and pages variables from the top-level.

Here's how it works:

const deasync = require("deasync");
const sync = deasync((promise, callback) => promise.then(result) => callback(null, result)));

It uses the deasync package mentioned in other answers. deasync creates a partial application to the anonymous function, which adds callback as the last argument, and blocks until callback has been called. callback receives the error condition as its first argument (if any), and the result as its second (if any).

Solution 5

I have to have a way to make sure I have the source code from a uri before continuing the control flow of my application - so if that's not by making the function synchronous, how can it be done?

Given this entry point to your application:

function app(body) {
    // Doing lots of rad stuff
}

You kick it off by fetching the body:

request({ uri: uri }, function (error, response, body) {
    if(err) return console.error(err);

    // Start application
    app(body);
}

This is something you will have to get used to when programming for node.js (and javascript in general). There are control flow modules like async (which I, too, recommend) but you have to get used to continuation passing style, as it's called.

View more solutions

Comments

[Vikas Gautam]

Please make us enlighten with your hidden knowledge. Express your reason for downvote in the comment. thanks!

[Shad]

This is good solution. deasync is much better and elegant way of doing sync call.

[aroth]

"You should avoid synchronous requests" is not an accurate blanket statement. You should avoid synchronous requests performed on an application's main thread or otherwise in situations that will cause your UI to block. But in the case of say, a headless webservice whose only purpose is to delegate to some other webservice and report back with the result? There a synchronous request is perfectly fine and in fact preferable/cleaner to an asynchronous one. It's important to understand your use-case, and choose the solution that best fits. It's not a case of always one and never the other.

[JustAMartin]

Agree with @aroth - I'm using nodejs scripts for custom complex deployment tasks where I need every command to be executed in strict sequence. I could use promises, but even then() chaining can get somewhat ugly if you have more than 10 tasks to run in strict sequence.

[Bryan Grace]

That's not an answer to this question.

[S Balough]

This is helpful for earlier node versions that don't support async/await.

[wolfsbane]

How do you use the "data" in the next callback? I tried but it not displaying anything.