How to get data out of a Node.js http get request

Question

I'm trying to get my function to return the http get request, however, whatever I do it seems to get lost in the ?scope?. I'm quit new to Node.js so any help would be appreciated

function getData(){
  var http = require('http');
  var str = '';

  var options = {
        host: 'www.random.org',
        path: '/integers/?num=1&min=1&max=10&col=1&base=10&format=plain&rnd=new'
  };

  callback = function(response) {

        response.on('data', function (chunk) {
              str += chunk;
        });

        response.on('end', function () {
              console.log(str);
        });

        //return str;
  }

  var req = http.request(options, callback).end();

  // These just return undefined and empty
  console.log(req.data);
  console.log(str);
}

Solution 1

Of course your logs return undefined : you log before the request is done. The problem isn't scope but asynchronicity.

http.request is asynchronous, that's why it takes a callback as parameter. Do what you have to do in the callback (the one you pass to response.end):

callback = function(response) {

  response.on('data', function (chunk) {
    str += chunk;
  });

  response.on('end', function () {
    console.log(req.data);
    console.log(str);
    // your code here if you want to use the results !
  });
}

var req = http.request(options, callback).end();

Solution 2

Simple Working Example of Http request using node.

const http = require('https')

httprequest().then((data) => {
        const response = {
            statusCode: 200,
            body: JSON.stringify(data),
        };
    return response;
});
function httprequest() {
     return new Promise((resolve, reject) => {
        const options = {
            host: 'jsonplaceholder.typicode.com',
            path: '/todos',
            port: 443,
            method: 'GET'
        };
        const req = http.request(options, (res) => {
          if (res.statusCode < 200 || res.statusCode >= 300) {
                return reject(new Error('statusCode=' + res.statusCode));
            }
            var body = [];
            res.on('data', function(chunk) {
                body.push(chunk);
            });
            res.on('end', function() {
                try {
                    body = JSON.parse(Buffer.concat(body).toString());
                } catch(e) {
                    reject(e);
                }
                resolve(body);
            });
        });
        req.on('error', (e) => {
          reject(e.message);
        });
        // send the request
       req.end();
    });
}

Solution 3

Shorter example using http.get:

require('http').get('http://httpbin.org/ip', (res) => {
    res.setEncoding('utf8');
    res.on('data', function (body) {
        console.log(body);
    });
});

Solution 4

from learnyounode:

var http = require('http')  

http.get(options, function (response) {  
  response.setEncoding('utf8')  
  response.on('data', console.log)  
  response.on('error', console.error)  
})

'options' is the host/path variable

Solution 5

from learnyounode:

var http = require('http')
var bl = require('bl')

http.get(process.argv[2], function (response) {
    response.pipe(bl(function (err, data) {
        if (err)
            return console.error(err)
        data = data.toString()
        console.log(data)
    }))
})

View more solutions

Comments

[Eric]

I would recommend pushing the chunks into an array and then use join('') in the end. That will avoid issues if there is lots of data

[Hugo Koopmans]

this seems not to work as callbackData() is not defined as a function?

[Tyler Durden]

How do I get the HTTP response code of the response (200 or 404 etc.)? Is there any documentation about the keyword 'on' (response.on), 'data' and 'end'? Are these keywords? There seems to be nothing here: nodejs.org/api/http.html#http_class_http_serverresponse

[Phoca]

@TylerDurden statusCode is a property of the response object. I couldn't find proper documentation for the ServerResponse object either, just examples in the docs for the get and request methods.

[Daniel]

But this makes the code messy! Why is javascript designed like this?

[Denys Séguret]

@Daniel There are now facilities to handle the asynchronous event model: Promises and async/await.

[Matthew Mullin]

@DenysSéguret Would using a promise library (ie Axios) correctly concatenate the returned data then? So can I assume the .then with return all the data and the .catch will return a complete error if any)?

[kenshinji]

Thanks dude, I ran into the same situation like you, it sucks we can only use the https for issuing https request.

[Altieres de Matos]

Thank for share!! This was the unique sample with return data without use console.log.

[NickW]

thanks, been looking all over for a simple example and every one I found threw half a dozen new concepts at me. This just laid out how the http.get() works nice and simple. Excellent!

[dturvene]

This example is about as short as you can get and still run. This assumes a good url and small response. I prefer the http examples that chunk the data response, use the response end event, and use the request errror event.

[rags2riches-prog]

This answer is out of context given the question asked. Beside that, you are not explicitly listening for the error event which will be triggered if the connection is lost while the request is in-progress or if any other issues occur during transmission.

[Adam Cameron]

Seconded re @AltieresdeMatos's comment. This is a good, complete, practical example, that actually answers the original question as stated. This should be the accepted answer (in 2021) I reckon. Thanks pal.

[moltenform]

Warning: I think 'data' can give partial updates, the 'response' event is more practical.

[moltenform]

Warning: I think 'data' can give partial updates, the 'response' event is more practical