NodeJS - How to get cookies from server response

Question

I want to use nodeJS as tool for website scrapping. I have already implemented a script which logs me in on the system and parse some data from the page.

The steps are defined like:

1. Open login page

2. Enter login data

3. Submit login form

4. Go to desired page

5. Grab and parse values from the page

6. Save data to file

7. Exit

Obviously, the problem is that every time my script has to login, and I want to eliminate that. I want to implement some kind of cookie management system, where I can save cookies to .txt file, and then during next request I can load cookies from file and send it in request headers.

This kind of cookie management system is not hard to implement, but the problem is how to access cookies in nodejs? The only way I found it is using request response object, where you can use something like this:

 request.get({headers:requestHeaders,uri: user.getLoginUrl(),followRedirect: true,jar:jar,maxRedirects: 10,},function(err, res, body) {
        if(err) {
            console.log('GET request failed here is error');
            console.log(res);
        }

        //Get cookies from response
        var responseCookies = res.headers['set-cookie'];
        var requestCookies='';
        for(var i=0; i<responseCookies.length; i++){
            var oneCookie = responseCookies[i];
            oneCookie = oneCookie.split(';');
            requestCookies= requestCookies + oneCookie[0]+';';
        }
    }
);

Now content of variable requestCookies can be saved to the .txt file and can loaded next time when script is executed, and this way you can avoid process of logging in user every time when script is executed.

Is this the right way, or there is a method which returns cookies?

NOTE: If you want to setup your request object to automatically resend received cookies on every subsequent request, use the following line during object creation:

var request = require("request");
request = request.defaults({jar: true});//Send cookies on every subsequent requests

Solution

In my case, i've used 'http'library like the following:

http.get(url, function(response) {
    variable = response.headers['set-cookie'];
})

Comments

[Pogrindis]

why not store the cookie into a reference var within nodeJS ? If you want to save it to file, you can via nodeJS fs module, even localstorage?

[MrD]

It is not problem to save the cookies (file, local storage etc...) the questio is "is this good approach to get cookies from server"

[Pogrindis]

I don't see why not, you have the cookie, you're not re-requesting it.. You're saving it for use later.. What could be the problem ?

[MrD]

Then OK, if there is no construction like response.getCookies() or similar, than this is perfect.

[Pogrindis]

I might be wrong, but as far as i know there is not built in one function to retrieve cookies, you've written as good as anything else!

[MrD]

Then this is perfect, thx!

[Sid]

I am struggling with almost the same problem. I have extracted the cookie, but when I want to use it outside the current request-response cycle, node says cookie is not defined. Plus, how do I use that cookie on subsequent requests if I am using the baked in https module?