Maintaining cookies between Mechanize requests

Question

I'm trying to use the Ruby version of Mechanize to extract my employer's tickets from a ticket management system that we're moving away from that does not supply an API.

Problem is, it seems Mechanize isn't keeping the cookies between the post call and the get call shown below:

require 'rubygems'
require 'nokogiri'
require 'mechanize'

@agent = Mechanize.new

page = @agent.post('http://<url>.com/user_session', {
                                            'authenticity_token' => '<token>',
                                            'user_session[login]' => '<login>',
                                            'user_session[password]' => '<password>',
                                            'user_session[remember_me]' => '0',
                                            'commit' => 'Login'
})

page = @agent.get 'http://<url>.com/<organization>/<repo-name>/tickets/1'
puts page.title

user_session is the URL to which the site's login page POSTs, and I've verified that this indeed logs me in. But the page that returns from the get call is the 'Oops, you're not logged in!' page.

I've verified that clicking links on the page that returns from the post call works, but I can't actually get to where I need to go without JavaScript. And of course I've done this successfully on the browser with the same login.

What am I doing wrong?

Solution 1

Okay this might help you - first of all what version of mechanize are you using? You need to identify, if this problem is due to the cookies being overwritten/cleaned by mechanize between the requests or if the cookies are wrong/not being set in the first place. You can do that by adding a puts @agent.cookie_jar.jar inbetween the two requests, to see what is stored.

If its a overwriting issue, you might be able to solve it by collecting the cookies from the first request, and applying them to the second. There are many ways to do this:

One way is to just do a temp_jar = agent.cookie_jar.jar an then just going through each cookie and add it again using the .add method

HOWEVER - the easiest way is by just installing the latest 2.1 pre release of mechanize (many fixes), because you will then be able to do it very simply. To install the latest do a gem install mechanize --pre and make sure to get rid of the old version of mechanize gem uninstall mechanize 'some_version' after this, you can simply do as follows:

require 'rubygems'
require 'nokogiri'
require 'mechanize'

@agent = Mechanize.new

page = @agent.post('http://<url>.com/user_session', {
                                        'authenticity_token' => '<token>',
                                        'user_session[login]' => '<login>',
                                        'user_session[password]' => '<password>',
                                        'user_session[remember_me]' => '0',
                                        'commit' => 'Login'
})
temp_jar = @agent.cookie_jar
#Do whatever you need an use the cookies again in a new session after that
@agent = Mechanize.new
@agent.cookie_jar = temp_jar

page = @agent.get 'http://<url>.com/<organization>/<repo-name>/tickets/1'
puts page.title

BTW the documentation is here http://mechanize.rubyforge.org/index.html

Solution 2

Mechanize would automatically send cookies obtained from the response in the consecutive request. You can use the same agent without re-new.

require 'mechanize'

@agent = Mechanize.new
@agent.post(create_sessions_url, params, headers)
@agent.get(ticket_url)

Tested with mechanize 2.7.6.

Comments

[Ditmar Wendt]

this is pretty bad, why is a new mechanize instance needed?

[Yakob Ubaidi]

@DitmarWendt this is just an example how to re-use the cookies obtained earlier after login post executed.