How to set custom user-agent for Mechanize in Rails

Question

I know you have a set of pre-defined aliases you can use by setting agent.user_agent_alias = 'Linux Mozilla' for instance, but what if I want to set my own user agent, as I'm writing a web crawler and want to identify it, for the sites I'm indexing's sake. Just like Googlebot.

There seems to be a user_agent method, but I can't seem to find any documentation about it's function.

Solution 1

You can set the user agent from an alias

a = Mechanize.new
a.user_agent_alias = 'Mac Safari'

Available aliases are stored in the AGENT_ALIASES constant.

p Mechanize::AGENT_ALIASES

Otherwise, use #user_agent to set your custom user agent.

a = Mechanize.new
a.user_agent = 'Custom agent'

Solution 2

I was looking for a way to set a random user agent for Mechanize, so I ended up doing this:

a = Mechanize.new do |agent|
  agent.user_agent_alias = (Mechanize::AGENT_ALIASES.keys - ['Mechanize']).sample
end

Solution 3

As replied by @Arkhitech, now you would want to do:

  agt = Mechanize.new 
  agt.agent.user_agent = Mechanize::AGENT_ALIASES[(Mechanize::AGENT_ALIASES.keys - ['Mechanize']).sample] 

user_agent_alias is not available any more.