asynchronous http request in ruby

Question

 require 'net/http'

urls = [
  {'link' => 'http://www.google.com/'},
  {'link' => 'http://www.facebook.com/'},
 {'link' => 'http://www.yahoo.com/'}
]

urls.each do |u|
  u['content'] = Net::HTTP.get( URI.parse(u['link']) )
end

print urls

This will work as procedural code.. I just want to hit a server, no issues about the order. How can i do that in ruby. One option is using threads.

Here's an example using threads.

require 'net/http'

urls = [
  {'link' => 'http://www.google.com/'},
  {'link' => 'http://www.facebook.com/'},
  {'link' => 'http://www.yahoo.com/'}
]

urls.each do |u|
  Thread.new do
    u['content'] = Net::HTTP.get( URI.parse(u['link']) )
    puts "Successfully requested #{u['link']}"

    if urls.all? {|u| u.has_key?("content") }
      puts "Fetched all urls!"
      exit
    end
  end
end

Any better solution..??

PS:- i want to hit mixpanel, so that's why I just want to make a http call and dont wait for the response.

Solution 1

Lightweight Async handling is the job of Threads (as you said) or Fibers.

Otherwise, you should consider EventMachine which is a very powerful tool.

EDIT: The above URL for Event Machine is dead. Here is their GitHub account, https://github.com/eventmachine/eventmachine . It serves as a good starting point.

Solution 2

Here is a great article covering the topic.

Generally, viable alternatives to using threads for this would be the use of a Fiber or you could use em-http-request. In the latter example you could leave out the callback handling for your particular purpose.

Solution 3

If its just about plain http requests in async style, probably Unirest is the best fit to achieve it.

Asnc request is as simple as:

response = Unirest.post "http://httpbin.org/post", 
                    headers:{ "Accept" => "application/json" }, 
                    parameters:{ :age => 23, :foo => "bar" } {|response|
response.code # Status code
response.headers # Response headers
response.body # Parsed body
response.raw_body # Unparsed body
}