Asynchronous HTTP Client for Java

Question

As a relative newbie in the Java world, I am finding many things frustratingly obtuse to accomplish that are relatively trivial in many other frameworks. A primary example is a simple solution for asynchronous http requests. Seeing as one doesn't seem to already exist, what is the best approach? Creating my own threads using a blocking type lib like httpclient or the built-in java http stuff, or should I use the newer non-blocking io java stuff - it seems particularly complex for something which should be simple.

What I am looking for is something easy to use from a developer point of view - something similar to URLLoader in AS3 - where you simply create a URLRequest - attach a bunch of event handlers to handle the completion, errors, progress, etc, and call a method to fire it off.

If you are not familiar with URLLoader in AS3, its so super easy and looks something like this:

private void getURL(String url)
{
    URLLoader loader = new URLLoader();
    loader.addEventListener(Event.Complete, completeHandler);
    loader.addEventListener(HTTPStatusEvent.HTTP_STATUS, httpStatusHandler);
    loader.addEventListener(IOErrorEvent.IO_ERROR, ioErrorHandler);

    URLRequest request = new URLRequest(url);

    // fire it off - this is asynchronous so we handle
    // completion with event handlers
    loader.load(request);
}

private void completeHandler(Event event)
{
    URLLoader loader = (URLLoader)event.target;
    Object results = loader.data;

    // process results
}

private void httpStatusHandler(Event event)
{
    // check status code
}

private void ioErrorHandler(Event event)
{
    // handle errors
}

Solution 1

You have several choices for Async HTTP Clients in Java

1. Java 8: Use the async-http-client formerly called ning http client library.
2. Java 11 and above: JDK now comes with the java.net.http. HttpClient which is fully asynchronous.
3. Square's OkHttpClient. Supports both sync blocking and async calls with callbacks. Quite popular on Android.

Solution 2

Version 4.0 of Apache Commons HttpClient (now in HttpComponents/HttpCore) also support Java's NIO (non-blocking IO). I think this is your best bet.

Solution 3

The Jetty HTTP client is asynchronous.

Solution 4

If you haven't looked at it already, check out the Java 5 java.util.concurrent -- it makes multi-threaded apps much easier to develop. You can set up a ThreadPoolExecutor that manages, say, four Threads. You then feed the pool any number of tasks to complete. Each task is a Runnable. The ThreadPoolExecutor will queue up the Runnable tasks and feed them to available Threads in parallel. The Pool's afterExecute() method is called when each Runnable task completes.

I vividly remember writing a fetch thread pool for a web browser written in Java back in 1999, and it was a bear to get right. Last month I wrote a load tester for a web server. The tester has a ThreadPoolExecutor that has n threads, and the Runnable tasks I feed it each fetch a page using Apache HTTP Client. It took just an hour or two to get it working reasonably well. I think you'll like java.util.concurrent coupled with Apache HTTP Client, though it sounds like you'll need to do some customization for progress indication.

(Note that Apache HTTP Client does its own thread pooling, and the default configuration limits you to 20 threads max, and only two to each web server.)

Update: Here's the link to Apache HTTP Client. Be sure to read up on MultiThreadedHttpConnectionManager, it's what handles the connection pool, and it's not shown in the most basic example.

Solution 5

Looks like you want (a part of) NIO -- there's a good tutorial here, the asynchronous networking part starts at p. 30 and there are many useful links at the end.

View more solutions

Comments

[helifreak]

Totally understand the different niches. However, in this instance, we are using Java as a client UI framework and a lot of the data to be presented is coming from external web services.

[helifreak]

Well, using other languages that compile to java bytecode is a future option, but we definitely need something Java based as well. Thanks for the suggestions.

[cwash]

I was going to mention this, but neither of these are asynch. They do make pretty quick work of automation tasks, though.

[helifreak]

Good information. Thanks. I still can't believe how client unfriendly Java is. Maybe Java is just starting to show its age.

[Jim Ferrans]

I did almost no development in Java during the 1.4 and Java 5 years, and now I find Java 6 significantly better than 1.3. In addition to java.util.concurrent, generics and the Collections framework stand out. Technology layered on Java can seem unfriendly though: compare JAXP with Groovy's concise XML processing. Also. dynamic languages like Ruby, Python, and Scala are serious competition these days.

[StaxMan]

And it's at github at github.com/ning/async-http-client. I can recommend this, having used it, although mostly just as a simple reliable synchronous http client. But with simple Future-based results, async operation is simple to use too.

[StaxMan]

No, it's not that Java is showing age; check out multiple non-JDK-bundled non-blocking (asynchronous) http clients. No need to explicitly mess with threads just to do concurrent accesses. While it would be convenient to have non-blocking alternative in JDK there are so many useful things that it is probably better to try to keep JDK from ballooning any bigger.

[StaxMan]

Actually there are multiple async http clients, listed in other answers (jetty has had one for years, jakarta hc quite some time too; ning's async http client is newest and actively developed). They just haven't gotten enough attention they deserve.

[jfager]

They've changed GitHub repos, they're now at github.com/AsyncHttpClient/async-http-client

[Mike Deck]

They actually have changed GitHub repos again and now are at github.com/sonatype/async-http-client. I've updated the answer to reflect the correct repo location now.

[trillions]

even if i fired separated threads, i still feel the threads are blocking and not as fast as i expected...