Highly concurrent HTTP with Netty and NIO

Question

I am working through the example Netty HTTP Client code in order to make http requests within a concurrent, threaded environment.

However, my system breaks completely (with a slew of exceptions) at fairly low throughput.

In almost pseudo-code:

ClientBootstrap bootstrap = new ClientBootstrap(new NioClientSocketChannelFactory()) 
bootstrap.setPipelineFactory(new HttpClientPipelineFactory());

ChannelFuture future = bootstrap.connect(new InetSocketAddress(host, port));
Channel channel = future.awaitUninterruptibly().getChannel();

HttpRequest request = new DefaultHttpRequest();
channel.write(request);

In the example, to make a request I create a ClientBootstrap, and from there (through a few hoops) a Channel to write the HTTPRequest.

This all works and is good.

However, in a concurrent situation, should every request be going through the same hoops? I think that is what's breaking things for me at the moment. Should I be reusing the connection or structuring my client in an entirely different way?

Also: I am doing this in Clojure, if that makes any difference at all.

Solution 1

No, you're doing things right. You must, however, keep a reference to your Channel instance. Once you have that channel, as long as it is open, you don't need to create another bootstrap. (If that's what you're doing.)

This is what I used in a recent project :

class ClientConnection (constructor)

// Configure the client.
bootstrap = new ClientBootstrap(
    new NioClientSocketChannelFactory(
        Executors.newCachedThreadPool(),
        Executors.newCachedThreadPool()
    )
);

// Set up the pipeline factory.
bootstrap.setPipelineFactory(
    new ChannelPipelineFactory() {
        public ChannelPipeline getPipeline() throws Exception {
            return Channels.pipeline(
                // put your handlers here
            );
        }
    }
);

class ClientConnection.connect(String host, int port)

if (isConnected()) {
    throw new IllegalStateException("already connected");
}

// Start the connection attempt.
ChannelFuture future = bootstrap.connect(new InetSocketAddress(host, port));

channel = future.awaitUninterruptibly().getChannel();

// Wait until the connection is closed or the connection attempt fails.
channel.getCloseFuture().addListener(new ChannelFutureListener() {
    @Override
    public void operationComplete(ChannelFuture future) throws Exception {
        new Thread(new Runnable() {
            public void run() {
                // Shut down thread pools to exit
                // (cannot be executed in the same thread pool!
                bootstrap.releaseExternalResources();

                LOG.log(Level.INFO, "Shutting down");
            }
        }).start();
    }
});

So, basically, I only keep a reference to bootstrap and channel, however the former is pretty much not used outside of these lines of code.

Note: you should only execute bootstrap.releaseExternalResources(); once, when the application is exiting. In my case, the client sends some files then close the channel and exit.

Once you have a connected Channel instance, you need only to use that one until you close it again. Once it is closed, you can recall the bootstrap to create a new Channel again.

Personally, I find Netty a little bit hard to understand at first, but once you grasp how it works, it is simply the best NIO framework in Java. IMO.

Solution 2

Netty is the best approach to writing highly concurrent HTTP services in the JVM but it is very complicated and hard to interop with directly, especially if using Clojure. Take a look at Donkey which provides interop with Vert.x which uses Netty as the backend. This layering abstracts away a lot of the nuanced low level details that, if done wrong, can result in an unstable service. Donkey is relatively new so there isn't a lot of documentation about it yet. Check out this blog that features an open source project which implements a rudimentary news feed microservice in Clojure using Donkey. It goes into details on architecture, design, coding, and performance under load.

Comments

[Piotr Findeisen]

bootstrap will also be useful if you want to make another connection and utilize existing machinery (underlying Netty NIO worker threads).