Springboot TomcatEmbeddedServletContainer KeepAliveTimeout not working

Question

I have set the keep alive timeout in spring boot embeded tomcat server to 30 seconds. So i use below in the Application.java,

@Bean   
public EmbeddedServletContainerFactory getEmbeddedServletContainerFactory() {
    TomcatEmbeddedServletContainerFactory containerFactory = new TomcatEmbeddedServletContainerFactory();
    containerFactory
            .addConnectorCustomizers(new TomcatConnectorCustomizer() {
                @Override
                public void customize(Connector connector) {
                    ((AbstractProtocol) connector.getProtocolHandler())
                            .setKeepAliveTimeout(30000);
                }
            });

    return containerFactory;
}

Then i sleep a request thread for 40 seconds from my rest controller. But when i make a request via postman it successfully return HTTP status code 200 instead it should return gateway timeout error.

I try both setConnectionTimeout and setKeepAliveTimeout and it did not work.

What am i missing here?

Edit question: My initial problem

Let me explain the original question of mine, which lead me to ask above question.

Well i have a long poll process which normally runs about more than 5 minits.

So what happen is when i call the Rest API for longpoll, After 2.2 minits i get a 504 http error in browser.

I am using a AWS environment, where i have a ELB and a HAProxy which is installed in AWS EC2 instance.

As per AWS doc, it says the default Idle Connection Timeout of ELB is 60 seconds. So i have increase it to up to 30 mins.

Moreover it says,

If you use HTTP and HTTPS listeners, we recommend that you enable the keep-alive option for your EC2 instances. You can enable keep-alive in your web server settings or in the kernel settings for your EC2 instances.

So have increase the embedded tomcat keep-alive timeout like above code snippet to 30.2 mins

So now i expect my long poll request to be completed, with out getting a 504 error. But still i get 504 error in browser?

Ref: AWS dev guide

Solution

It looks like you want to close abandoned HTTP connections which might occur on mobile devices.

@RestController
@SpringBootApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Bean
    public EmbeddedServletContainerFactory getEmbeddedServletContainerFactory() {
        TomcatEmbeddedServletContainerFactory containerFactory = new TomcatEmbeddedServletContainerFactory();
        containerFactory
                .addConnectorCustomizers(new TomcatConnectorCustomizer() {
                    @Override
                    public void customize(Connector connector) {
                        ((AbstractProtocol) connector.getProtocolHandler()).setConnectionTimeout(100);
                    }
                });

        return containerFactory;
    }

    @RequestMapping
    public String echo(@RequestBody String body) {
        return body;
    }
}

Connection timeout has been set to 100 millisencods in order to run my tests fast. Data is sent in chunks. Between every chunk the running thread is suspended for x milliseconds.

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = DemoApplication.class)
@WebIntegrationTest("server.port:19000")
public class DemoApplicationTests {

    private static final int CHUNK_SIZE = 1;
    private static final String HOST = "http://localhost:19000/echo";

    @Rule
    public ExpectedException expectedException = ExpectedException.none();

    @Test
    public void slowConnection() throws Exception {
        final HttpURLConnection connection = openChunkedConnection();
        OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());

        writeAndWait(500, out, "chunk1");
        writeAndWait(1, out, "chunk2");

        out.close();

        expectedException.expect(IOException.class);
        expectedException.expectMessage("Server returned HTTP response code: 400 for URL: " + HOST);

        assertResponse("chunk1chunk2=", connection);
    }

    @Test
    public void fastConnection() throws Exception {
        final HttpURLConnection connection = openChunkedConnection();
        OutputStreamWriter out = new OutputStreamWriter(connection.getOutputStream());

        writeAndWait(1, out, "chunk1");
        writeAndWait(1, out, "chunk2");

        out.close();

        assertResponse("chunk1chunk2=", connection);
    }

    private void assertResponse(String expected, HttpURLConnection connection) throws IOException {
        Scanner scanner = new Scanner(connection.getInputStream()).useDelimiter("\\A");
        Assert.assertEquals(expected, scanner.next());
    }

    private void writeAndWait(int millis, OutputStreamWriter out, String body) throws IOException, InterruptedException {
        out.write(body);
        Thread.sleep(millis);
    }

    private HttpURLConnection openChunkedConnection() throws IOException {
        final URL url = new URL(HOST);
        final HttpURLConnection  connection = (HttpURLConnection) url.openConnection();
        connection.setDoOutput(true);
        connection.setChunkedStreamingMode(CHUNK_SIZE);
        return connection;
    }
}

Set log level for package org.apache.catalina.core to DEBUG

logging.level.org.apache.catalina.core=DEBUG

and you can see a SocketTimeoutException for slowConnection test.

I don't know why you want HTTP status code 502 as error response status. HTTP 502 says:

The 502 (Bad Gateway) status code indicates that the server, while acting as a gateway or proxy, received an invalid response from an inbound server it accessed while attempting to fulfill the request.

The client Postman calls your server application. I don't see any gateway or proxy in between.

If you just condensed your question to a bare minimum and in reality you want to build a proxy on your own, you might consider using Netflix Zuul.

---

Update 23.03.2016:

That is the root cause for OP's question on Stackoverflow:

What i did with longpolling was, from service api, i sleep the thread for some time and wake it and do it again and again untill some db status is completed.

That implementation actually prevents the Tomcat worker thread from processing new HTTP requests. As a result your request throughput reduces with every additional long running operation.

I propose to offload the long running operation into a separate thread. The client (browser) initiates a new request to fetch the result. Depending on the processing status, server returns either the result or a notification/error/warning/.

Here's a very simple example :

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.NOT_FOUND;
import static org.springframework.http.HttpStatus.OK;

@RestController
@SpringBootApplication
public class DemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    private ExecutorService executorService = Executors.newFixedThreadPool(10);
    private Map<String, String> results = new ConcurrentHashMap<>();

    @RequestMapping(path = "put/{key}", method = RequestMethod.POST)
    public ResponseEntity<Void> put(@PathVariable String key) {
        executorService.submit(() -> {
            try {
                //simulate a long running process
                Thread.sleep(10000);
                results.put(key, "success");
            } catch (InterruptedException e) {
                results.put(key, "error " + e.getMessage());
                Thread.currentThread().interrupt();
            }
        });
        return new ResponseEntity<>(CREATED);
    }

    @RequestMapping(path = "get/{key}", method = RequestMethod.GET)
    public ResponseEntity<String> get(@PathVariable String key) {
        final String result = results.get(key);
        return new ResponseEntity<>(result, result == null ? NOT_FOUND : OK);
    }
}