What is recommended way for spawning threads from a servlet in Tomcat

Question

Probably a repeat! I am using Tomcat as my server and want to know what is best way to spawn threads in the servlet with deterministic outcomes. I am running some long running updates from a servlet action and would like for the request to complete and the updates to happen in the background. Instead of adding a messaging middleware like RabbitMQ, I thought I could spawn a thread that could run in the background and finish in its own time. I read in other SO threads that the server terminates threads spawned by the server in order for it to manage resources well.

Is there a recommended way of spawning threads, background jobs when using Tomcat. I also use Spring MVC for the application.

Solution 1

In a barebones servletcontainer like Tomcat or Jetty, your safest bet is using an applicaton wide thread pool with a max amount of threads, so that the tasks will be queued whenever necessary. The ExecutorService is very helpful in this.

Upon application startup or servlet initialization use the Executors class to create one:

executor = Executors.newFixedThreadPool(10); // Max 10 threads.

Then during servlet's service (you could ignore the result for the case that you aren't interested, or store it in the session for later access):

Future<ReturnType> result = executor.submit(new YourTask(yourData));

Where YourTask must implement Runnable or Callable and can look something like this, whereby yourData is just your data, e.g. populated with request parameter values (just keep in mind that you should absolutely not pass Servlet API artifacts such as HttpServletRequest or HttpServletResponse along!):

public class YourTask implements Runnable {

    private YourData yourData;

    public YourTask(YourData yourData) {
        this.yourData = yourData;
    }

    @Override
    public void run() {
        // Do your task here based on your data.
    }
}

Finally, during application's shutdown or servlet's destroy you need to explicitly shutdown it, else the threads may run forever and prevent the server from properly shutting down.

executor.shutdownNow(); // Returns list of undone tasks, for the case that.

In case you're actually using a normal JEE server such as WildFly, Payara, TomEE, etc, where EJB is normally available, then you can simply put @Asynchronous annotation on an EJB method which you invoke from the servlet. You can optionally let it return a Future<T> with AsyncResult<T> as concrete value.

@Asynchronous
public Future<ReturnType> submit() {
    // ... Do your job here.

    return new AsyncResult<ReturnType>(result);
}

see also:

• Using special auto start servlet to initialize on startup and share application data
• How to run a background task in a servlet based web application?
• Is it safe to manually start a new thread in Java EE?

Solution 2

You could maybe use a CommonJ WorkManager (JSR 237) implementation like Foo-CommonJ:

CommonJ − JSR 237 Timer & WorkManager

Foo-CommonJ is a JSR 237 Timer and WorkManager implementation. It is designed to be used in containers that do not come with their own implementation – mainly plain servlet containers like Tomcat. It can also be used in fully blown Java EE applications servers that do not have a WorkManager API or have a non-standard API like JBoss.

Why using WorkManagers?

The common use case is that a Servlet or JSP needs to aggregate data from multiple sources and display them in one page. Doing your own threading a managed environement like a J2EE container is inappropriate and should never be done in application level code. In this case the WorkManager API can be used to retrieve the data in parallel.

Install/Deploy CommonJ

The deployment of JNDI resources vendor dependant. This implementation comes with a Factory class that implements the javax.naming.spi.ObjectFactory interface with makes it easily deployable in the most popular containers. It is also available as a JBoss service. more...

Update: Just to clarify, here is what the Concurrency Utilities for Java EE Preview (looks like this is the successor of JSR-236 & JSR-237) writes about unmanaged threads:

2.1 Container-Managed vs. Unmanaged Threads

Java EE application servers require resource management in order to centralize administration and protect application components from consuming unneeded resources. This can be achieved through the pooling of resources and managing a resource’s lifecycle. Using Java SE concurrency utilities such as the java.util.concurrency API, java.lang.Thread and java.util.Timer in a server application component such as a servlet or EJB are problematic since the container and server have no knowledge of these resources.

By extending the java.util.concurrent API, application servers and Java EE containers can become aware of the resources that are used and provide the proper execution context for the asynchronous operations to run with.

This is largely achieved by providing managed versions of the predominant java.util.concurrent.ExecutorService interfaces.

So nothing new IMO, the "old" problem is the same, unmanaged thread are still unmanaged threads:

• They are unknown to the application server and do not have access to Java EE contextual information.
• They can use resources on the back of the application server, and without any administration ability to control their number and resource usage, this can affect the application server's ability to recover resources from failure or to shutdown gracefully.

References

• Concurrency Utilities for Java EE interest site
• Concurrency Utilities for Java EE Preview (PDF)

Solution 3

I know it is an old question, but people keep asking it, trying to do this kind of thing (explicitly spawning threads while processing a servlet request) all the time... It is a very flawed approach - for more than one reason... Simply stating that Java EE containers frown upon such practice is not enough, although generally true...

Most importantly, one can never predict how many concurrent requests the servlet will be receiving at any given time. A web application, a servlet, by definition, is meant to be capable of processing multiple requests on the given endpoint at a time. If you are programming you request processing logic to explicitly launch a certain number of concurrent threads, you are risking to face an all but inevitable situation of running out of available threads and choking your application. Your task executor is always configured to work with a thread pool that is limited to a finite reasonable size. Most often, it is not larger than 10-20 (you don't want too many threads executing your logic - depending on the nature of the task, resources they compete for, the number of processors on your server, etc.) Let's say, your request handler (e.g. MVC controller method) invokes one or more @Async-annotated methods (in which case Spring abstracts the task executor and makes things easy for you) or uses the task executor explicitly. As your code executes it starts grabbing the available threads from the pool. That's fine if you are always processing one request at a time with no immediate follow-up requests. (In that case, you are probably trying to use the wrong technology to solve your problem.) However, if it is a web application that is exposed to arbitrary (or even known) clients who may be hammering the endpoint with requests, you will quickly deplete the thread pool, and the requests will start piling up, waiting for threads to be available. For that reason alone, you should realize that you may be on a wrong path - if you are considering such design.

A better solution may be to stage the data to be processed asynchronously (that could be a queue, or any other type of a temporary/staging data store) and return the response. Have an external, independent application or even multiple instances of it (deployed outside your web container) poll the staging endpoint(s) and process the data in the background, possibly using a finite number of concurrent threads. Not only such solution will give you the advantage of asynchronous/concurrent processing, but will also scale because you will be able to run as many instances of such poller as you need, and they can be distributed, pointing to the staging endpoint. HTH

Solution 4

Spring supports asynchronous task (in your case long running) through spring-scheduling. Instead of using Java threads direct I suggest to use it with Quartz.

Recourses:

• Spring reference: Chapter 23

Solution 5

Strictly speaking, you're not allowed to spawn threads according to the Java EE spec. I would also consider the possibility of a denial of service attack (deliberate or otherwise) if multiple requests come in at once.

A middleware solution would definitely be more robust and standards-compliant.

View more solutions

Comments

[Colin Hebert]

The thing with the JEE spec is that in JEE5 you can't use asynchronous calls since @Asynchronous was only added in JEE6. I agree most of the time it's a bad idea to use threads, but sometimes you have to.

[bwawok]

+1 Spring + quartz is very easy to set up, and does a decent job for many situations.

[nos]

I'd consider it good practice to mark those executor threads as daemon threads, otherwise you'll prevent tomcat to shut down if one of your tasks have gone wild. You can do that by passing in a ThreadFactory to one of the Executors.newXXX methods. Having a ServletContextListener to shut down the executor you created is a must too.

[Devanshu Mevada]

@Colin Java EE 5 has JMS or the WorkManager API, @Asynchronous is definitely not a panacea for all kind of background jobs.

[BalusC]

@nos: good point wrt daemon threads. But sometimes you'd like to have them to finish their task to avoid broken results (they might be writing to a file or DB, etc). Depends all on functional requirement which is yet unclear in the original question. Also, if those are to be used by a single servlet, creating and shutting down them in servlet's init() and destroy() is sufficient. Else doing in a ServletContextListener is indeed better, you could then make it available to more servlets.

[BalusC]

Noted should be that this JSR is from 2003, far before java.util.concurrent in Java 1.5 was introduced which makes thread management much more robust.

[Devanshu Mevada]

@BalusC Yes, the JSR has been started in 2003. But the JSR-236 & JSR-237 are still moving AFAIK (even if it's under another name). See the Java EE Concurrency Interest Site maintained by Doug Lea for the latest versions. I'm not sure of the exact status though. Anyway, the problem is not really robustness, the problem is that containers are by definition not aware of unmanaged threads.

[BalusC]

Yes, the awareness of the container of those threads would overall have more benefits in the thread management. It's then controllable at container level instead of webapp level. Thanks for the update, interesting information.

[Devanshu Mevada]

@BalusC Yeah, exactly. I don't know why JSR-236 & JSR-237 didn't make it in Java EE 6. According to the homepage of the JSR-316, it has been considered. And there is a need... Not saying delaying Java EE 6 would have been a good thing of course, just wondering why.

[Ritesh M Nayak]

I like this solution but wont I lose context of what the Job is for? I would want some logic built into the thread based on who spawns the thread and perform some operations on it. The Quartz solution looks really good as well.

[Ritesh M Nayak]

The middleware solution is what I thought of first, problem is that its a lot of re-architecting and generally very time consuming. And the number of such operations aren't too many either.

[BalusC]

Then just pass that through as constructor argument of CallableTask.

[Ritesh M Nayak]

I found the quartz scheduler mechanism to be really good. I am using the TaskScheduler bean to do all of the background processing.

[Timo Ernst]

How could I determine a good value for the max. number of threads that I need for Executors.newFixedThreadPool(n)? I am running a multi-user application and I expect most users to spawn 1-10 threads (maybe more). I don't know how many users I will end up though.

[Raedwald]

The question is about servlets, not JEE.