FixedThreadPool vs CachedThreadPool: the lesser of two evils

81,297

Solution 1

A CachedThreadPool seems appropriate for your situation as there are no negative consequence to using one for long running threads directly. The comment in the java doc about CachedThreadPools being suitable for short tasks merely suggest that they are particularly appropriate for such cases, not that they cannot be used for long running tasks.

The main concern with a CachedThreadPool is that it will create up to Integer.MAX_VALUE number of threads as it will always spawn a new thread if an unused one does not exist in the cache. So if you have long running tasks it is then more likely that you could grow the number of concurrent threads more than you desire since this type of thread pool will not limit how many execute concurrently itself. This does not seem to be a problem for your use case as described, but it is something to be aware of.

To elaborate further on the difference between a CachedThreadPool and a FixedThreadPool, Executors.newCachedThreadPool and Executors.newFixedThreadPool are both backed by the same thread pool implementation (at least in the open JDK) via an instance of ThreadPoolExecutor, just with different parameters. The differences just being their thread minimum, maximum, thread kill time, and queue type.

public static ExecutorService newFixedThreadPool(int nThreads) {
     return new ThreadPoolExecutor(nThreads, nThreads,
                                   0L, TimeUnit.MILLISECONDS,
                                   new LinkedBlockingQueue<Runnable>());
 }

public static ExecutorService newCachedThreadPool() {
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                 60L, TimeUnit.SECONDS,
                                 new SynchronousQueue<Runnable>());
}

A FixedThreadPool does have its advantages when you do in fact want to work with a fixed number of threads, since then you can submit any number of tasks to the executor service while knowing that the number of threads will be maintained at the level you specified. If you explicitly want to grow the number of threads, then this is not the appropriate choice.

This does however mean that the one issue that you may have with the CachedThreadPool is in regards to limiting the number of threads that are running concurrently. The CachedThreadPool will not limit them for you, so you may need to write your own code to ensure that you do not run too many threads, which you can do relatively easily by instantiating your own ThreadPoolExecutor with your desired behaviour characteristics. This really depends on the design of your application and how tasks are submitted to the executor service.

Solution 2

Both FixedThreadPool and CachedThreadPool are evils in highly loaded applications.

CachedThreadPool is more dangerous than FixedThreadPool

If your application is highly loaded & demands low latency, better to get rid of both options due to below drawbacks

  1. Unbounded nature of task queue : It may cause out of memory or high latency
  2. Long running threads will cause CachedThreadPool to go out of control on Thread creation

Since you know that both are evils, lesser evil doesn't do any good. Prefer ThreadPoolExecutor, which provides granular control on many parameters.

  1. Set the task queue as bounded queue to have better control
  2. Have right RejectionHandler - Your own RejectionHandler or Default handlers provided by JDK
  3. If you have something to do on before/after completion of task, override beforeExecute(Thread, Runnable) and afterExecute(Runnable, Throwable)
  4. Override ThreadFactory, if thread customization is required
  5. Control Thread pool size dynamically at run time ( related SE question : Dynamic Thread Pool)

Solution 3

So I have a program that spawns threads (~5-150) which perform a bunch of tasks.

Are you sure you understand how threads are actually processed by your OS and hardware of choice? How Java maps threads to OS threads, how that maps threads to CPU threads etc.? I'm asking because creating 150 threads within in ONE JRE only makes sense if you have massive CPU cores/threads underneath, which most likely is not the case. Depending on the OS and RAM in use, creating more than n threads might even result in your JRE being terminated because of OOM errors. So you should really distinguish between threads and work to do by those threads, how many work you are even able to process etc.

And that's the problem with CachedThreadPool: It doesn't make sense to queue up long running work in threads which actually can't run because you only have 2 CPU cores able to process those threads. If you end up with 150 scheduled threads you might create a lot of unnecessary overhead for the schedulers used within Java and the OS to concurrently process them. This is simply impossible if you only have 2 CPU cores, unless your threads are waiting for I/O or such all the time. But even in that case a lot of threads would create a lot of I/O...

And that problem doesn't occur with FixedThreadPool, created with e.g. 2+n threads, where n is reasonable low of course, because with that hardware and OS resources are used with far less overhead for managing threads which can't run anyway.

Share:
81,297

Related videos on Youtube

Daniel
Author by

Daniel

I'm a Master's CS student focusing on AI. I'm interesting in AI, Game Design, and Malware. Hit me up if you're interested in collaborating or teaching me.

Updated on July 08, 2022

Comments

  • Daniel
    Daniel over 1 year

    I have a program that spawns threads (~5-150) which perform a bunch of tasks. Originally, I used a FixedThreadPool because this similar question suggested they were better suited for longer lived tasks and with my very limited knowledge of multithreading, I considered the average life of the threads (several minutes) "long lived".

    However, I recently added the capability to spawn additional threads and doing so takes me above the thread limit I set. In this case, would it be better to guess and increase the number threads I can allow or to switch to a CachedThreadPool so I have no wasted threads?

    Trying them both out preliminarily, there doesn't seem to be a difference so I'm inclined to go with the CachedThreadPool just to avoid the waste. However, does the life span of the threads mean I should instead picked a FixedThreadPool and just deal with the unused threads? This question makes it seem like those extra threads aren't wasted but I would appreciate the clarification.

  • Michel Feinstein
    Michel Feinstein over 6 years
    Sometimes there isn't a better choice, you could just have 1 CPU core but if you are running a server where every user request would trigger a thread to process the request, there won't be any other reasonable choice, specially if you plan to scale the server once you grow your user base.
  • Thorsten Schöning
    Thorsten Schöning over 6 years
    @mFeinstein How can one not have a choice if one is in the position to choose a thread pool implementation? In your example with 1 CPU core only spawning more threads simply doesn't make any sense, it's fitting perfectly to my example using a FixedThreadPool. That scales easily as well, first with on or two worker threads, later with 10 or 15 depending on the number of cores.
  • Michel Feinstein
    Michel Feinstein over 6 years
    The vast majority of the web servers implementations will create one new thread for each new HTTP request... They won't care about how many actual cores the machine have, this makes the implementation more simple and easier to scale. This applies to many other designs where you just want to code once and deploy, and not have to recompile and redeploy if you change the machine, which could be a cloud instance.
  • Thorsten Schöning
    Thorsten Schöning over 6 years
    @mFeinstein Most web servers use thread pools for requests on their own, simply because spawning threads which can't run doesn't make sense, or they use event loops for connections and process the requests in pools afterwards or such. Additionally, you are missing the point, which is that the question is about one being able to choose the correct thread pool and spawning threads which can't run anyways still doesn't make sense. A FixedthreadPool configured to a reasonable amount of threads per machine depending on cores scales just fine.
  • Michel Feinstein
    Michel Feinstein over 6 years
    Well Tomcat I am pretty sure uses 1 thread per request. Although I understand your point, I would rather make a code that runs fine on any machine and scales nicely without my interference... Unless it was very performance demanding then I would go and fine tune it for my current machine.
  • Thorsten Schöning
    Thorsten Schöning over 6 years
    @mFeinstein Regarding Tomcat: stackoverflow.com/a/33241110/2055163 And regarding your code, then simply check how many cores are available or provide some configuration and forward that value to FixedThreadPool. The latter is the best anyway, because it provides admins full control.
  • Michel Feinstein
    Michel Feinstein over 6 years
    I agree there will be performance improvements, but it's good to know that this way some users will have a faster response and others might wait for while for a server response, since their request might get to a queue... Whereas in the original installation all the users will have the same experience, good or bad....I am just pointing out the differences, not making any recommendations
  • Paul Draper
    Paul Draper almost 6 years
    @ThorstenSchöning, having 50 CPU-bound threads on a 2-core machine is unhelpful. Having 50 IO-bound threads on a 2-core machine can be very helpful.
  • Crosk Cool
    Crosk Cool almost 5 years
    What if someone decides to use commonPool?
  • Nishit
    Nishit over 4 years
    "A CachedThreadPool is exactly what you should use for your situation as there are no negative consequence to using one for long running threads". I don't think I agree. CachedThreadPool dynamically creates threads with no upper limit. Long running tasks on a large number of threads can potentially hog all the resources. Also, having more threads than ideal can result in too many resources wasted on context switching of these threads. Although you explained at the end of the answer that custom throttling is required, the beginning of the answer is a bit misleading.
  • Ayaskant
    Ayaskant over 4 years
    @Ravindra - You have beautifully explained the cons of both CachedThreadPool and FixedThreadPool. This shows you have got a deep understanding of the concurrency package.
  • Abhijit Sarkar
    Abhijit Sarkar over 4 years
    Why not simply create a bounded ThreadPoolExecutor like ThreadPoolExecutor(0, maximumPoolSize, 60L, TimeUnit.SECONDS, SynchronousQueue())?
  • garnet
    garnet over 2 years
    The same is explained in detail in baeldung.com/java-executors-cached-fixed-threadpool, especially, the section baeldung.com/….