Java thread stack size on 64-bit linux

Question

My goal is to come up with figure of max threads which can run in parallel. I was pointed to many links by Google, where they give simple math by dividing the RAM/StackSize. In 64 bit Linux, we have thread stack size defined as 10 MB(ulimit -s = 10240kb) and RAM was 4GB, leaving 1 GB for OS and going with this math I can have ~300 threads or so but small test application which I wrote goes upto ~32297 and then gives out of memory error.

I tried different values with -Xss but these values hardly have any effect on thread count, it remains same as ~32297).

This gave me an impression that stack size is variable and decided by OS and goes upto max defined by us whenever needed, but wherever I read, they size stack size is static

What exactly I'm missing here?

Solution 1

Try checking/changing linux maximum stack size using

ulimit -s

Also check for linux threads limit

cat /proc/sys/kernel/threads-max  

Solution 2

I have also found a limit of about 32K for thread in Java. If you have this many threads, its usually a better idea to use a different approach. On my machine 32K thread doing while(true) Thread.sleep(1000) will consume 3 cores just context switching.

Java: What is the limit to the number of threads you can create?

Solution 3

Linux implements max number of threads per process indirectly!!

number of threads = total virtual memory / (stack size*1024*1024)

Thus, the number of threads per process can be increased by increasing total virtual memory or by decreasing stack size. But, decreasing stack size too much can lead to code failure due to stack overflow while max virtual memory is equals to the swap memory.

Check you machine:

Total Virtual Memory: ulimit -v (default is unlimited, thus you need to increase swap memory to increase this)

Total Stack Size: ulimit -s (default is 8Mb)

Command to increase these values:

ulimit -s newvalue

ulimit -v newvalue

*Replace new value with the value you want to put as limit.

References:

http://dustycodes.wordpress.com/2012/02/09/increasing-number-of-threads-per-process/

Comments

[Gaurav]

cat /proc/sys/kernel/threads-max is 81920 . I want to know the relation b/w ulimit -s and threads-max, will threads-max value gets updated if I change the stack size?

[Gaurav]

In the link you shared, they mention on 64 bit with stack size of 128K, limit was 32,072 and with stack size of 512K limit is again 32,072. Do you know why stack size is not having any impact on number of threads which can be created?

[Gaurav]

Fine, the question which is still unanswered is why max thread count reamin same even on changing the stack size to very large or very small value. Does OS choose the stack size dynamically?