Check the max allowed threads count for sure

Question

I'm running an Ubuntu 17.10 server on the VDS. I'm running a 3proxy on it.
The problem is that VDS somehow does not allow running more than 5000 threads per process, as I think. I think so because the 3proxy process never exceed 5000 threads.

EDIT: the error I see in the logs is:

pthread_create():_Resource_temporarily_unavailable

So I want to check if I am right about it by checking the real thread-per-process limit. My idea is to create a script, which will start as many threads as system will allow, and check the result of how many will it be.

And my problem is that I don't know how to create such a script

Solution

There is no threads per process limit!

But there is a limit on the total number of processes on the system (threads are essentially just processes with a shared address space on Linux)

cat /proc/sys/kernel/threads-max

The default is the number of memory pages divided by 4. You can increase this like:

echo 100000 > /proc/sys/kernel/threads-max

There is also a limit of processes (threads) that a single user may create, see ulimit for this, but this should be unlimited on Ubuntu.

Comments

[Doug Smythies]

Please see here and here. I was able to get to 118000 threads.

[Vladyslav Matviienko]

I have tried that already, as well as playing with ulimit setting large values. I forgot to mention the erro message I get: pthread_create():_Resource_temporarily_unavailable

[Robert Riedl]

What is the output of cat /proc/sys/kernel/threads-max ?

[Vladyslav Matviienko]

also echo 100000 > /proc/sys/kernel/threads-max doesn't change proc/sys/kernel/threads-max, if I check it after I run the command, it is the same as before: 63814

[Robert Riedl]

so this is not a system limit but rather a limit of your software, 3proxy

[Vladyslav Matviienko]

As for actual process limit: I have a bash script, which starts 20000 sleep 20 & commands, and as for top maximum Tasks: 10980 total, and I'm getting fork: retry: Resource temporarily unavailable after it reaches 10980

[Robert Riedl]

looking at some documentation it says A number of simulatineous connections per service is limited by 'maxconn' option. Default maxconn value since 3proxy 0.8 is 500. and ou may want to set 'maxconn' to higher value. Under this configuration: maxconn 1000

[Vladyslav Matviienko]

limit of your software I thought same way until I got another server, on which it is able to start 18000 threads just fine with same config. Config maxconn is 100000

[Robert Riedl]

Let us continue this discussion in chat.