fork: retry: Resource temporarily unavailable
Solution 1
This is commonly caused by running out of file descriptors.
There is the systems total file descriptor limit, what do you get from the command:
sysctl fs.file-nr
This returns counts of file descriptors:
<in_use> <unused_but_allocated> <maximum>
To find out what a users file descriptor limit is run the commands:
sudo su - <username>
ulimit -Hn
To find out how many file descriptors are in use by a user run the command:
sudo lsof -u <username> 2>/dev/null | wc -l
So now if you are having a system file descriptor limit issue you will need to edit your /etc/sysctl.conf file and add, or modify it it already exists, a line with fs.file-max and set it to a value large enough to deal with the number of file descriptors you need and reboot.
fs.file-max = 204708
Solution 2
Another possibility is too many threads. We just ran into this error message when running a test harness against an app that uses a thread pool. We used
watch -n 5 -d "ps -eL <java_pid> | wc -l"
to watch the ongoing count of Linux native threads running within the given Java process ID. After this hit about 1,000 (for us--YMMV), we started getting the error message you mention.
Related videos on Youtube
user1260391
Updated on April 07, 2020Comments
-
user1260391 about 4 years
I tried installing Intel MPI Benchmark on my computer and I got this error:
fork: retry: Resource temporarily unavailable
Then I received this error again when I ran
ls
andtop
command.What is causing this error?
Configuration of my machine:
Dell precision T7500 Scientific Linux release 6.2 (Carbon)
-
theglauber over 11 yearsCheck your Linux documentation on how to increase the number of processes.
-
theglauber over 11 yearsPerhaps this would help: stackoverflow.com/questions/344203/…
-
-
EmmEff about 10 yearsFYI, you can use "sysctl -p" to apply the current settings in /etc/sysctl.conf saving the reboot.
-
Ondrej Galbavý over 8 yearsAlso, if you have read only /etc, you can use 'sysctl -w fs.file-max=204708' at runtime.
-
Sanghyun Lee almost 8 yearsWhen I tested,
ps -eL
worked for showing all processes andps -L <pid>
worked for showing processes regarding the<pid>
.ps -eL <pid>
just shows all processes regardless of the<pid>
. -
Brandon Elliott almost 8 years@Willie Wheeler What did you do to overcome the thread limit? I have been searching and testing numerous solutions found online, to no avail, for DAYS. Nothing will allow more than 1k threads, except for a restart of systemd-logind, which lasts for about an hour before the errors start again!
-
Admin over 7 yearsMachines have finite resources. If you are hitting a limit, use fewer threads (e.g. controlled with thread pool) or else more machines.
-
thedanotto over 7 yearsI restarted my rails server
-
frankster almost 7 years1000 threads isn't a lot though
-
mati kepson over 5 yearsyou can also add it directly to kernel without reboot but not permanently 'echo 999999 > /proc/sys/fs/file-max'