Limit memory usage for a single Linux process
Solution 1
There's some problems with ulimit. Here's a useful read on the topic: Limiting time and memory consumption of a program in Linux, which lead to the timeout tool, which lets you cage a process (and its forks) by time or memory consumption.
The timeout tool requires Perl 5+ and the /proc
filesystem mounted. After that you copy the tool to e.g. /usr/local/bin
like so:
curl https://raw.githubusercontent.com/pshved/timeout/master/timeout | \
sudo tee /usr/local/bin/timeout && sudo chmod 755 /usr/local/bin/timeout
After that, you can 'cage' your process by memory consumption as in your question like so:
timeout -m 500 pdftoppm Sample.pdf
Alternatively you could use -t <seconds>
and -x <hertz>
to respectively limit the process by time or CPU constraints.
The way this tool works is by checking multiple times per second if the spawned process has not oversubscribed its set boundaries. This means there actually is a small window where a process could potentially be oversubscribing before timeout notices and kills the process.
A more correct approach would hence likely involve cgroups, but that is much more involved to set up, even if you'd use Docker or runC, which among things, offer a more user-friendly abstraction around cgroups.
Solution 2
Another way to limit this is to use Linux's control groups. This is especially useful if you want to limit a process's (or group of processes') allocation of physical memory distinctly from virtual memory. For example:
cgcreate -g memory:myGroup
echo 500M > /sys/fs/cgroup/memory/myGroup/memory.limit_in_bytes
echo 5G > /sys/fs/cgroup/memory/myGroup/memory.memsw.limit_in_bytes
will create a control group named myGroup
, cap the set of processes run under myGroup
up to 500 MB of physical memory with memory.limit_in_bytes
and up to 5000 MB of physical and swap memory together with memory.memsw.limit_in_bytes
.
More info about these options can be found here: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/sec-memory
To run a process under the control group:
cgexec -g memory:myGroup pdftoppm
Note that on a modern Ubuntu distribution this example requires installing the cgroup-tools
package (previously cgroup-bin
):
sudo apt install cgroup-tools
and editing /etc/default/grub
to change GRUB_CMDLINE_LINUX_DEFAULT
to:
GRUB_CMDLINE_LINUX_DEFAULT="cgroup_enable=memory swapaccount=1"
and then running sudo update-grub
and rebooting to boot with the new kernel boot parameters.
Solution 3
If your process doesn't spawn more children that consume the most memory, you may use setrlimit
function. More common user interface for that is using ulimit
command of the shell:
$ ulimit -Sv 500000 # Set ~500 mb limit
$ pdftoppm ...
This will only limit "virtual" memory of your process, taking into account—and limiting—the memory the process being invoked shares with other processes, and the memory mapped but not reserved (for instance, Java's large heap). Still, virtual memory is the closest approximation for processes that grow really large, making the said errors insignificant.
If your program spawns children, and it's them which allocate memory, it becomes more complex, and you should write auxiliary scripts to run processes under your control. I wrote in my blog, why and how.
Solution 4
On any systemd-based distro you can also use cgroups indirectly through systemd-run. E.g. for your case of limiting pdftoppm
to 500M of RAM, starting with cgroupsv2, you can simply do:
systemd-run --scope -p MemoryMax=500M --user pdftoppm
Previously, this required booting with the systemd.unified_cgroup_hierarchy
kernel parameter, but tested as of Ubuntu 22.04 cgroup-tools 2.0-2, this does not seem to be the case anymore, the command just worked without any changes to the kernel parameters, and systemd.unified_cgroup_hierarchy
is not set.
Before cgroupsv2 you could not use --user
, and would instead run:
systemd-run --scope -p MemoryMax=500M pdftoppm
but without --user
, this will ask you for a password every time, even though the app gets launched as your user. Do not allow this to delude you into thinking that the command needs sudo
, because that would cause the command to run under root, which was hardly your intention.
Solution 5
I'm using the below script, which works great. It uses cgroups through Update: it now uses the commands from cgmanager
.cgroup-tools
. Name this script limitmem
and put it in your $PATH and you can use it like limitmem 100M bash
. This will limit both memory and swap usage. To limit just memory remove the line with memory.memsw.limit_in_bytes
.
edit: On default Linux installations this only limits memory usage, not swap usage. To enable swap usage limiting, you need to enable swap accounting on your Linux system. Do that by setting/adding swapaccount=1
in /etc/default/grub
so it looks something like
GRUB_CMDLINE_LINUX="swapaccount=1"
Then run sudo update-grub
and reboot.
Disclaimer: I wouldn't be surprised if cgroup-tools
also breaks in the future. The correct solution would be to use the systemd api's for cgroup management but there are no command line tools for that a.t.m.
edit (2021): Until now this script still works, but it goes against Linux's recommendation to have a single program manage your cgroups. Nowadays that program is usually systemd. Unfortunately systemd has a number of limitations that make it difficult to replace this script with systemd invocations. The systemd-run --user
command should allow a user to run a program with resource limitations, but that isn't supported on cgroups v1. (Everyone uses cgroups v1 because docker doesn't work on cgroupsv2 yet except for the very latest versions.) With root access (which this script also requires) it should be possible to use systemd-run
to create the correct systemd-supported cgroups, and then manually set the memory and swap properties in the right cgroup, but that is still to be implemented. See also this bug comment for context, and here and here for relevant documentation.
According to @Mikko's comment using a script like this with systemd runs the risk of systemd losing track of processes in a sessions. I haven't noticed such problems, but I use this script mostly on a single-user machine.
#!/bin/sh
# This script uses commands from the cgroup-tools package. The cgroup-tools commands access the cgroup filesystem directly which is against the (new-ish) kernel's requirement that cgroups are managed by a single entity (which usually will be systemd). Additionally there is a v2 cgroup api in development which will probably replace the existing api at some point. So expect this script to break in the future. The correct way forward would be to use systemd's apis to create the cgroups, but afaik systemd currently (feb 2018) only exposes dbus apis for which there are no command line tools yet, and I didn't feel like writing those.
# strict mode: error if commands fail or if unset variables are used
set -eu
if [ "$#" -lt 2 ]
then
echo Usage: `basename $0` "<limit> <command>..."
echo or: `basename $0` "<memlimit> -s <swaplimit> <command>..."
exit 1
fi
cgname="limitmem_$$"
# parse command line args and find limits
limit="$1"
swaplimit="$limit"
shift
if [ "$1" = "-s" ]
then
shift
swaplimit="$1"
shift
fi
if [ "$1" = -- ]
then
shift
fi
if [ "$limit" = "$swaplimit" ]
then
memsw=0
echo "limiting memory to $limit (cgroup $cgname) for command $@" >&2
else
memsw=1
echo "limiting memory to $limit and total virtual memory to $swaplimit (cgroup $cgname) for command $@" >&2
fi
# create cgroup
sudo cgcreate -g "memory:$cgname"
sudo cgset -r memory.limit_in_bytes="$limit" "$cgname"
bytes_limit=`cgget -g "memory:$cgname" | grep memory.limit_in_bytes | cut -d\ -f2`
# try also limiting swap usage, but this fails if the system has no swap
if sudo cgset -r memory.memsw.limit_in_bytes="$swaplimit" "$cgname"
then
bytes_swap_limit=`cgget -g "memory:$cgname" | grep memory.memsw.limit_in_bytes | cut -d\ -f2`
else
echo "failed to limit swap"
memsw=0
fi
# create a waiting sudo'd process that will delete the cgroup once we're done. This prevents the user needing to enter their password to sudo again after the main command exists, which may take longer than sudo's timeout.
tmpdir=${XDG_RUNTIME_DIR:-$TMPDIR}
tmpdir=${tmpdir:-/tmp}
fifo="$tmpdir/limitmem_$$_cgroup_closer"
mkfifo --mode=u=rw,go= "$fifo"
sudo -b sh -c "head -c1 '$fifo' >/dev/null ; cgdelete -g 'memory:$cgname'"
# spawn subshell to run in the cgroup. If the command fails we still want to remove the cgroup so unset '-e'.
set +e
(
set -e
# move subshell into cgroup
sudo cgclassify -g "memory:$cgname" --sticky `sh -c 'echo $PPID'` # $$ returns the main shell's pid, not this subshell's.
exec "$@"
)
# grab exit code
exitcode=$?
set -e
# show memory usage summary
peak_mem=`cgget -g "memory:$cgname" | grep memory.max_usage_in_bytes | cut -d\ -f2`
failcount=`cgget -g "memory:$cgname" | grep memory.failcnt | cut -d\ -f2`
percent=`expr "$peak_mem" / \( "$bytes_limit" / 100 \)`
echo "peak memory used: $peak_mem ($percent%); exceeded limit $failcount times" >&2
if [ "$memsw" = 1 ]
then
peak_swap=`cgget -g "memory:$cgname" | grep memory.memsw.max_usage_in_bytes | cut -d\ -f2`
swap_failcount=`cgget -g "memory:$cgname" |grep memory.memsw.failcnt | cut -d\ -f2`
swap_percent=`expr "$peak_swap" / \( "$bytes_swap_limit" / 100 \)`
echo "peak virtual memory used: $peak_swap ($swap_percent%); exceeded limit $swap_failcount times" >&2
fi
# remove cgroup by sending a byte through the pipe
echo 1 > "$fifo"
rm "$fifo"
exit $exitcode
Ben Dilts
Updated on September 17, 2022Comments
-
Ben Dilts over 1 year
I'm running
pdftoppm
to convert a user-provided PDF into a 300DPI image. This works great, except if the user provides an PDF with a very large page size.pdftoppm
will allocate enough memory to hold a 300DPI image of that size in memory, which for a 100 inch square page is 100*300 * 100*300 * 4 bytes per pixel = 3.5GB. A malicious user could just give me a silly-large PDF and cause all kinds of problems.So what I'd like to do is put some kind of hard limit on memory usage for a child process I'm about to run--just have the process die if it tries to allocate more than, say, 500MB of memory. Is that possible?
I don't think ulimit can be used for this, but is there a one-process equivalent?
-
Sridhar Sarnobat almost 8 yearsMaybe
docker
? -
Robert Strauch almost 5 years
-
Andrew almost 4 years
-
-
Adam Mills about 13 yearsif i understand the question correctly then OP whats the limit per subprocess (child) .. not in total.
-
Admin about 13 years@MarkR, anyway, virtual address space is a good approximation for the memory used, especially if you run a program that's not controlled by a virtual machine (say, Java). At least I don't know any better metric.
-
MarkR about 13 yearsVirtual address space is the best we really have; there isn't an easily measurable alternative. The pages measured don't need to be in core, they don't need to be private to that process, they're still counted.
-
sdaau about 11 yearsJust wanted to say thanks - this
ulimit
approach helped me withfirefox
's bug 622816 – Loading a large image can "freeze" firefox, or crash the system; which on a USB boot (from RAM) tends to freeze the OS, requiring hard restart; now at leastfirefox
crashes itself, leaving the OS alive... Cheers! -
ddzzbbwwmm over 7 yearsWhat are the soft and hard limits?
-
user2167582 over 7 yearsdo you just run the command after this?
-
Aaron Franke about 7 years
call to cgmanager_create_sync failed: invalid request
for every process I try to run withlimitmem 100M processname
. I'm on Xubuntu 16.04 LTS and that package is installed. -
kvz about 7 yearsSeems to be working for me now (again?) but here's the google cache version: webcache.googleusercontent.com/…
-
Totor almost 7 yearsOn 32 bits machines, the maximum limit is 2 GiB (or unlimited), cf
RLIMIT_AS
in man (2) setrlimit. -
ransh over 6 yearsCan we use timeout together with taskset (we need to limit both memory and cores) ?
-
nerkn about 6 yearsI want to use this approach for chrome. As far I understand, only child process will fail that is 16G, not the whole chrome?
-
R Kiselev about 6 yearsUps, I get this error message:
$ limitmem 400M rstudio limiting memory to 400M (cgroup limitmem_24575) for command rstudio Error org.freedesktop.DBus.Error.InvalidArgs: invalid request
any idea? -
JanKanis about 6 years@RKiselev cgmanager is deprecated now, and not even available in Ubuntu 17.10. The systemd api that it uses was changed at some point, so that's probably the reason. I have updated the script to use cgroup-tools commands.
-
Ned64 about 6 yearsThe
firejail
program will also let you start a process with memory limits (using cgroups and namespaces to limit more than just memory). On my systems I did not have to change the kernel command line for this to work! -
user1404316 about 6 yearsIt should be noted that this answer is not referring to the linux standard
coreutils
utility of the same name! Thus, the answer is potentially dangerous if anywhere on your system, some package has a script expectingtimeout
to be the linux standardcoreutils
package! I am unaware of this tool being packaged for distributions such as debian. -
Willi Ballenthin almost 6 yearsif the calculation for
percent
results in zero, theexpr
status code is 1, and this script exits prematurely. recommend changing the line to:percent=$(( "$peak_mem" / $(( "$bytes_limit" / 100 )) ))
(ref: unix.stackexchange.com/questions/63166/…) -
stason over 5 yearsDo you need the
GRUB_CMDLINE_LINUX_DEFAULT
modification to make the setting persistent? I found another way to make it persistent here. -
xxx374562 over 5 yearsDoes
-t <seconds>
constraint kill the process after that many seconds? -
d9ngle about 5 yearshow can I config cgroup to kill my process if I go above the limit?
-
JanKanis about 5 years@d9ngle You don't really need to, Linux won't allow the process to go above the limit. Memory allocations will fail or the OOM killer will kill the process. Otherwise there's the
memory.failcnt
andmemory.memsw.failcnt
files in the cgroup directory that you could monitor to see if the count gets above 0, but that may not be what you want as when that first happens there is usually memory in your process somewhere that the OS can reclaim without harm (e.g. mapped files). -
d9ngle about 5 years@sourcejedi added it :)
-
d9ngle about 5 years@JanKanis I forgot to tag you. Please see my answer below.
-
JanKanis about 5 yearsRight, I edited my answer. To enable swap limits you need to enable swap accounting on your system. There's a small runtime overhead to that so it isn't enabled by default on Ubuntu. See my edit.
-
Robert Strauch almost 5 years
-
stewbasic almost 5 yearsIt would be useful to note in this answer that on some distributions (eg Ubuntu) sudo is required for cgcreate, and also the later commands unless permission is given to the current user. This would save the reader from having to find this information elsewhere (eg askubuntu.com/questions/345055). I suggested an edit to this effect but it was rejected.
-
Geradlus_RU over 4 yearsThank you, made my day
-
Daniel over 4 yearsMight also be helpful that -m is accepting kilobytes. The example above suggests its using MB.
-
Andrei Sinitson over 4 yearsShort and sweet. Before I tried
firejail
, but that seemed to be overkill with too many side effects for just limiting memory consumption. Thanks! -
Andrei Sinitson over 4 yearsAnother note: this is using
cgroups
under the hood and I think there is lot more sense doing it with this command instead of fiddling withcgroups
manually as suggested in other answers. This should have more upvotes. -
Ferroao over 4 yearsseems modern ubuntu requires cgroup-tools not cgroup-bin
-
skrat about 4 yearsThis is the easiest most sensible way IMO, thank you
-
EdiD almost 4 yearsVery good answer. Do you know is there a way not to kill the process after exceeding a limit but rather give a restriction to max memory usage ?
-
Hi-Angel almost 4 years@EdiD I don't know a systemd option for that, but I presume disabling OOM-killer system-wide may make apps to receive
ENOMEM
instead of being killed. But in practice I can't think of much projects beside systems kernels that can gracefully handle OOM condition. Even those pure C projects that keep checking for malloc results, I don't they ever being tested in OOM conditions. So they'll misbehave, most likely crash. With that said, please check if you find MemoryHigh more useful for your usecase -
EdiD almost 4 years@Hi-Angel thanks I have just found this option too and testing now ...
-
Simon Boehm over 3 yearsOf note: If you use the
--user
option but don't have cgroupsv2 enabled (which is still the default in many distros), systemd-run will fail to set a memory limit but won't throw any errors. Github issue -
Ben Slade about 3 yearsThe ulimit on my system doesn't have any -S option. I wonder what that was for? The -v500000 parameter is said to specify 500 mbytes. My "man" page for ulimit doesn't specify any units for the -v parameter. I'm guessing -s takes a kbytes param. In that case, in the answer here, I'm guessing it was supposed to be -v512000 to specify 500 mbytes.
-
Mikko Rantalainen over 2 yearsPoettering has an opinion that if your system is running
systemd
no other apps should be messing withcgroup
s. As a result, you should be usingsystemd-run --user --pty -p ... ...
but unless your system is already runningcgroupv2
all memory limits will fail silently and no limitations apply. For example, you need Ubuntu 21.10 or greater in practice. -
JanKanis over 2 years@mikko you're right, see also the first comment line of the script. When I wrote the scripts originally the right tools didn't exist yet. It still works for me but maybe it's time to figure out how systemd-run works.
-
Mikko Rantalainen over 2 yearsIt's my understanding that
systemd
session handling may fail if you run anything that touches thecgroup
hierarchies. For most cases you should be fine on single user desktop system. And in my experience, at least Ubuntu 18.04 has many other bugs that cause some processes to leak out of session handling so it's not perfect in any case. -
JanKanis over 2 years@mikko I have never noticed any session handling problems, but I run this on a single user desktop. For my use case the risk is an acceptable price for my whole system not crashing due to OOM.
-
Volomike over 2 yearsSort of related -- do you know how to slow down I/O resource usage with it?
-
Hi-Angel over 2 years@Volomike there's a number of IO-related hooks.
IOWeight
might be something you're interested in. Please see the documentation -
Admin almost 2 years
system-run
now appears to work perfectly on Ubuntu 22.04 without requiring modifying kernel params: unix.stackexchange.com/a/536046/32558 I wonder how. But it does. That is the way to go now. -
Admin almost 2 years@Ferroao I updated that now.
-
Admin almost 2 years
/sys/fs/cgroup/memory
is not being created for me on Ubuntu 22.04. I have setcgroup_enable=memory swapaccount=1
. -
Admin almost 2 years@CiroSantilli are you sure the command really worked for you? Unfortunately
--user
is just ignored when the kernel parameter is not set, so this may have silently failed on you and you just didn't notice. You can check if it works by limiting an application to ridiculously low amount of memory, e.g. 1K, and seeing if it fail to launch (which it should). -
Admin almost 2 yearsI've just retested, and it does seem to work, a C hello world gets killed with
systemd-run --scope -p MemoryMax=1 --user ./hello.out
or1K
, but runs fine withsystemd-run --scope -p MemoryMax=1M --user ./hello.out
. Andsudo dmesg | less
shows CLI:Command line: BOOT_IMAGE=/BOOT/ubuntu_uvs1fq@/vmlinuz-5.15.0-27-generic root=ZFS=rpool/ROOT/ubuntu_uvs1fq ro
. I did get the notification here as well BTW :-) -
Admin almost 2 years(1) You seem to be contradicting Hi-Angel’s answer. If so, you should say so. (2) This doesn’t seem to be an actual answer, but just commentary on one or more of the existing answer(s).
-
Admin almost 2 yearsAlso, please don’t post nearly-identical answers to multiple questions within Stack Exchange without linking between them.