How to limit memory of a OS X program? ulimit -v neither -m are working

Question

My programs run out of memory like half of the time I run them. Under Linux I can set a hard limit to the available memory using ulimit -v mem-in-kbytes. Actually, I use ulimit -S -v mem-in-kbytes, so I get a proper memory allocation problem in the program and I can abort.

But... ulimit is not working in OSX 10.6. I've tried with -s and -m options, and they are not working.

In 2008 there was some discussion about the same issue in MacRumors, but nobody proposed a good alternative. The should be a way a program can learn it's spending too much memory, or setting a limit through the OS.

Solution 1

You can't. Apple can (using the ledger() system call, which is private), but you can't. I'm not entirely sure whether launchd's options work or not - certainly if it was still using the code visible in the last open source version (from 10.9.5), it wouldn't, because it calls setrlimit(), but it's been substantially rewritten since then, though I can't see it calling ledger(), which I'd expect it to if this was supposed to work.

Why? Because the RLIMIT_DATA and RLIMIT_AS options to setrlimit() don't actually do anything in current versions of XNU (the macOS kernel).

Solution 2

After struggling with this myself (with limited success), I have determined there seems to be two ways to do it...

You can setup a launchd item for your executable.. The important part of the plist is a section, such as..

<key>SoftResourceLimits</key>
<dict>
    <key>Stack</key>
    <integer>10000000000</integer>
</dict>

There are various keys available... which can be found on Apple's MAN page.

Another way to do it, I think, is by setting a value in either /etc/launchd.conf (system) or /etc/launchd-usr.conf (peruser). For example, your launchd.conf could contain...

umask 002
limit stack 67104768 67104768
limit maxproc 3400 4500
limit maxfiles 256 unlimited
setenv PATH /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

The documentation for all of launchd's functionality is shotty, if you ask me.. It's as if Apple might not care / want people outside their walls to actually understand how it all works. There is so much power to be had by mastering launchd and it's intricacies... but there are few concrete/official resources available as to how to properly implement them.

Solution 3

setrlimit should do the job. I believe that's the BSD equivalent of ulimit...

Comments

[JWWalker]

I looked at the man page for setrlimit, and couldn't see how to set a limit on virtual memory, though you can set a limit on physical memory.

[hectorpal]

I know bash ulimit is implemented directly calling setrlimit. In Linux man page there is the option RLIMIT_AS, that limits "the maximum size of the process’s virtual memory (address space) in bytes", that is what I want to control. I really don't care about the limit of physical memory. I want to know when the whole program is over 2Gb. Well, on OS X manpage for setrlimit, there is RLIMIT_AS. The nearest is RLIMIT_RSS. Indeed, IMHO OS X seems to allocate a lot of virtual memory, given what I see in the Activity Monitor.

[Yuji]

Mmm... SUSv3 demands RLIMIT_AS, (see opengroup.org/onlinepubs/009695399/functions/getrlimit.html) and OS X sells as an SUSv3 UNIX, so it should support RLIMIT_AS. Indeed <sys/resource.h> defines RLIMIT_AS, although it's not mentioned in the man page. Could you try RLIMIT_AS to see if it's really implemented?

[hectorpal]

It seems I should try... Thanks.

[ephemient]

There are varied reports on the Internet that setrlimit is just plain broken on Darwin (OS X). lists.apple.com/archives/Unix-porting/2007/Apr/msg00026.html

[chiggsy]

+1 I am up against launchd, and I can't let go of it. If it just worked in any consistent way, i would use it all the time.

[krlmlr]

See also the feature matrix in Section 7.11 of APUE -- it says clearly that limiting memory is not supported on OS X (at least not in 10.3, have to check current edition). Limiting virtual memory is supported in OS X, though...