How to increase limits on sockets on osx for load testing?

Question

I'm creating a load tester that uses libev to create lots of open tcp connections to an app i'm working on. Currently it bombs out at 256 connections, due to the nofiles limit:

ulimit -n
256

I can increase this to 1024 by doing the below:

ulimit -n 1024

But i cannot increase it further. Ideally i want to set it to 1048576. It gives the following error:

ulimit: open files: cannot modify limit: Invalid argument

How can i increase the ulimit further on osx?

Solution 1

(answer updated to use -S as several commenters suggested)

$ sysctl kern.maxfiles
kern.maxfiles: 12288
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 10240
$ sudo sysctl -w kern.maxfiles=1048600
kern.maxfiles: 12288 -> 1048600
$ sudo sysctl -w kern.maxfilesperproc=1048576
kern.maxfilesperproc: 10240 -> 1048576
$ ulimit -S -n
256
$ ulimit -S -n 1048576
$ ulimit -S -n
1048576

Solution 2

One more thing: Limit on ports is 65535. So you may not get as many as you want to.

Solution 3

Try running as root (e.g. do a "sudo -s" before running the ulimit command and your program).

Note that I'm not sure that 1-million-plus TCP sockets at once is realistically achievable (although I'm interesting in hearing about what happens when you try it ;^))

Also, check out this.

Comments

[Chris]

Well, 1M connections might not be realistic, but we'll see how far i can go. Just don't want to be artificially limited by ulimits.

[Chris]

Odd - i tried that on my home mac (snow leopard) and it didn't work, but on my work mac (also snow leopard) it worked fine. Hmm...

[Nathan Long]

Can you elaborate on these settings? I assume maxfilesperproc means "per process," and it makes sense it has to be less than maxfiles, but is there any reason you chose that specific number?

[Grrrr]

The original question mentions "Ideally i want to set it to 1048576.". The maxfiles was choosen as "something slightly larger than 1048576", there was no reason for it to be this particular value.

[Nathan Long]

I just found this article that explains more about these settings: krypted.com/mac-os-x/maximum-files-in-mac-os-x

[Nathan Long]

Also, I had to use ulimit -S -n 2048 to see it change.

[Dmytro Zavalkin]

Doesn't work for os x 10.8.2 :( However, with -S option it works. Please update your answer.

[yetanothercoder]

seems not working on 10.7.5, it's again reset to 1024, tried even from sudo -s

[Bob Ebert]

Is there a way to make it persistant, because it resets when I open a new window.

[Grrrr]

Bob Ebert: the sysctl part should be permanent until reboot. The ulimit part can be made permanent by putting it into your shell profile file, such as /etc/bashrc, /etc/zshrc, ~/.bashrc, ~/.zshrc. Alternatively, you can do the equivalent of ulimit from within your program by using the setrlimit(2) syscall.