Python on Snow Leopard, how to open >255 sockets?

Question

Consider this code:

import socket
store = []
scount = 0
while True:
    scount+=1
    print "Creating socket %d" % (scount)
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    store.append(s)

Gives the following result:

Creating socket 1
Creating socket 2
...
Creating socket 253
Creating socket 254
Traceback (most recent call last):
  File "test_sockets.py", line 9, in <module>
  File     "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 159, in __init__
socket.error: (24, 'Too many open files')

Checking sysctl for the allowed number of open files gives:

$ sysctl -A |grep maxfiles
kern.maxfiles = 12288
kern.maxfilesperproc = 10240
kern.maxfiles: 12288
kern.maxfilesperproc: 10240

Which is way more than the 253 sockets I could successfully open...

Could someone please help me in getting this number up to over 500? I am trying to simulate a peer to peer network using real sockets (requirement), with only 50 simulated nodes and 5 outgoing and 5 incoming connections each, would give the number of 500 needed sockets.

By the way, running this same code under Linux gives me about 1020 sockets, which is more the way I like it.

Solution 1

You can increase available sockets with ulimit. Looks like 1200 is the max for non-root users in bash. I can get up to 10240 with zsh.

$ ulimit -n 1200
$ python sockets
....
Creating socket 1197
Creating socket 1198
Traceback (most recent call last):
  File "sockets", line 7, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 182, in __init__
socket.error: [Errno 24] Too many open files

Solution 2

Did you install XCode and the developer tools off the Snow Leopard install disk? I'm able to open way more ports than you're able to:

Creating socket 1
Creating socket 2
...
Creating socket 7161
Creating socket 7162
Creating socket 7163
Creating socket 7164
Creating socket 7165
Creating socket 7166
Traceback (most recent call last):
  File "socket-test.py", line 7, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/socket.py", line 159, in __init__
socket.error: (24, 'Too many open files')

sysctl shows me a lot more info then your output shows (even with the grep) but the four lines you have match mine exactly, so all I can think of is needing something from the dev tools on the disk.

Comments

[Tader]

I think I did install the Xcode which came with Snow Leopard. It says Version 3.2, 64-bit, Xcode IDE: 1610.0, Xcode Core: 1608.0. Could you tell me what "which python" says at your computer?