I'm designing a system to handle 10000 TCP connections per second, what problems will I run into?
Solution 1
This is commonly known as the c10k problem. That page has lots of good info on the problems you will run into.
Solution 2
you should be able to do it [ although that's probably bad idea ].
on resin appserv i can get ~5k req/sec on quad core 2.6ghz xeon. requests invoke simple servlet that reads 1 row from mysql and sends very small xml response.
test was done with
ab -n 10000 -c 16 http://some/url/
test results:
Concurrency Level: 16
Time taken for tests: 1.904 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 3190000 bytes
HTML transferred: 1850000 bytes
Requests per second: 5252.96 [#/sec] (mean)
Time per request: 3.046 [ms] (mean)
Time per request: 0.190 [ms] (mean, across all concurrent requests)
Transfer rate: 1636.42 [Kbytes/sec] received
but i think you'll be much better off using simple c program, surely without spawning new threads for each request. link from Greg Hewgill should give you good idea about it.
even during prolonged test i dont get any problems with connectivity [ mentioned half-opened sockets ]; test runs between two linux boxes connected over gigabit ethernet [ although as you see bandwidth is not a bottleneck ].
Solution 3
You may be interested in a Linux kernel limit I hit while load testing Apache. In my case, the kernel produced some useful error messages so my advice is write your program and if you seem to be hitting a limit, pay attention to the kernel logs.
Related videos on Youtube
![Admin](/assets/logo_square_200-5d0d61d6853298bd2a4fe063103715b4daf2819fc21225efa21dfb93e61952ea.png)
Admin
Updated on September 17, 2022Comments
-
Admin almost 2 years
I have a relatively new 8-core box running CentOS. I would like to develop a stats server that uses TCP. It's very simple, it accepts a TCP connection, increments a counter and closes the connection. The catch is it needs to do this at at least 10k requests a second. I'm suspecting CPU/Memory won't be a problem, but I'm more concerned about artificial limits (like half-open connections) that I might need to configure on my server to allow for this kind of volume. So, is this possible? Which settings should I be aware of? Will my NIC not be able to handle it?
-
Admin over 14 yearsmake sure not to spawn threads for every incoming connect, it'll kill performance
-
agsamek over 14 years+1 for reporting your final results here :)
-
-
sybreon over 14 yearsyea, good link!
-
fpmurphy over 14 yearsI agree. UDP would be much more lightweight
-
cmeerw over 14 yearsI would expect to see more/different problems than those mentioned on the c10k page. Establishing and closing 10k connections per second is different from having 10k open connections. Connections staying in the TIME_WAIT state would be one, hitting the backlog limit for a listening socket might be another. And I wouldn't be surprised if that use-case hasn't received as much profiling/optimisation in the kernel code than the more common 10k open connections case.
-
SaveTheRbtz over 14 yearsUDP has it's drawbacks, like sender and delivery verifications, so one should consider them before using UDP in production.
-
Nate about 12 yearsAre your connections closed after every response like the OPs? Is ab sending Connection:close header?