No buffer space available and tuning with sysctl

Question

Not really a network guy so please bear with me.

I'm currently hosting a web application that sends UDP to all IP addresses in it's subnet periodically. That's 2048 IPs. My problem is it throws No buffer space available after sometime.

I adjusted some values using sysctl namely:

• net.ipv4.neigh.default.gc_thresh1 = 2048
• net.ipv4.neigh.default.gc_thresh2 = 4096
• net.ipv4.neigh.default.gc_thresh3 = 8192

And it worked, but severely slows down the response time of the server. It's written in Java and running in a fit-pc2 (1.1ghz, 1gb of RAM, SD card) Oh the horror.

Given the limited resources, how do I optimize the server/kernel parameters to address the No buffer space available problem?

Solution

It's not the sysctl setting as such, your server is slowing down as it has to continuously scan through an arp table with 2k entries.

Things worth trying are:

• increasing the default "arp cache timeout" from 60s to 3600s. This will lower the average arp-request rate from 34 req / s to 0.5 req / s.
• adding all interesting MAC addresses statically to the arp table

However, your best option is to separate your server from the 2k hosts by a router.

• If you can change the IP from your server just put it in another subnet
• If you can't change the IP you can split a /30 subnet of the /21 subnet and use a proxy-arp feature to glue them

Comments

[Bro Kevin D.]

Hey, thanks for the explanation and tips. I'll try those out. But it's rather interesting to point out that I've enabled net.ipv4.conf.all.arp_filter after reading austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.arp_problem.html and the issue seems to go away. I will have to do more research. Excuse my ignorance.