UDP packets seen on interface level but not delivered to application on RedHat

Question

I have a problem with receiving UDP traffic on a RHEL6 server. When capturing the traffic with tcpdump, the packets are visible (checksums are correct, the destination IP and MAC match the machine's ones). However the application listening on the particular port fails to see any data, even though the socket is successfully open and shows up in netstat -panu output. I tried a simple perl UDP server script (http://www.perlmonks.org/?node_id=659588) instead of the actual application, but the result is the same (socket open, no data received).

No firewall rules are configured. Here's the iptables --list output:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

Also netstat -s -u doesn't show any UDP packets received by the system:

Udp:
    0 packets received
    0 packets to unknown port received.
    0 packet receive errors
    0 packets sent

This is some secure customer environment and I can't log in to another machine in the network to try sending test UDP packets manually.

Could anyone point me in a direction to troubleshoot it further?

Solution

Check if this helps. Linux Kernel not passing through multicast UDP packets Basically if net.ipv4.all.rp_filter = 1, then kernel performs Strict Reverse Path validation. More documentation can be found at https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt

Comments

[iSee]

As a side note, nc -u -l 5000 would have been enough for a simple UDP server.

[thrig]

Hmm, that code should listen on *:urport though confirming that with netstat or the like might be good. Also, is there anything interesting in the routing table that might be sending the packets elsewhere? ip r or ip -6 r should show that.

[wojciech_k]

Netstat confirms that the application is listening on *:port. There are no routing rules defined.

[iSee]

Have you tried stracing your (dummy) server?

[DarkMoon]

Welcome to ServerFault. This is intended to be a (mostly) standalone answer site, so it's preferred if you put the relevant bits from the link into your answer. Too many links go dead and the answers therein are lost.