ICMP time exceeded in-transit

Question

In the last few days my server suffers an attack of this kind: (bandwith > 60MBit/s, XXX.XXX.XXX.XXX are multiple IPs)

    tcpdump -n proto ICMP    
    17:15:19.267464 IP XXX.XXX.XXX.XXX > my_ip: ICMP time exceeded in-transit, length 36
    17:15:19.325217 IP XXX.XXX.XXX.XXX > my_ip: ICMP time exceeded in-transit, length 36
    17:15:19.345561 IP XXX.XXX.XXX.XXX > my_ip: ICMP time exceeded in-transit, length 56
    17:15:19.484865 IP XXX.XXX.XXX.XXX > my_ip: ICMP time exceeded in-transit, length 36
    17:15:19.529616 IP XXX.XXX.XXX.XXX > my_ip: ICMP time exceeded in-transit, length 36
    17:15:19.957058 IP XXX.XXX.XXX.XXX > my_ip: ICMP YYY.YYY.YYY.YYY tcp port 39692 unreachable, length 36
    17:15:19.968957 IP XXX.XXX.XXX.XXX > my_ip: ICMP host YYY.YYY.YYY.YYY unreachable, length 56
    17:15:20.112520 IP XXX.XXX.XXX.XXX > my_ip: ICMP host YYY.YYY.YYY.YYY unreachable, length 56
    17:15:20.203199 IP XXX.XXX.XXX.XXX > my_ip: ICMP host YYY.YYY.YYY.YYY unreachable, length 36
    17:15:20.204803 IP XXX.XXX.XXX.XXX > my_ip: ICMP host YYY.YYY.YYY.YYY unreachable, length 36

I've FreeBSD 9.1 and my pf.conf is

ext_if="em0"
table <blockedips> persist file "/etc/pf-blocked-ips.conf"

set skip on lo0

block drop in log (all) quick on $ext_if from <blockedips> to any

block in
pass out flags S/SA keep state

pass in on $ext_if proto tcp to port 80 flags S/SA keep state
pass in on $ext_if proto tcp to port ssh flags S/SA synproxy state

There's anything that i can do?

Solution

It is astoundingly unlikely that you are actually seeing over 60Mbit/sec of nothing but ICMP replies indicating dropped packets. If you are, chances are you are either under DoS, or there is some kind of runaway process sending spurious traffic from your host.

A good first step is to capture a sample of the traffic (use pcap or wireshark or tcpdump or something), and analyze it; see if you are actually sending packets that correspond with the ICMP replies. You need to capture more than just the ICMP traffic. If you are sending stuff related to it, there are two things to do:

• Determine why you are sending the traffic at such a high velocity or with excessive retries
• Determine if the process sending the traffic should be running at all (netstat can sometimes help with this, using the -p option)
• Use a traceroute utility to isolate possible causes of routing issues and correct them (or ask your ISP to correct them)

If you are not sending traffic that generates these replies and the ICMP traffic is overwhelming your link, you need to get in touch with your ISP and ask them to help you mitigate it.

Comments

[user16081-JoeT]

try blocking all incoming ICMP packets. I think this is the syntax you need: block in on $ext_if inet proto icmp all -- though there is probably more happening to trigger these requests, I'd guess your system is compromised.

[mattia]

ICMP requests are already blocked with the "block in" rule.

[user16081-JoeT]

why is your system sending so many ICMP requests? Did you look in /tmp folder for suspicious files?

[mattia]

My server is receiving this requests, it doesn't send anything.

[user16081-JoeT]

are you sure? contact your ISP and have them filter this at the router.

[citrin]

ICMP time exceeded in-transit is sign of routing loop.