How does one capture traffic on virtual interfaces?

Question

I would like to capture traffic on Linux virtual interfaces, for debugging purposes. I have been experimenting with veth, tun and dummy interface types; on all three, I am having trouble getting tcpdump to show anything.

Here is how I set up the dummy interface:

ip link add dummy10 type dummy
ip addr add 99.99.99.1 dev dummy10
ip link set dummy10 up

In one terminal, watch it with tcpdump:

tcpdump -i dummy10

In a second, listen on it with nc:

nc -l 99.99.99.1 2048

In a third, make an HTTP request with curl:

curl http://99.99.99.1:2048/

Although in terminal 2 we can see the data from the curl request, nothing shows up from tcpdump.

A Tun/Tap tutorial clarifies some situations where the kernel may not actually send any packets when one is operating on a local interface:

Looking at the output of tshark, we see...nothing. There is no traffic going through the interface. This is correct: since we're pinging the interface's IP address, the operating system correctly decides that no packet needs to be sent "on the wire", and the kernel itself is replying to these pings. If you think about it, it's exactly what would happen if you pinged another interface's IP address (for example eth0): no packets would be sent out. This might sound obvious, but could be a source of confusion at first (it was for me).

However, it is hard to see how this could apply to TCP data packets.

Maybe tcpdump should be bound to the interface a different way?

Solution 1

The traffic is going over the lo interface.

When an IP is added to a box, a route for that address is added to the 'local' table. All the routes in this table route traffic over the loopback interface.

You can view the contents of the 'local' table with the following:

ip route show table local

Which on my system looks like this:

local 10.230.134.38 dev tun0  proto kernel  scope host  src 10.230.134.38 
broadcast 10.230.134.38 dev tun0  proto kernel  scope link  src 10.230.134.38 
broadcast 127.0.0.0 dev lo  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  proto kernel  scope link  src 127.0.0.1 
broadcast 172.17.0.0 dev docker0  proto kernel  scope link  src 172.17.42.1 
local 172.17.42.1 dev docker0  proto kernel  scope host  src 172.17.42.1 
broadcast 172.17.255.255 dev docker0  proto kernel  scope link  src 172.17.42.1 
broadcast 192.168.0.0 dev enp6s0  proto kernel  scope link  src 192.168.0.20 
local 192.168.0.20 dev enp6s0  proto kernel  scope host  src 192.168.0.20 
broadcast 192.168.0.255 dev enp6s0  proto kernel  scope link  src 192.168.0.20 

So basically if I send any traffic to 10.230.134.38, 127.0.0.0/8, 127.0.0.1(redundant), 172.17.42.1, or 192.168.0.20, the traffic will get routed over the loopback interface, even though those IPs are really on a different interface.

Solution 2

You can use tcpdump with any interface on the host (tcpdump -i any ...)

Comments

[derobert]

Not sure why it's hard to see that happening to TCP packets. Like pings, those are processed in the kernel. Same thing is happening.

[solidsnack]

@derobert In the case of pings, the kernel can respond. In the case of data packets, I rather imagined they would have to go "over" the interface so the application could respond to them.

[derobert]

Applications speak to the kernel using read, write, etc. Many network apps don't even have to be aware that interfaces exist. To get traffic to go over one of them, the kernel needs to see it as non-local. E.g., set up an OpenVPN tunnel, then you can capture traffic going over tun0. (Of course, tun0 is a special way for apps to speak to the kernel, to implement a network interface in userland, which is what OpenVPN does.)