Linux bridging for KVM

Question

I have a Debian Wheezy machine on which I tried to setup KVM with bridged networking. Unfortunately, the bridge does not appear to forward traffic correctly.

My setup is the following:

The physical machine has an eth0 that is connected to a router with the IP address 192.168.0.1.
This eth0 is a member of the bridge interface br0, which is configured statically to the IP address 192.168.0.101.
The virtual machine's network interface is vnet0 on the physical host and eth0 in the virtual machine. In the virtual machine, the interface is configured statically to the IP address 192.168.0.110.

From the physical host, I can ping both the virtual machine and the router:

# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_req=1 ttl=64 time=0.331 ms

# ping 192.168.0.110
PING 192.168.0.110 (192.168.0.110) 56(84) bytes of data.
64 bytes from 192.168.0.110: icmp_req=1 ttl=64 time=0.417 ms

From the virtual machine I can ping the physical machine:

# ping 192.168.0.101:
PING 192.168.0.101 (192.168.0.101) 56(84) bytes of data.
64 bytes from 192.168.0.101: icmp_req=1 ttl=64 time=0.133 ms

But I can't ping the router:

# ping 192.168.0.1:
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
From 192.168.0.110 icmp_seq=10 Destination Host Unreachable

From what I can tell, the bridge is setup properly and all attached ports are in their forwarding state:

# brctl show
bridge name bridge id           STP enabled interfaces
br0         8000.2cd444acf8ad   no          eth0
                                            vnet0
# brctl showstp br0
br0
 bridge id               8000.2cd444acf8ad
 designated root         8000.2cd444acf8ad
 root port               0                    path cost        0
 max age                 20.00                bridge max age           20.00
 hello time              2.00                 bridge hello time         2.00
 forward delay           0.00                 bridge forward delay      0.00
 ageing time             300.01
 hello timer             1.20                 tcn timer                 0.00
 topology change timer   0.00                 gc timer                 28.95
 flags          


eth0 (1)
 port id            8001                state            forwarding
 designated root    8000.2cd444acf8ad   path cost              4
 designated bridge  8000.2cd444acf8ad   message age timer      0.00
 designated port    8001                forward delay timer    0.00
 designated cost       0                hold timer             0.20
 flags          

vnet0 (2)
 port id            8002                state            forwarding
 designated root    8000.2cd444acf8ad   path cost            100
 designated bridge  8000.2cd444acf8ad   message age timer      0.00
 designated port    8002                forward delay timer    0.00
 designated cost       0                hold timer             0.20
 flags          

Both iptables and ebtables are empty with a default policy of ACCEPT:

# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination   
# ebtables -L
Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

Does anyone have an idea what causes this problem and how I can solve it?

Edit:

Apparently, the bridge recognizes that there is something connected to it, as it builds a MAC table:

# brctl showmacs br0
port no mac addr            is local?   ageing timer
  1     2c:76:8a:ff:88:1d   no            1.28   <- ???
  1     2c:d4:44:ac:f8:ad   yes           0.00   <- MAC of br0 and physical eth0
  2     52:54:00:53:dd:34   no          143.80   <- MAC of VM's eth0
  1     c8:1f:66:ba:83:33   no            0.00   <- MAC of router interface
  2     fe:54:00:53:dd:34   yes           0.00   <- MAC of vnet0

Edit 2:

I just created a second virtual machine. This machine has the interface vnet1 on the physical machine, its virtual eth0 was assigned the IP address 192.168.0.111. This VM can also only ping the physical machine and neither the router nor the original VM. brctl showstp shows all ports, including vnet1, in forwarding state and brctl showmacs shows the MACs of vnet1 and the new machine's virtual eth0 in addition to what I wrote above.

Solution

Check that the kernel is set to enable IP forwarding:

sysctl -a | grep forwarding

You can enable with:

sudo sysctl net.ipv4.conf.all.forwarding=1
sudo sysctl net.ipv6.conf.all.forwarding=1

There may also be an issue with ARP proxying. Check with:

sysctl -a | grep proxy_arp

And set with the command:

sudo sysctl net.ipv4.conf.eth0.proxy_arp=1

You can put the keys and values in a file under /etc/sysctl.d to have the values reset on reboot.

Testing from another device on the router's subnet may help determine the problem.

• Pinging the virtual machine may provide useful diagnostics.
• Checking if you can ARP for the virtual machine will indicate if you can find the MAC address for the server. Use "arp -a" after pinging it to see if the MAC address is successfully found.
• Traceroute may indicate where the problem starts.

Testing with tcpdump on the eth0 interface may also indicate where the connection is failing.

• Repeated arp requests without a valid response indicates a reachability issue.
• Missing echo or echo reply traffic may indicate which side has an issue.
• Traceroute responses to the router or addresses behind it may provide additional information.

Comments

[Michael Hampton]

You have STP off in your bridge. Try turning it back on.

[BillThor]

@tonioc I would expect the virtual server to respond to ARP requests. I have multiple bridge interface, but none of them are bridged to the external interface. Proxy ARP is required for them to be visible.

[Biplab1985]

After a little more digging, it turns out it actually was a ARP proxying and IP forwarding issue. But more sysctls had to be set to 1:

[Biplab1985]

After a little more digging, it turns out it actually was a ARP proxying and IP forwarding issue. But more sysctls had to be set to 1: net.ipv4.conf.default.proxy_arp, net.ipv4.conf.default.proxy_arp_pvlan, net.ipv4.conf.all.forwarding, and net.ipv6.conf.all.forwarding. Then, ping 192.168.0.1 is possible although it prints some "Redirect host" messages for a while. After about 50 transmitted packets those messages cease and ping works as expected. Also, STP has to be activated on the bridge or it did not work stable.