RHEL 6 KVM NIC configuration with bonding + vlan + bridge

Question

We are having trouble with our bonded NICs + vlan + bridged network configuration. We have a blade server running RHEL6, and are trying to set up the networking for KVM virtualisation.

The blade server has 2 NICs (p2p1 and p2p2) which connect to the internal blade chassis 10 Gb switches. We want to bond the 2 blade NICs and setup

1. default vlan (vlan 1) for our production network (172.16.x.x) assigned via DHCP, and
2. vlan 3 on our storage network (192.168.10.x) statically assigned.

There is an added complication that the KVM guests need to connect to our production network, and so we also need a bridge on the production network.

Setting up the bonding (as described here) together with vlan 3 (as described here) seems fine. We used the following configuration files

DEVICE=”p2p1”
BOOTPROTO=”none”
HWADDR=[Removed]
NM_CONTROLLED=”no”
ONBOOT=”yes”
TYPE=”Ethernet”
UUID=[Removed]
MASTER=bond0
SLAVE=yes
USERCTL=no

DEVICE=”p2p2”
BOOTPROTO=”none”
HWADDR=[Removed]
NM_CONTROLLED=”no”
ONBOOT=”yes”
TYPE=”Ethernet”
UUID=[Removed]
MASTER=bond0
SLAVE=yes
USERCTL=no

DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

DEVICE=bond0.3
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.10.22
NETMASK=255.255.255.0
USERCTL=no
VLAN=yes

The problem comes when we try to set up a bridged network connection for the default vlan (vlan 1). For example we might try modifying bond0 and adding br0 as described here.

DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
BRIDGE=br0

DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0

However, br0 never obtains an IP over DHCP. We know it's not a DHCP issues since DHCP works fine if we set it up directly on bond0, but that's not a bridge interface so the VMs can't use it.

How should we configure the interfaces for both vlan 3 (storage) and default vlan for KVM guests and administration?

Solution

An incompatibility between Cisco Rapid Spanning Tree and our blade centre switches led to our network engineers changing the design, so the question no longer applies. However, I found the following discussions informative and they may help others in future.

• Similar setup, which was eventually confirmed as working here.
• A known issue with ARP with bridges on bonded interfaces here.

Comments

[David Corsalini]

Can you post the versions in use? Kernel, RHEL version, kvm/qemu/libvirt, what hardware is in use (lspci)

[David Corsalini]

The scheme you describe should work, but if you mentioned the versions, as requested, it would be possible to point you to the culprit.