Linux bond with VLAN question

Question

Do you think following configuration make sense? Does BONDTING_OPT supported in VLAN interface? I want to make sure my interface fails over when the upstream device down.

ifcfg-bond0

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0
NAME=bond0
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
BONDING_OPTS="mode=1 miimon=500 downdelay=1000 primary=eno1 primary_reselect=always"

ifcfg-bond0.10

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0.10
NAME=bond0.10
DEVICE=bond0.10
ONPARENT=yes
BOOTPROTO=dhcp
VLAN=yes
BONDING_OPTS="mode=1 arp_interval=1000 arp_ip_target=10.10.0.1 miimon=500 downdelay=1000 primary=eno1 primary_reselect=always"
NM_CONTROLLED=no

ifcfg-bond0.20

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0.20
NAME=bond0.20
DEVICE=bond0.20
ONPARENT=yes
BOOTPROTO=dhcp
VLAN=yes
BONDING_OPTS="mode=1 arp_interval=1000 arp_ip_target=74.xx.xx.1 miimon=500 downdelay=1000 primary=eno1 primary_reselect=always"
NM_CONTROLLED=no

Solution

You need to configure the devices, then the bond and finally the VLAN config files. You also have to pay attention on what attributes work where, such as the bonding_opts, which can only be in the bond its self, not the VLAN config files as the VLAN files won't be able to access the Ethernet connections directly.

I have also found this RHEL 6 document that explains it more. It's also still mostly reliable for RHEL 7 / CentOS 7.

• CONFIGURING A VLAN OVER A BOND

Added:

The bond is responsible for the management of the physical connections the bond uses. the VLANs have no visibility into the physical connections as shown from the driver output below; they only see the device responsible for them, in this case, bond0.

Also explained here for another point of view: bonded-and-primary-virtual-ip-addresses-and-vlan-tagged Answer

Notes:

• I know this works as I have it working on a RHEL 7 system.
• If you need more VLAN's, just cp ifcfg-bond0.20 ifcfg-bond0.30 and update the required fields.

Diagram:

eth0         vlan1
    \       / 
      bond0 -vlan2
    /       \
eth1         vlan3

Example:

[Working config]

 ==> ifcfg-eth0 <==
 DEVICE=eth0
 NAME=bond0-slave
 HWADDR=xx:xx:xx:xx:xx:xx
 ONBOOT=yes
 BOOTPROTO=none
 MASTER=bond0
 SLAVE=yes
 USERCTL=no
 NM_CONTRLLED=no

 ==> ifcfg-eth1 <==
 DEVICE=eth1
 NAME=bond0-slave
 HWADDR=xx:xx:xx:xx:xx:xx
 ONBOOT=yes
 BOOTPROTO=none
 MASTER=bond0
 SLAVE=yes
 USERCTL=no
 NM_CONTRLLED=no

 ==> ifcfg-bond0 <==
 DEVICE=bond0
 TYPE=bond
 BONDING_MASTER=yes
 NAME=bond0
 ONBOOT=yes
 BONDING_OPTS="miimon=100 mode=active-backup"

 ==> ifcfg-bond0.10 <==
 VLAN=yes
 TYPE=vlan
 DEVICE=bond0.10
 PHYSDEV=bond0
 VLAN_ID=10
 BOOTPROTO=none
 NAME=bond0.10
 ONBOOT=yes
 IPADDR=x.x.x.x
 PREFIX=24
 GATEWAY=x.x.x.1
 IPV6INIT=no
 DEFROUTE=yes

 ==> ifcfg-bond0.20 <==
 VLAN=yes
 TYPE=vlan
 DEVICE=bond0.20
 PHYSDEV=bond0
 VLAN_ID=20
 BOOTPROTO=none
 NAME=bond0.20
 ONBOOT=yes
 IPADDR=x.x.x.x
 PREFIX=24
 GATEWAY=x.x.x.1
 IPV6INIT=no
 DEFROUTE=no

drivers in use:

[thebtm@server network-scripts]$ sudo cat /proc/net/bonding/bond0 
 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

 Bonding Mode: fault-tolerance (active-backup)
 Primary Slave: None
 Currently Active Slave: eth0
 MII Status: up
 MII Polling Interval (ms): 100
 Up Delay (ms): 0
 Down Delay (ms): 0

 Slave Interface: eth0
 MII Status: up
 Speed: 10000 Mbps
 Duplex: full
 Link Failure Count: 0
 Permanent HW addr: xx:xx:xx:xx:xx:xx
 Slave queue ID: 0

 Slave Interface: eth1
 MII Status: up
 Speed: 10000 Mbps
 Duplex: full
 Link Failure Count: 0
 Permanent HW addr: xx:xx:xx:xx:xx:xx
 Slave queue ID: 0

[thebtm@server network-scripts]$ sudo cat /proc/net/vlan/bond0.10
bond0.10    VID: 10      REORDER_HDR: 1  dev->priv_flags: 1
         total frames received  29091167441
          total bytes received 139953896100912
      Broadcast/Multicast Rcvd           18

      total frames transmitted  21506143557
       total bytes transmitted 14822425401382
Device: bond0
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings: 
[thebtm@server network-scripts]$ sudo cat /proc/net/vlan/bond0.20
bond0.20    VID: 20      REORDER_HDR: 1  dev->priv_flags: 1
         total frames received      2637498
          total bytes received    290061293
      Broadcast/Multicast Rcvd            5

      total frames transmitted            6
       total bytes transmitted          252
Device: bond0
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
 EGRESS priority mappings: 

Comments

[Satish]

If VLAN doesn't support BONDING_OPTS then How does bonding will detect in above scenario if my uplink trunk is down?

[thebtm]

That is the responsibility of the bond and not the vlan config files. the VLAN config files will point to the bond for its traffic, then the BOND is responsible for how the traffic is managed on the physical connections based on your configurations in BONDING_OPTS. you have to look at the connection in layers. Physical, Bonds, VLANs as shown in my text diagram.