No Default Gateway route for bond0 / Slave eth* interface gets IP assigned

Question

I have set up a 4-NIC bond on my Debian Wheezy box. After a fresh boot, the system has LAN access, but cannot resolve outside host names or access outside IPs. It seems that this is because the bond0 interface is not getting a default gateway route, but instead one of the slave interfaces (usually eth3) is getting it.

Result of route -n after booting:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.2.1     0.0.0.0         UG    0      0        0 eth3
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eth3
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 bond0
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth3

My immediate network problem could be alleviated by doing: route add default gw 192.168.2.1 bond0. But I at least need the computer to get outside network access after a reboot when I may not have physical access.

The supposedly slaved interface is getting its own IP address every single time. I suspect that may be the root of the problem. What is going on here? Result of ifconfig, note eth3:

bond0     Link encap:Ethernet  HWaddr bc:5f:f4:be:4a:cc
          inet addr:192.168.2.3  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::be5f:f4ff:febe:4acc/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:522992 errors:0 dropped:4871 overruns:0 frame:0
          TX packets:885760 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:53459302 (50.9 MiB)  TX bytes:736701670 (702.5 MiB)

eth0      Link encap:Ethernet  HWaddr bc:5f:f4:be:4a:cc
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:85101 errors:0 dropped:5 overruns:0 frame:0
          TX packets:412890 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:8568619 (8.1 MiB)  TX bytes:37311255 (35.5 MiB)
          Memory:f7400000-f7480000

eth1      Link encap:Ethernet  HWaddr bc:5f:f4:be:4a:cc
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1285 errors:0 dropped:12 overruns:0 frame:0
          TX packets:75 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:185579 (181.2 KiB)  TX bytes:16930 (16.5 KiB)
          Memory:f7300000-f7380000

eth2      Link encap:Ethernet  HWaddr bc:5f:f4:be:4a:cc
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:2039 errors:0 dropped:11 overruns:0 frame:0
          TX packets:472583 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:326122 (318.4 KiB)  TX bytes:699347095 (666.9 MiB)
          Memory:f7200000-f7280000

eth3      Link encap:Ethernet  HWaddr bc:5f:f4:be:4a:cc
          inet addr:192.168.2.135  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:434565 errors:0 dropped:410 overruns:0 frame:0
          TX packets:210 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:44378800 (42.3 MiB)  TX bytes:26282 (25.6 KiB)
          Memory:f7100000-f7180000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:78 errors:0 dropped:0 overruns:0 frame:0
          TX packets:78 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9310 (9.0 KiB)  TX bytes:9310 (9.0 KiB)

and /etc/network/interfaces is like so:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
bond-master bond0
bond-mode 4

auto eth1
iface eth1 inet manual
bond-master bond0
bond-mode 4

auto eth2
iface eth2 inet manual
bond-master bond0
bond-mode 4

auto eth3
iface eth3 inet manual
bond-master bond0
bond-mode 4

auto bond0
iface bond0 inet static
address 192.168.2.3
netmask 255.255.255.0
gateway 192.168.2.1
dns-nameservers 192.168.2.1
slaves eth0 eth1 eth2 eth3
bond-slaves none
bond-mode 4
bond-miimon 100
bond-downdelay 200
bond-updelay 200

I have tried several combinations of ports on my switch, and each time come up with roughly the same result (sometimes a different interface than eth3). This problem did arise after upgrading my motherboard; with the old board I had a 2-NIC bond, which worked fine. (Is there any possible issue with upgrading hardware on an existing installation of Wheezy?) After upgrading, I removed the old /etc/udev/rules.d/70-persistent-net.rules file, and let it regenerate, so the file now contains:

# PCI device 0x8086:/sys/devices/pci0000:00/0000:00:1c.2/0000:07:00.0 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="bc:5f:f4:be:4a:cc", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:/sys/devices/pci0000:00/0000:00:1c.4/0000:09:00.0 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="bc:5f:f4:be:4a:bd", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x8086:/sys/devices/pci0000:00/0000:00:1c.3/0000:08:00.0 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="bc:5f:f4:be:4a:ce", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:/sys/devices/pci0000:00/0000:00:1c.5/0000:0a:00.0 (igb)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="bc:5f:f4:be:4a:bf", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

One last thing, I did try adding the following to my /etc/network/interfaces file, hoping to get a sort of 'work-around' solution.

auto bond0
...
post-up route add default gw 192.168.2.1 bond0
pre-down route del default gw 192.168.2.1 bond0

That seems to make no difference, and after a boot, there is no default GW route for bond0 in the routing table (it is exactly as above). I would love to know what I'm doing wrong there, and at least use that as an interim fix.

Solution

Well that led me to the easy answer: update-rc.d network-manager disable and then reboot. I had no idea NetworkManager was still running at all, but it was, and it was assigning the IP.

Comments

[wurtel]

Somehow eth3 is getting the address 192.168.2.135 (and the default route). How?! It looks like there may be some dhcp client running...

[Gavin Stansill]

For some reason it hadn't occurred to me to check this before, but looking at ps -ax|grep dhcp i see this: 3757 ? S 0:00 /sbin/dhclient -d -4 -sf /usr/lib/NetworkManager/nm-dhcp-client.action -pf /var/run/dhclient-eth3.pid -lf /var/lib/dhcp/dhclient-c39bf91e-a818-42fa-8352-831c69c05902-‌​eth3.lease -cf /var/run/nm-dhclient-eth3.conf eth3. So NetworkManager is maybe mucking things up? How can I just disable that completely?