Can only bring up one of two interfaces

Question

I'm having a bizarre issue with my HP Proliant DL 360 G4p server. It has two gigabit ethernet interfaces but I can bring up only one of them. This is starting to freak me out and that's why I turned here. I'm running the x64 ubuntu 11.10 server edition.

lshw -c network shows that the second interface is disabled. I have no idea why ans how to enable it.

$ sudo lshw -c network
  *-network:0
       description: Ethernet interface
       product: NetXtreme BCM5704 Gigabit Ethernet
       vendor: Broadcom Corporation
       physical id: 2
       bus info: pci@0000:02:02.0
       logical name: eth0
       version: 10
       serial: 00:18:71:e3:6d:26
       size: 100Mbit/s
       capacity: 1Gbit/s
       width: 64 bits
       clock: 66MHz
       capabilities: pcix pm vpd msi bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=tg3 driverversion=3.119 duplex=full firmware=5704-v3.27b, ASFIPMIc v2.36 ip=10.48.8.x latency=64 link=yes mingnt=64 multicast=yes port=twisted pair speed=100Mbit/s
       resources: irq:25 memory:fdf70000-fdf7ffff
  *-network:1 DISABLED
       description: Ethernet interface
       product: NetXtreme BCM5704 Gigabit Ethernet
       vendor: Broadcom Corporation
       physical id: 2.1
       bus info: pci@0000:02:02.1
       logical name: eth1
       version: 10
       serial: 00:18:71:e3:6d:25
       capacity: 1Gbit/s
       width: 64 bits
       clock: 66MHz
       capabilities: pcix pm vpd msi bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=tg3 driverversion=3.119 firmware=5704-v3.27b latency=64 link=no mingnt=64 multicast=yes port=twisted pair
       resources: irq:26 memory:fdf60000-fdf6ffff

If I try to ifup eth1, then I get

$ sudo ifup eth1
Ignoring unknown interface eth1=eth1.

I figured that's what happens when there is no eth1 listed in /etc/network/interfaces. But when I add the configuration for eth1, I still can't ifup.

$ sudo ifup eth1
RTNETLINK answers: File exists
Failed to bring up eth1.
I've also tried ifconfig eth1 up but without any result.

For clarity, I have added a masked version of /etc/network/interfaces. I don't think it is the cause of the problem though.

$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 10.48.8.x
        netmask 255.255.255.y
        network 10.48.8.z
        broadcast 10.48.8.t
        gateway 10.48.8.u

auto eth1
iface eth1 inet static
        address   193.190.253.x
        netmask   255.255.255.y
        network   193.190.253.z
        broadcast 193.190.253.t
        gateway   193.190.253.u

I really need some help fixing this. It's driving me crazy. Thanks.

Solution 1

I found the solution. There were two gateways defined in /etc/network/interfaces while you simple can't have more then one gateway. It makes no sense.

A gateway is an IP you send ALL traffic to. If you would have two, your routing table would have a double entry for dest 0.0.0.0 and the system can't handle this. The double route is what causes the RTNETLINK answer "File exists", meaning there is already a route for 0.0.0.0.

I've commented out one of the gateways and now I can ifup both eth0 and eth1.

tl;dr delete gateway entries until you've got only one left.

Solution 2

From this link i learned a lot it helped me fixing the error.

Just add the following line to your /etc/network/interfaces below your eth0 config:

pre-up ip addr del 192.168.1.254/24 dev eth0 2> /dev/null || true

It's not the most beautiful solution but ad least it works. Credits go to the owner of grapsus.net and google.

Comments

[Allan Jude]

Can you provide the contents of both your /etc/network/interfaces eth0 and eth1

[Kyle Brandt]

In the past I made the mistake of copying an interface config but not changing the MAC address with CentOS, not sure if that applies here, would need to see /etc/network/interfaces as Allan said

[mstaessen]

done. Don't think it's very relevant though. Seems correct to me :)

[mstaessen]

Cables are perfectly fine. I can bring up either eth1 or eth0 but never both. Both NICs have already worked so that won't probably be it.

[mstaessen]

The ilo port is not visible to the operating system, so no. There are three ethernet cables plugged in. All working fine. The problem is the NIC itself. It won't allow me to activate the two interfaces simultaneously.

[SimonJGreen]

Just FYI, you can have multiple gateways, just not 2 DEFAULT gateways. That's exactly what the routing table is for, to define the gateway for different destinations. A default gateway is simply the final match in the list.

[Lisandro]

Look out! On Ubuntu 12.04 (The only one I tested this in) a full reboot is required if you miss configure the second NIC.

[mstaessen]

Don't think I had that issue. /etc/init.d/networking restart works fine

[Michael]

I add post-up ip route delete default via 10.0.0.1 after the iface declaration in /etc/network/interfaces, changing 10.0.0.1 to the IP of the default gateway you want to delete.