Device eth0 has different MAC address than expected

Question

I've got a HP Proliant 460c blade server running CentOS 5.x. For some reason CentOS thinks the mac address for ETH0 is something (18:a9:05:71:xx:xc) but I'm not able to find that MAC at all on the on-board administrator. In fact, if I use the MAC address that shows up in the onboard administrator and try to hardcode it to the nic (under /etc/sysconfig/network-script/ifcfg-eth0), I get an error:

Device eth0 has different MAC address than expected.

Where can I definitively confirm what the system sees at the MAC address? ifconfig -a shows the "incorrect" one.

---

On the CenTOS host, I'm seeing this in ifconfig results (slightly modified to protect the innocent):

eth0 HW 18:a9:05:71:xx:xc
eth0:0 HW 18:a9:05:71:xx:xc
eth1 HW 18:A9:05:71:xx:xd  
eth1:0 HW 18:A9:05:71:xx:xd
eth2 HW F4:CE:46:8X:XX:40
eth3 HW F4:CE:46:8X:XX:44

I'm not sure where the 18:a9 MAC addresses are coming from.

When I check the port mapping for the device bay that belongs to the server, I see:

Mezzanine Device Port: ENET1  
Device ID: d4:85:64:49:xx:x8

Mezzanine Device Port: iSCSI Port 1  
Device ID: F5:CE:46:83:xx:x1

Mezzanine Device Port: ENET2  
Device ID: d4:85:64:49:xx:xc

Mezzanine Device Port: ISCSI Port 2   
Device ID: F5:CE:46:xx:xx:x1

I also see

Mezzanine Device Port: Port 1  
Device ID: d4:85:64:xx:xx:be

Mezzanine Device Port: Port 2   
Device ID: d4:85:64:xx:xx:bf

Note that none of them pertain to the MAC that CentOS thinks should be there. I've also completely reinstalled CentOS and the problem persists. So it seems likely that something on the system is telling CentOS that the MAC should be used but I'm not sure where.

Solution 1

This is something that may be unique to the HP blade chassis setup. Since you mention that this is a blade server, do you know if the blade chassis has HP Virtual Connect modules installed? These are often used instead of passthrough ethernet uplinks to help consolidate and virtualize the blade networking.

If you do have Virtual Connect modules, check the server profile in the Virtual Connect Manager. There are options to manage the server MAC addresses using Virtual Connect. See page 64 of the Virtual Connect Installation Guide for an explanation of the feature:

Each server blade Ethernet NIC ships with a factory default MAC address. The MAC address is a 48-bit number that uniquely identifies the Ethernet interface to other devices on the network. While the hardware ships with default MAC addresses, Virtual Connect can assign MAC addresses that override the factory default MAC addresses while the server remains in that Virtual Connect enclosure.

The other answers posted certainly apply to multiple NICS in physical rack/tower servers, but the HP blade solution is a very special case.

Solution 2

Normally, this is because of udev.

Remove /etc/udev/rules.d/70-persistent-net.rules

Then unload and load the Ethernet driver (module), or restart. You can find the driver name by trying these commands

dmesg | grep 'Ethernet driver'
grep 'Ethernet driver' /var/log/dmesg
grep eth0 /etc/modules.conf

Then when you have the module name (eg. e1000e) just run,

modprobe -r e1000e
modprobe e1000e
ifdown eth0
ifdown eth0 (again!)
ifup eth0

That should prompt udev to re-detect your NIC and mac address. You might need to update the contents of /etc/sysconfig/network-scripts/ifcfg-eth0 to suit the mac address.

Solution 3

Copied from my documentation:

• The part of the OS that stores this is in the udev config. Edit this config: vi /etc/udev/rules.d/70-persistent-net.rules

• Find the lines that pertain to the old MAC address (aka not the one you just noted down) and delete them

• The lines that correspond to the new device probably mention eth1 instead of eth0. Replace it with eth0.

• Reboot (easiest way to get the new setting in effect)

• If you try to bring up eth0 by using ifup, it will complain that the MAC address is not what it expected. Surprise! The MAC address is also stored in the device's network script. Edit the network script: vi /etc/sysconfig/network-scripts/ifcfg-eth0

• Change the HWADDR line to reflect the correct MAC address

• While you're in here, set a valid IP address. Use nmap if you're not sure what's available

• Now you can bring up the interface: ifup eth0

• Don't forget to update: yum update

Comments

[Mike B]

I updated the post to include the Mac examples. I'll check virtual connect manager. Thanks.

[ewwhite]

Looks like virtual connect is managing your MAC addresses. Check first.

[Mike B]

In the interests of time, we ended up switching to another blade which didn't have the issue. So while I don't have definitive confirmation, this seems to be the most likely explanation. Especially considering that the issue persisted after a full free OS install. Thanks for your time and help.