Can't set NIC speed to 10Gbps using ethtool

Question

We're using IBM Blade Center HS-22 servers with Broadcom NetXtreme II 10Gbps NICs to a Nortel 10Gbps switch. With one of the chassis the NICs can establish a 10G rate with the command "ethtool eth2 speed 10000 duplex full autoneg off", but the same command to a different server on a different chassis gives me the help menu. When I take off the extra zero and set the speed to 1Gbps it seems to work.

The output from "ethtool eth2" shows the "Supported Link Mode" as 1000baseT/Full (1Gbps), but the "Advertised Link Mode" and "Speed" as 10000baseT/Full (10Gbps). Doing the network tests seems to show only 1Gbps throughtput on two servers directly connected to the 10Gbps fabric.

Here are the vitals for the one that does work and the one that doesn't work.

item                 Doesn't Work   Works
Firmware:            5.0.11         4.6.3
OS:                  RHEL 5.4 x64   RHEL 5.6 x64
Kernel:              2.6.18-164-el5 2.6.18-238.el5
Driver (bnx2x)       1.52.12        1.52.53-4

I know I'd like to isolate the variables and get all of these the same, but are there any other items I'm missing that would prevent me from setting the speed and "Supported Link Mode" to be 10Gbps?

Any help is much appreciated.

Solution

What is the device two servers connected to?

If the peer device ( switch, router, ... ) only supports 1Gbps, cause your server only has 1Gbps speed.

Comments

[Marcel]

Supported modes and advertised modes are different concepts, advertised mode is the operation mode that the switch port tell the interface. supported modes are the modes that the NIC support from the device module. Check your loaded modules.

[MartyE]

@Marcel Thanks, what command would you use to list the loaded modules? I tried lsmod and it didn't show anything different.

[MartyE]

It's connected to a Nortel 10Gbps switch in the IBM BladeCenter H. We have verified that in both chassis it gets 10Gbps links to the external connections.