How to configure Broadcom 5709 NIC as iSCSI Hardware Initiator

Question

I'm having some trouble to setup an iSCSI Hardware Initiator within ESX4. The host-machine has a "Broadcom NetXtreme II BCM5709 1000Base-SX" NIC which supports TOE and can be used as an iSCSI HBA. The question is... what steps are necessary so that the NIC shows up in the list of storage adapters?

The SAN-iSCSI Guide for ESX4 only has a few lines about this and is not a big help. The NIC itself has a boot-menu which includes some iSCSI configuration. At that place I configured the ip-address for our IP-SAN but no change within ESX.

Does anybody have a clue about this? It is really starting to get frustrating for me.. and I don't want to use Vmware's software iSCSI initiator.

Thanks in advance,

Lemmy

Solution 1

I can't find any documentation that indicates that iSCSI Offload functionality is supported by ESX for hybrid devices like the Broadcom 5709 which I've always understood is not a HBA (at least according to Dell's tech folks).

ESX 4 does support IPv4 TCP Offload, specifically TSO (TCP Segmentation Offload) and LRO (Large Receive Offload), on all supported NIC's that support it AFAIK and it is enabled by default. You can check that it is enabled by looking at the MSS result from esxcfg-vmknic -l to see if the NICs used by the iSCSI VMKernel ports have TSO enabled. IPv6 offload is currently only suported for (some) Qlogic iSCSI HBA's but since it is supported within Guests using the VMXNET 3 Network adapter I'd expect to see support at the host level NICs soon.

The ESX 4 Software initiator is substantially better than previous versions, it has substantially lower CPU utilization, better multi-threading, multi-pathing and Jumbo frame support so the advantages of using HBA's for iSCSI are much less significant than they were with earlier ESX versions.

Solution 2

According to VMware (as of ESXi 5.1), a NIC using the Broadcom 5709 chipset is a "dependant hardware iSCSI initiator". It is not a full hardware iSCSI HBA initiator like a QLogic 4062C, but does support and contain much useful hardware that makes it better than just using a regular NIC card. It has an iSCSI initiator and an TCP/IP offload engine, but still requires configuration as a NIC and as an iSCSI initiator.

According to VMware "A dependant hardware iSCSI initiator, or adapter, (i.e., the 5709) depends on VMware networking and on iSCSI configuration and management interfaces that are provided by VMware. This type of adapter presents a standard network management adapter and iSCSI offload function on the same port. To make this adapter functional, you must set up networking for the iSCSI traffic and bind the adapter and an appropriate VMKernel iSCSI port."

I have several of them, and they are both cheap and efficient. They make a good choice, as they have good performance, and are significantly cheaper than a full hardware iSCSI initiator like a QLogic 4062C. Don't get me wrong, the QLogic is a great HBA, but is four or five times more expensive.

Solution 3

Historically, ESX has very limited support for hardware-based iSCSI HBA's. Their own iSCSI software is however 100% supported.

The only hardware iSCSI HBA's that ESX 4.0 U1 supports are listed here

Solution 4

Well, Helvick's answer is right. Until today, Esx(i) does not support TOE but it does support TSO. Also. Broadcom 5709 NIC has compatibility issue with Esxi 4.1 and 5.0 (NOT Sure any other version). The iSCSI connection has problem when using it as HW iSCSI intitiator on our Dell R710 even updated to lastest BIOS, firmware, and Driver. It can be only used as software iSCSI initiator. So anyone trying to get Hosts machine for vitualiztion, BE AWARE of the Broadcom NIC. As my friend told "Intel NIC is usually the preference".

BTW, VMWare states that "You cannot use Jumbo Frames on a Broadcom card that is configured as a hardware initiator performing iSCSI offload functions. You can either use Jumbo Frames or iSCSI Offload and you cannot use both together with the Broadcom adapters"

so, JF or HBA, either one.

See: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007654

Solution 5

You have to "bind" the vmhba and the vmnic ports. Below is from the iSCSI config guide:

1. Use the vSphere CLI command to determine the name of the physical NIC, with which the iSCSI adapter is associated. esxcli swiscsi vmnic list -d vmhba# (vmhba# is the name of the iSCSI adapter)
2. In the output, find the vmnic name: vmnic# line. ---You can also see the vmhba#'s if you view the storage adapter section in vSphere--- vmnic# is the name of the network adapter that corresponds to the iSCSI adapter.

What to do next After you determined the name of the NIC, create an iSCSI port on a vSwitch connected to the NIC. You then bind this port to the dependent hardware iSCSI adapter, so that your host can direct the iSCSI traffic through the NIC.

I hope this helps.

View more solutions

Comments

[pauska]

Just a note: Every modern network chip supports ToE, and can be used fine for iSCSI in ESX.

[Admin]

Did you get the Broadcom NetXtreme II 5709 to works? Did you used the iSCSI Offload Key for LOM NICs? Phil