NIC teaming and EtherChannel

Question

Cisco Catalyst 2960S Switch 
Intel Dual Port NIC (teamed)
BroadComm Quad Port NIC (teamed 2 ports)

We hired a contractor to setup a fairly simple SQL cluster and SAN storage. With the above switch and NICs, they have teamed two cards on each server (3 servers total) for the domain network communication. There is no problem with the NIC teams, they are functioning fine, I have tested failover of each (unplug one of the cords).

The question I have is, would I gain any benefit from also creating an EtherChannel in my switch for each of these NIC teams? The contractor has not done this, and although everything is working fine, I want to get the top performance and reliability of this buildout.

Solution 1

Is the link currently saturated? If not, then configuring Etherchannel is not likely to do anything for you.

Link aggregation != better network performance (throughput) if the current link isn't saturated.

Solution 2

More likely than not the teams are using LACP for increased throughput and failover. If this is the case manually creating an EtherChannel is redundant and will lock the port group into specific manually configured ports (rather than LACP's dynamic assignment of ports).

Basically it sounds like the configuration is 100% correct and you shouldn't mess with it. You should probably verify that the NIC teams are configured for LACP (if they're not it may be for good reason, don't change it if you don't understand the reasons it was setup the way it is).

Side note: As long as you're poking around, the Broadcom NIC is probably a 57xx Series, if it's firmware/driver/management software isn't from 2011, it's got some serious bugs that updates will fix.

Solution 3

Well yes, setting up two NICs in a team only gives you failover redundancy, if it's done using an LACP-capable stack then etherchanneling on the switch allows you to, theoretically at least, double your bandwidth.

Comments

[SpacemanSpiff]

I've actually been meaning to ask this... is LACP better than Broadcom's "smart" load balancing?

[Chopper3]

dunno sorry, haven't used the broadcom-specific stuff (I'm a HP-bitch anyway), though I'll have a look now, presumably they've got a wider range of failover/back options.

[EkoostikMartin]

So if not done on LACP with etherchanneling I am not getting a bandwidth benefit to only doing the NIC teaming?

[EkoostikMartin]

Well, the link isn't saturated because our hardware isn't in produciton yet. But perhaps it will be saturated. Should I configure EtherChannel just in case?

[Chopper3]

No, the port-group needs to be defined on the switch too.

[joeqwerty]

I would say that you shouldn't do anything "just in case". Identify a real need before implementing Etherchannel. Try running some load tests, approximating your expected production load as closely as possible and baseline the network utilization of the servers. If the test justifies the need for link aggregation then go ahead and do it. If not, then don't. Implementing unneccessary components, functions, methods, etc. only serves to unneccessarily complicate the setup, configuration, management, troubleshooting, etc.

[EkoostikMartin]

Very good advice, thank you. Am I correct though in assuming that the NIC teaming is still very useful in giving a bit of failover capability?

[joeqwerty]

In the right scenario, yes. It appears that NIC teaming (in failover mode) is justified in your scenario, now it's just a matter of determining whether link aggregation is justified or not.

[EkoostikMartin]

Chris - I did update the drivers and management software, but thank you for the tip. We had a similar problem with some old Intel NIC drivers which caused us a headache for several days and we even had to open a case with Dell to resolve. Question though, what are they other options than LACP, and why would they be used? Just asking for more of an educational purpose than a desire to change the configuration at this point. Thanks.

[Philip]

See this other answers for a LACP description and alternatives: serverfault.com/q/145426#145458

[EkoostikMartin]

Thanks for the link Chris, very helpful. Another question though - someone is telling me that LACP requires different physical NICs, that it is not configurable using one multiple port NIC. Is this correct?

[Philip]

I know Intel and Broadcom NICs can use LACP even on the dual and quad Port models (those NICs have multiple ports, but only one chip). It should be noted this isn't a Best Practice because if the the chip goes bad, it takes all the ports with it, so you'll no longer be redundant. This wouldn't affect redundancy if other parts of the network fail however. And if you don't care about redundancy you'd still get the increased throughput.