Does a single LACP channel over multiple switches increase redundancy?

Question

I am curious for opinions, findings, or evidence that having multiple interfaces bonded using LACP to ports in multiple switches can increase redundancy. Previously bonded interfaces have always been to a single switch, with a redundant channel to another port.

Without getting into vendor specifics, my thought is that as this is a single LACP, the likelihood that an event or change could lead to a wide service outage. Without having the spare equipment or time to test this single channel over diverse switches, could anyone with a greater networking knowledge than myself, tell me if there a network side event that would bring down the network connectivity to a server that had created a bonded interface to two ports on separate switches?

Does the use of bonded ethernet channels across multiple switches (that we are advised that we can use) from the server, provide both improved throughput (unquestionably), and improved redundancy (uncertain). Could/would network events such as switch failure, port migration, patching, recovery, etc, cause the channel for both server network interfaces to be unavailable?

Thanks in advance.

Solution 1

LACP itself doesn't provide the ability to bond across multiple switches; it bonds across multiple ports on a single ethernet switch, and depending on the vendor there might even be restrictions on which ports on a switch can be bonded together.

Some vendors have proprietary protocols (typically called MLAG) that allow for bonded ethernet channels across different ethernet switches. As an example Cisco Nexus vPC (or generically MLAG) works with switches, or bonding a single LACP port channel on a server across two connected switches.

Does the use of bonded ethernet channels across multiple switches (that we are advised that we can use) from the server, provide both improved throughput (unquestionably), and improved redundancy (uncertain). Could/would network events such as switch failure, port migration, patching, recovery, etc, cause the channel for both server network interfaces to be unavailable?

LACP should provide protection against a single physical port or cable failure within the LACP channel.

LACP cannot protect against human factors, such as accidentally shutting down the LACP-interface, removing the vlan, or running a TDR on a port-channel member link. LACP also cannot protect against over-provisioning bandwidth through a single member link on that LACP channel, spanning-tree events, broadcast storms, excessive unknown unicast flooding, etc...

If you are concerned about recovery time, be sure to use short LACP protocol timeouts on your interfaces.

Solution 2

LACP aggregates multiple physical links between the same two devices into one logical link that has higher throughput. If you are looking for redundancy in case of a switch failure, then you need to set up links to two switches, and configure the server to bridge the two links with the spanning tree protocol enabled. STP will automatically choose one link to be active, and the other backup. If the primary link goes down ( switch died ), it will switch to the backup. This doesn't really have anything to do with LACP.

Solution 3

LACP can usually only be accomplished to a single device or group of device that act as one. So... you could do it between a switch and a switch stack, but not spread out across multiple unique devices.

Solution 4

LACP is a protocol for the host to communicate to the switch(es) that it wants to aggregate multiple physical connections. This aggregation might not always be possible however. If the connection is made to multiple switches those switches must support some method of cooperating to make the bonded link or they will be unable to.

Most switches do not support this cooperation. Some however do. The most common method is the SMLT protocol, which allows switches to cooperate in creating bonded links. Some switches also support stacking protocols that enable multiple switches to act as a single logical switch. In either case you'll want to check the documentation to be sure of support and configuration.

Solution 5

4nics aggregation (2of4 active + auto failover):

Linux: eth0-3 -> bond0, bonding mode=4

Switches (Juniper for example): port 1 + port 2 -> ae1 set interfaces ae1 aggregated-ether-options lacp passive set interfaces ae1 (...vlan assignments here...)

passive on switch side ensures that it's Linux who'll device the partner to talk to. If you need PXE boot on same nics - looks for force-up (juniper) option on A-switch,ge-0/0/1

View more solutions

Comments

[Sirch]

And its the use of bonded ethernet channels across multiple switches that we are advised that we can use from the server. Providing both improved throughput (unquestionably), and improved redundancy (uncertain). Could/would network events such as switch failure, port migration, patching, recovery, cause the channel to be unavailable?

[J. Singh]

Note that some of the Multi-Chassis Link Aggregation (MLAG) technologies in newer switches work without "stacking." That is, each switch maintains its own active control plane, and can function independently. Arista MLAG, Brocade's VCS, Cisco vPC, HP's IRF are all examples of this approach.

[SpacemanSpiff]

hence the "usually" part of things :)

[dognose]

This is an imprecise Statement. If you use LACP between two devices, where both are not Switches it will not increase throughput, unless you use round-robin for load balancing which is not recommended due to out-of-order packets. LACP balances the load depending on the conversation, which might be long running and always using the same physical link to communicate... If one device is a switch with multiple clients you'll get multiple conversations leading to increased throughput on the attached LACP'd device for sure.

[psusi]

@dognose, LACP increases throughput by definition. That is to say: the entire reason for LACP to exist is to increase throughput. You are confusing LACP with the linux bonding driver, which routes packets over multiple interfaces via software and where maintaining the ordering guarantees is difficult if not impossible ( though honestly, who cares about ordering? ). A single interface that has multiple physical ports and hardware support for LACP ( and connected to a switch with LACP support ) does not have such problems.