Is it possible to use multiple load balancers to redirect traffic to my application servers?

Question

I'm new to load balancing and I'm wondering if it's possible to use multiple load balancers to redirect traffic to my application servers. I don't really understand how this can be done. Shouldn't a domain name match one to one with a certain server's IP address (in this case the IP of one load balancer)? If each load balancing server has a different IP, how can the request be received by both load balancers (or by 10 load balancers or 50 or 100)?

Solution 1

Using round robin DNS is not that great for high availability - if one server goes offline, clients will still try to connect to it and wait for a timeout.

There are other ways to achieve this.
1) Active/Passive load balancers
Basically one load balancer handles all traffic for one IP address.
If that balancer goes down, the passive node jumps in and takes over the IP.
Keep in mind that load balancers are pretty much only forwarding traffic, so for small to medium sized sites this can work out OK.

2) Active/Active load balancers
The same traffic IP is configured on both (or many more) load balancers.
Incoming traffic gets sent to all load balancers but a algorithm chooses which balancer should respond, all others discard that traffic.
Simple way to think of it, you have two load balancers:
When the requesting IP ends with an even number then load balancer A answers, otherwise load balancer B answers.

Of course your infrastructure must support this and there is overhead due to traffic getting sent but discarded.
More information, e.g. here: http://community.brocade.com/t5/SteelApp-Docs/Feature-Brief-Deep-dive-on-Multi-Hosted-IP-addresses-in-Stingray/ta-p/73867

Solution 2

High Availability with load balancers is commonly implemented using a virtual ip address (VIP) protocol which allows several hosts (i.e. load balancers) to answer to one common ip address in one of several possible ways (variations on active/passive, active/active).

There are a good number of these protocols, the ones I have seen most with regular load balancers are VRRP and NLB (as well as a good many nondescript blackboxed protocols in appliances). Expanding to routers and firewalls one may also encounter CARP, HRSP, GLSP for instance.

This strategy has a number of benefits over DNS load balancing which is a simpler strategy (and which is taken care of in another answer).

DNS load balancing is burdened for instance with:

• the slow turnover of dns caching mechanisms
• limited load balancing algorithms (typically just round-robin)
• the outsourcing of the load balancing decision to the client (through caching of the dns record)
• Slow drain of service queues when a server (i.e. a load balancer) is taken out of rotation (based on dns record TTLs as handled by ISPs and clients)
• Slow failover on load balancer failure

Using a virtual ip protocol for HA one may have a choice to achieve for instance:

• Choice of load balancing algorithm amongst the load balancers
• Server centric load balancing decisions (fascilitating for example service health based measures and routing)
• Quicker drain of service queues when a load balancer is taken out of rotation.
• Instant failover on load balancer failure

Only you know which strategy and protocol fits your scenario best.

Comments

[user3790827]

Thank you for your response. So basically, if I want to use multiple load balancers to handle my traffic I only have to setup a different CNAME for each one of them? Specifically, if I need 10 load balancers to handle the traffic to my site, that's the only way to do it?

[Andrew B]

I recommend leaving questions open for at least a day before closing them. Even that is usually being hasty. Just because you've received an answer doesn't mean it's necessarily the only (or best) one, and marking your Q&A answered usually means it gets less attention.

[user3790827]

@Anatoly I didn't make a decision yet. I've reviewed the solutions presented here and also talked with some of my friends who recommended me other solutions. I think that for my use case the best solution so far would be to use VPS servers from a cheap provider like DO or Vultr that don't offer Virtual IP and use the method used by Algolia with client load balancing. I need HA and scalability for the API only therefore there wouldn't be such a big deal if I create different subdomains for each load balancer. These end user of the widget will never notice them anyway.

[Andrew B]

I'd also add that some load balancers support establishing BGP sessions with nearby routers, which allows you to set up Anycast solutions. If the load balancer goes down or otherwise stops advertising for the VIP (failed health check), the next best routing candidate wins. The last sentence of this answer is imperative though: you really need to talk to your company's network admins.

[user3790827]

When you say "of course your infrastructure must support this" you mean I need an additional machine or VM that will send requests to the load balancers?

[Jenny D]

@user3790827 Infrastructure in this context is the network equipment, not servers.'

[user3790827]

I'm planning use a cloud provider therefore I don't have direct control over the physical infrastructure. What should I ask my vps service provider for?

[faker]

There is only abstract recommendations because it depends on a ton of details. We don't even know if it makes sense to have a multi hosted IP here - maybe his traffic is only a few hundred Mbit/s. If you need this, I would evaluate the proper software, check the requirements and figure out which provider supports it. Would DNS RR work? Sure. Would I use it? Depends on what kind of availability the owner of the business I'm working for is aiming for!

[user3790827]

@faker I'm sorry, I guess it's my fault because I didn't gave enough details. I want to build a javascript script that will be inserted into other people's websites and will gather traffic data( think Google Analytics), also it will access the server to display statistics for each page it's loaded on. Basically there would be a javascript file that will be loaded for each website it's used on.

[user3790827]

Now solely for these js files maybe I can get away by using a amazon S3 and/or a CDN, but I would still have to do at least a GET request and a POST request for each page visited by the users of the sites this script is embedded in.

[user3790827]

Obviously initially I probably wouldn't need more than a server but I would probably need it to have High Availability. But as you might imagine if this script would be embedded even on 100 sites or I would probably need to handle twice the number of page visits of those sites which can hundreds of millions. I would like to be able to handle millions of requests per second.

[user3790827]

Hardware is also another problem. Initially I intended to use ARM servers from www.scaleway.com. But these servers have only 4 ARM cores and 2GB of RAM, you can't go beyond that in terms of hardware. Also the don't seem to offer Virtual IP's yet. The second and third options would have been vultr.com and digitalocean, even though I can get better hardware, the also don't offer Virtual IP's. Then there's AWS and Rackspace, they offer Virtual IP's, the offer various types of VPSs but they are quite expensive.

[Martin Podval]

Here is a nice description of what you describe in the first paragraph cisco.com/c/en/us/support/docs/application-networking-servic‌​es/…