Redundant Load Balancers for Windows-based Web Servers?

Question

I'm having some trouble trying to figure out the technical details of the below design; specifically in the area of the redundant load-balancers.

The web servers are running Windows Server 2003 R2 64 and serve .NET applications via IIS. The database servers are running Windows Server 2008 Enterprise 64 in a SQL Server Failover Cluster with SQL Server 2008. There are currently two web servers and two database servers.

What I am looking to accomplish:

• Automatic redundant fail-over if a load-balancer goes down.
• The ability to seamlessly take a web server out of the load-balanced mix for maintenance without interrupting users. I'm not sure how possible this is because of the way our applications work - users tend to say logged into the application for their entire shift.
• The ability to scale up web servers, as needed (does not need to be done live).

This is a pure-Microsoft shop; so unfortunately the standard Linux tools are not available to us.

What I have tried:

• Microsoft NLB (Network Load Balancing Service): this works relatively well for a simple solution and is quite cost-effective since it just runs on the web servers, but I have yet to find a way to make this service meet the above requirements. Every time we have tried to take a system out of the load-balanced mix, clients making requests to the load-balanced url/ip are still trying to get directed to the offline machine. This can create huge problems, especially considering that our users will be submitting customer payments through these systems. Maybe we're doing something wrong here...

The design:

So, given all of the above, is Microsoft NLB the only answer? Or are there better tools available for our situation?

Edit 4/21/11

Thanks for the quick feedback. Just to clarify a few points:

• These are intranet web servers. They don't touch the Internet. Ever.
• Convincing my boss to let me deploy a pair of Linux servers wouldn't be too difficult. She isn't the roadblock to a Linux environment - it's our staff. Their only skill-set is Windows. It would kill my social life to start deploying Linux servers in our data center. ;-)
• I'm ultimately searching for the "Microsoft way" of load-balancing web servers, while at the same time providing redundancy in the load-balancing subsystem. If that really is Microsoft's NLB service, well... maybe I should start a new question about that. :)
• I'm open to hardware load-balancers if that is a better (or only) solution.

Solution 1

What you want is called Microsoft Application Request Routing 2 (ARR). (Maybe the clumsy name is part of why so few people know of its existence?)

Microsoft ARR is a free-of-charge HTTP layer load balancer, implemented as a module for IIS 7+. (ARR itself is gratis, but the Windows Server license is of course required for the underlying OS.)

Since ARR is just a thin shim on top of IIS, it is quite fast and absolutely robust. And administrating ARR will be familiar for you guys, since you're already an IIS shop. ARR just installs itself in the IIS Manager GUI.

For a true high-availability setup, you should combine NLB and ARR, so that NLB keeps the ARR server tier highly available, and ARR keeps the backend web server tier highly available. See Microsoft's docs, and see the long list of documentation at the end of the ARR overview page linked at the top.

The only real downside to ARR is that if you do true high-availability, then you will require at least 2 Windows Server licenses & physical servers. Given that, and given the time it takes to set up, then low-end load balancer appliances like Coyote Point or loadbalancer.org can sometimes be a cost-effective alternative (Or Kemp, Barracuda Networks, or any of the other low-end vendors).

ability to seamlessly take a web server out of the load-balanced mix for maintenance without interrupting users.

That will depend on how session state is handled, i.e. how your backend servers share or not share the "this user is logged in" information.

If the webapp tier is stateless (i.e. placing session state in a shared datastore, fx a shared RAM cache or MSSQL), then you can just pull a server out of the pool. If not, then you can use "sticky sessions" on the load balancer, and remove a backend server from the load balancer pool, and then wait until all users have 'drained off' the server in question.

Willy Tarreau, the author of HAProxy, has a nice overview of load balancing techniques and issues here.

Solution 2

If your shop is dead set on a Microsoft-only solution, their ForeFront Threat Management Gateway has a "Server Farm" feature that does some load-balancing (link). It isn't nearly as featured as the dedicated hardware loadbalancers out there, or as configurable as the Linux-stack software available, but it'll get you there for some use-cases. And importantly, it'll probably be on your Microsoft contract price-list somewhere.

The TMG can be loadbalanced itself through NLB.

Solution 3

Even where a particular answer is a "good" answer, if you're the only person in the organization who isn't scared to log into it, you'll own it forever. And everybody else will blame the scary foreign device for everything, including the brown plants on their desk that died from lack of water.

I've found that monoglot sysadmins aren't as intimidated by something with a web gui. (No need to tell them about the Linux kernel it runs on). What about a commercial appliance, like an an F5, Brocade/Foundry ServerIron, Cisco CSS, CoyotePoint, etc? We use an HA pair of ServerIrons, and I've used CSS/Arrowpoints in the past.

Comments

[gravyface]

It's too bad you can't use Linux: HAProxy, nginx, apache, etc. could all be capable load balancing solutions setup in a HA cluster (with heartbeat).

[Tom O'Connor]

Just run Linux inside VMware on a microsoft platform. It's free (ESXi).

[gravyface]

@Tom: like seriously. Tell your boss you can put up high-quality "cloud ready" load balancers (if he's of the pointy hair variety) if only you could use Linux...

[gravyface]

@Tom: I was agreeing with you.

[Keith Stokes]

It's been awhile since I've built a NLB cluster, but I don't remember having the trouble you described. In fact, it worked quite well for Terminal Services.

[Tom O'Connor]

@gravyface. Oh, sorry. "like seriously" isn't a grammatical construct i'm used to.

[Cypher]

+1 - This addresses my concerns exactly, from the comments about owning a system forever, to the scary black box to blame for everything, through the gui-based commercial appliances. Thank you for the pointers on the appliances. It's starting to seem like this may be the best way to go for our staff and environment.

[Tom O'Connor]

I bet it's actually a commodity server running Nginx or something ;)

[AndyN]

Cheers, glad it helped. I should add that we also do the "two paths" topology you show in your diagram (the application and reporting paths) by putting the two stacks in different VLANs, and using a Layer 3 device (a firewall, but could be a smart switch w/ static routes.) Because the LB is going to fiddle Layer 2 headers to do it's job, I suppose you could do it all in L2, and tell the LB to bridge the Reporting traffic without balancing it. Depends how much path or vlan independence you need.

[Cypher]

+1 - This is the gem I was hoping to find with this question. Thank you, this looks promising.