How to setup HAProxy with failover?

Question

I understand that, to get failover on an HAProxy load balancing setup, you need two machines running HAproxy (and route it to several webserver instances). But in this case, say abcd.com, how do we split/route this traffic to 2 IP addresses instead of one? DNS usually resolves domain names to a single IP. How do we do this in using free/cheap tools/services?

Solution 1

If you have so much load that you need to load balance across two haproxy instances then DNS round robin isn't a bad idea (I would be surprised if you have this load though). DNS round robin won't provide good failover though.

At Stack Overflow we use heartbeat to provide a single virtual IP, this IP is active on only one haproxy host at a time (if it goes down, the other takes over this IP). You could use heartbeat to have an IP on each machine and then DNS round robin between the two. If one were to fail, then the other would have both of those IPs.

HAProxy is using about 1-5% CPU on our physical server to balance our traffic which has a single Intel(R) Xeon(R) CPU E5504 @ 2.00GHz. So HAProxy can generally handle a lot of traffic easily.

Solution 2

As Kyle says, heartbeat can be used to make two haproxy servers act as a fail-over pair. However whilst many people do use heartbeat for the job, keepalived is suggest by haproxy's author.

He outlines the details on the haproxy mailing list: http://www.formilux.org/archives/haproxy/1003/3259.html

In brief it goes like this:

• heartbeat is cluster oriented. It ensure only one server has access to a resource (i.e. SAN storage)

• keepalived is network oriented. It ensures that at least one server has the IP in question.

Comments

[mixdev]

Check this too serverfault.com/questions/238605/…

[mixdev]

No. I definitely don't have SO kind of traffic. It is just an attempt to avoid SPOF.

[Kyle Brandt]

@mixdev: In that case just have DNS point to the single Virtual IP provided by heatbeat and you should be good to go. You just leave haproxy running on both machines. Be sure to test the heartbeat failover before going live of course.

[CarpeNoctem]

@KyleBrandt: What if the haproxy process were to die? Do you have anything configured that would allow the virtual IP to be moved based on that?

[Kyle Brandt]

@CarpeNoctem: We don't have that covered at the moment. I think both heartbeat and keepalived can be setup to watch processes as well though. If you do go this route, I would recommend setting it to require that the process be down for a good amount of time (i.e. 1 minute). You probably don't want the heartbeat to flapover on a config error. HAProxy checks most of these with -c, but it might not catch things like a binding config mistake.

[orokusaki]

@KyleBrandt - How does the inbound traffic get to the correct server, if HAProxy is running on the servers themselves (as opposed to something where the requests come in through some physical device which determines which of the 2 servers to route the request to)?

[Kyle Brandt]

For anyone who cares, at Stack we moved to keepalived well over a year ago

[Deer Hunter]

Please include the full content of the recipe in your post. We aren't exactly a blog-linking service.

[Dennis Nolte]

additionally blogs often go down or change URLs and then the info is lost.

[danieljimenez]

Down vote because it's totally unrelated to the question.

[Igor Tverdovskiy]

Kyle, why have you moved? Any issues with heartbeat?

[Kyle Brandt]

@IgorTverdovskiy For load balancers, better to have two conflicting IPs then know IP at all, see serverfault.com/a/582107/2561

[Igor Tverdovskiy]

Read the reference, am I right, understanding keepalived solution as a manipulation with ARP records to maintain the same IP on several hosts simultaneously? I mean we are trying to get rid of this approach as it is difficult to mantain cloned VIP addresses within different colocations due to different networks.. and having issues with ARP cache on routers. Instead we are going to use haproxy to load balance requests among all servers and in it's turn haproxy will be redundant by means of corosync+pacemaker via floating VIP.

[duct_tape_coder]

@danieljimenez, Not totally unrelated I think, just a misunderstanding. OP asked for Active/Passive for the HAProxy itself and user3595100 interpreted the question as Active/Passive for an HAProxy backend. Nonetheless, posting a link is not an acceptable answer by itself.