ELB, EC2, Instance has failed at least the UnhealthyThreshold number of health checks consecutively

Question

I am configuring ELB with EC2 in AWS and run into "Instance has failed at least the UnhealthyThreshold number of health checks consecutively" error to cause the instance out of service. I have checked,

1) as I m using 8080 for security check port and my path is "/index.html". I test the link http://ec2DNSname:8080/index.html in the browser, it returns,

Response header:
HTTP/1.1 200 OK
X-Powered-By: Express
Accept-Ranges: bytes
ETag: "150-1420697739000"
Date: Thu, 08 Jan 2015 06:16:21 GMT
Cache-Control: public, max-age=0
Last-Modified: Thu, 08 Jan 2015 06:15:39 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 150
Vary: Accept-Encoding
Connection: keep-alive

2) In my EC2 config, I allow the the inbound TCP traffic on 8080 from any machines

Any other possible cause of failing the health check?

Thanks Hammer

Solution

As long as the page returns a 200 response, it should eventually pass the health check. The Health Check configuration includes the number of times a Health Check needs to return a good result and how often to check it. So, a Healthy count of 5 with a check every 30 seconds would take 2.5 minutes to return as Healthy.

If the HTTP health check is failing, try it with a TCP health check on port 8080.

Also, just in case, check that your VPC configuration allows a connection from the Load Balancer to the EC2 instance. This will be automatic if they are in the same Subnet, but Network ACLs can impact a connection between different Subnets.

Comments

[Hammer]

assuming my port config is ELB 8080 -> EC2 8080 and healthy check port is 8080 http. For the source security, shall I enable, inbound , http:8080 to all, and outbound all traffic to all?

[Hammer]

I tried, 1) change healthy port to tcp 8080, still fails. 2) I change to the security group to allow inbound http 8080, tcp 8080 to all. Both not working :(

[John Rotenstein]

On the Security Group associated with the EC2 instances, try allowing Inbound "All traffic" from "0.0.0.0/0" for testing purposes. This will remove the security group as a possible cause of the problem. If that works, then it's a matter of appropriately configuring the Security Group. If it doesn't work, then something else is the cause.

[Hammer]

Thanks. John, mine is working now. It looks like there some syn issues that what I configure in security group is different from the one under "view rules". I change again and resave.

[NIKHIL CHAURASIA]

I can confirm that changing the "HTTP" health check to "TCP" health check resolved this issue for me. Thanks @John

[Md.]

For me, about 15 minutes later after launching the instance, issue was fixed automatically. I guess, initialization took some time.