google cloud http(s) load balancer returning 502 despite healthy backend service

Question

I've got a GCE http(s) load balancer configured to direct traffic to 2 backends: - the default backend - a second backend with a separate host/path rule (<host>:/*)

Both backends exist on the same instance group - a container cluster running two kubernetes services that expose different NodePort.

The instance group has two named ports, one for each of the exposed NodePorts.

Despite both backends are reporting healthy, all requests to the second backend return 502.

The services themselves are not the problem - if I switch the service behind the 2nd backend to be default, it works fine.

NOTE: I've updated/simplified this description considerably since it was first posted, as I'm able to reproduce the problem with a much simpler configuration.

Solution

Traffic from the load balancer to your instance is not enabled by default. Unfortunately this is not well documented, and really, when you create a load balancer this should happen automatically.

Try adding this firewall rule the network that your load balancer and VMs are on:

130.211.0.0/22   tcp:1-5000   Apply to all targets

Comments

[hawkett]

I've raised an issue here: code.google.com/p/google-compute-engine/issues/detail?id=254

[hawkett]

Responses in this google issue point to this being a config. problem with the host/path rule.

[hawkett]

The issue is not the firewall rule - the services work individually. From responses in the google issue noted in OP comment, it looks like a configuration issue on the host/path rule.