How to choose the external IP address of a Kubernetes load balancer in Google Kubernetes Engine

Question

I'm deploying a web app using Google Kubernetes Engine and I want to make it accessible via a load balancer on an existing static IP address that I control as part of the same project in Google Cloud Platform, because the domain name I want to use already points to this IP.

The yaml file I used for the pod is:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  labels:
    app: my-app
spec:
  containers:
  - name: my-container
    image: gcr.io/my-project/my-app:latest

I can set up a load balancer using:

apiVersion: v1
kind: Service
metadata:
  name: my-load-balancer
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-app
  type: LoadBalancer

This provides an external IP on which the app is accessible, but I can't find any way to configure it to use the IP I want. The services documentation mentions a spec.clusterIP setting, but this doesn't seem to relate to the external IP. Likewise, once the load balancer is set up, the service's status.loadBalancer.ingress.ip field is set to its external IP address, but this doesn't seem to be a configurable setting.

As an alternative, I tried manually creating a forwarding rule in the Google Compute Engine console to direct traffic from my static IP address to the target pool set up by Kubernetes, but when I try to connect the connection is refused.

Is there any way to do what I want - to expose a Kubernetes pod (or replication controller) on Google Kubernetes engine on a chosen static IP address?

Solution 1

TL;DR Google Container Engine running Kubernetes v1.1 supports loadBalancerIP just mark the auto-assigned IP as static first.

Kubernetes v1.1 supports externalIPs:

apiVersion: v1
kind: Service
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.10.10
  ...

So far there isn't a really good consistent documentation on how to use it on GCE. What is sure is that this IP must first be one of your pre-allocated static IPs.

The cross-region load balancing documentation is mostly for Compute Engine and not Kubernetes/Container Engine, but it's still useful especially the part "Configure the load balancing service".

If you just create a Kubernetes LoadBalancer on GCE, it will create a network Compute Engine > Network > Network load balancing > Forwarding Rule pointing to a target pool made of your machines on your cluster (normally only those running the Pods matching the service selector). It looks like deleting a namespace doesn't nicely clean-up the those created rules.

---

Update

It is actually now supported (even though under documented):

1. Check that you're running Kubernetes 1.1 or later (under GKE edit your cluster and check "Node version")
2. Under Networking > External IP addresses you should have already some Ephemeral marked as pointing to your cluster's VM instance (if not or unsure, deploy once without loadBalancerIP, wait until you've an external IP allocated when you run kubectl get svc, and look up that IP in the list on that page). Mark one of them as static, let's say it External Address is 10.10.10.10.
3. Edit your LoadBalancer to have loadBalancerIP=10.10.10.10 as above (adapt to the IP that was given to you by Google).

Now if you delete your LoadBalancer or even your namespace, it should preserve that IP address upon re-reploying on that cluster. If you need to change the cluster, some manual fiddling should be possible:

1. Under “Network load balancing” section, “Target pools” tab, click “Create target pool” button:
   • Name: cluster-pool (or any other name)
   • Region: Select the region of one of your clusters
   • Health Check: Optional, if you wish
   • Select existing instance groups: Your Kubernetes cluster
2. Under “Network load balancing” section, “Forwarding rules” tab, click “Create forwarding rule” button:
   • Name: http-cross-region-gfr (or any other name)
   • Region: Select the region of one of your clusters
   • External IP: Select loadbalancer-ip-crossregion you just reserved
   • Target pool: Select cluster-pool you just created

Solution 2

Good news. This will be fixed in Kubernetes v1.1. You can set the service.spec.loadBalancerIP field to an IP you know you own.

---

Previous:

This is a feature that fell through the cracks. It was supposed to work (and may even have worked at some point), but wasn't well tested and got broken along the way and then sort of accidentally designed out (temporarily).

It's on my short-list for things to fix after 1.0.

https://github.com/GoogleCloudPlatform/kubernetes/issues/10323

Comments

[Ward - Reinstate Monica]

This should probably be a comment, not an answer.

[Ben]

Are those still relevant ?