Kubernetes whitelist-source-range blocks instead of whitelist IP

Question

Running Kubernetes on GKE

Installed Nginx controller with latest stable release by using helm.

Everythings works well, except adding the whitelist-source-range annotation results in that I'm completely locked out from my service.

Ingress config

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: staging-ingress
  namespace: staging
  annotations:
    kubernetes.io/ingress.class: nginx
    ingress.kubernetes.io/whitelist-source-range: "x.x.x.x, y.y.y.y"
spec:
  rules:
    - host: staging.com
      http:
        paths:
        - path: /
          backend:
            serviceName:staging-service
            servicePort: 80

I connected to the controller pod and checked the nginx config and found this:

# Deny for staging.com/
geo $the_real_ip $deny_5b3266e9d666401cb7ac676a73d8d5ae {
    default 1;

    x.x.x.x 0;
    y.y.y.y 0;
}

It looks like he is locking me out instead of whitelist this IP's. But it also locking out all other addresses... I get 403 by going from staging.com host.

Solution 1

Yes. However, I figured out by myself. Your service has to be enabled externalTrafficPolicy: Local. That means that the actual client IP should be used instead of the internal cluster IP.

To accomplish this run kubectl patch svc nginx-ingress-controller -p '{"spec":{"externalTrafficPolicy":"Local"}}'

Solution 2

Your nginx controller service has to be set as externalTrafficPolicy: Local. That means that the actual client IP will be used instead of cluster's internal IP.

You need to get the real service name from kubectl get svc command. The service is something like:

NAME                                          TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                      AGE
nobby-leopard-nginx-ingress-controller        LoadBalancer   10.0.139.37    40.83.166.29   80:31223/TCP,443:30766/TCP   2d

nobby-leopard-nginx-ingress-controller is the service name you want to use.

To finish this, run kubectl patch svc nobby-leopardnginx-ingress-controller -p '{"spec":{"externalTrafficPolicy":"Local"}}'

When you setting up a new nginx controller, you can use the command below:

helm install stable/nginx-ingress \
  --namespace kube-system \
  --set controller.service.externalTrafficPolicy=Local`

to have a nginx ingress controller accept whitelist after installing.

Comments

[Frank Levering]

If this doesn't solve the entire problem, also restart your nginx controller pods. After that it should work.

[Pramod Setlur]

Which header would nginx whitelist against?