Error {"message":"failure to get a peer from the ring-balancer"} using kong ingress

Question

Getting error msg when I trying to access with public IP:

"{"message":"failure to get a peer from the ring-balancer"}"

Looks like Kong is unable to the upstream services.

I am using voting app

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: telehealth-ingress
  namespace: kong
  annotations:
    kubernetes.io/ingress.class: "kong"
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: voting-service
          servicePort: 80

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: voting-service
  labels:
    name: voting-service
    app: voting-app
spec:
  ports:
    - targetPort: 80
      port: 80
  selector:
    name: voting-app-pod
    app: voting-app

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: voting-app-pod
  labels:
    name: voting-app-pod
    app: voting-app
spec:
  template:
    metadata:
      labels:
        name: voting-app-pod
        app: voting-app
    spec:
      containers:
        - name: voting-app
          image: dockersamples/examplevotingapp_vote
          ports:
           - containerPort: 80
  replicas: 2
  selector:
    matchLabels:
      app: voting-app

Solution 1

There could be one of many things wrong here. But essentially your ingress cannot get to your backend.

If your backend up and running?

• Check backend pods are "Running"

  kubectl get pods
  

• Check backend deployment has all replicas up

  kubectl get deploy
  

• Connect to the app pod and run a localhost:80 request

  kubectl exec -it <pod-name> sh
  # curl http://localhost
  

• Connect to the ingress pod and see if you can reach the service from there

  kubectl exec -it <ingress-pod-name> sh
  # dig voting-service  (can you DNS resolve it)
  # telnet voting-sevice 80
  # curl http://voting-service
  

This issue might shed some insights as to why you can't reach the backend service. What http error code are you seeing?

Solution 2

The problem is resolved after deploying services and deployments in kong namespace instead of default namespace. Now I can access the application with Kong ingress public IP.

Looks like kong ingress is not able to resolve DNS with headless DNS. We need mention FQDN in ingress yaml