Kubernetes Ingress non-root path 404 Not Found

Question

I have the following that config that works when I try <NodeIP>:30080

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: app-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        name: app-node
    spec:
      containers:
        - name: app
          image: myregistry.net/repo/app:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
          env:
            - name: NODE_ENV
              value: production
---
apiVersion: v1
kind: Service
metadata:
  name: app-service
spec:
  selector:
    name: app-node
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
      nodePort: 30080
  type: NodePort

I am trying to use an Ingress:

 apiVersion: extensions/v1beta1
 kind: Ingress
 metadata:
   name: nginx-ingress
 spec:
   rules:
   - host: myhost.com
     http:
       paths:
       - path: /app
         backend:
           serviceName: app-service
           servicePort: 80

myhost.com works with the nginx intro screen, but myhost.com/app gives 404 Not Found. Where is the issue in my setup?

---

UPDATE:

   - path: /
     backend:
       serviceName: app-service
       servicePort: 80

If I do root path it works, but how come /app doesn't?

Solution 1

Your ingress definition creates rules that proxy traffic from the {path} to the {backend.serviceName}{path}. In your case, I believe the reason it's not working is that /app is proxied to app-service:80/app but you're intending on serving traffic at the / root. Try adding this annotation to your ingress resource: nginx.ingress.kubernetes.io/rewrite-target: /

Source: https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/rewrite

Solution 2

As specified by brandon-barnett the issue is coming from the path but after reading the link he shared, big ups to you Brandon, I realised that a more specific rewrite rule would have to specified for it to work. In my case

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  name: rewrite
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: rewrite.bar.com
    http:
      paths:
      - path: /something(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: http-svc
            port: 
              number: 80

The trick is with /$2 at annotation and (/|$)(.*) at the path. So what this rewrite does it picks everything that comes after something/ and replaces $2 with it, so:

• rewrite.bar.com/something rewrites to http-svc:80/
• rewrite.bar.com/something/ rewrites to http-svc:80/
• rewrite.bar.com/something/new rewrites to http-svc:80/new
• rewrite.bar.com/something/new/old rewrites to http-svc:80/new/old

Comments

[captainchhala]

root(/) might be pointing to default backend which would be nginx welcome page, but /app might not be configured properly that,s why when routing request to that gives 404. try other url e.g myhost.com/app1 and see if nginx page shows up.

[atkayla]

If I understand correctly, yes I would like myhost.com/app/foo => app-service:80/foo, not myhost.com/app/foo => app-service:80/app/foo. That explains why my config is wrong, and why only the root config (path: /) works, however nginx.ingress.kubernetes.io/rewrite-target: / does not seem to point it correctly. I have also tried ingress.kubernetes.io/rewrite-target: / with no luck.

[brandon-barnett]

How did you install nginx-ingress? Do you have manifests or instructions?

[atkayla]

console.bluemix.net/docs/containers/cs_ingress.html#ingress (Step 3: Create the Ingress resource)

[brandon-barnett]

Ah indeed. That annotation is specific to nginx, but your ingress controller is managed by IBM (according to the docs). If you were installing a custom nginx-ingress controller that annotation would probably work. In your case, check out the annotation options. Seems like rewrite-path should do it instead of nginx.ingress.kubernetes.io/rewrite-target

[atkayla]

I tried ingress.bluemix.net/rewrite-path: / as well, but it didn't work, and doesn't seem to function the same as rewrite-target. Confusing. Perhaps I should just consult their support, as this issue seems specific to them as a provider. I appreciate your help in pointing me towards the right direction though, thanks!

[brandon-barnett]

One last shot -- did you try this? console.bluemix.net/docs/containers/…

[atkayla]

It finally works! Thank you! I had to do two things: 1. ingress.bluemix.net/rewrite-path: "serviceName=app-service rewrite=/" AND 2. path: /app/ (missing trailing slash!!!).

[Amit Yadav]

@brandon-barnett you're a life savior. Works like charm.