Ingress configuration for k8s in different namespaces
Solution 1
Instead of creating the ingress app-ingress
in ingress-nginx
namespace you should create it in the namespace where you have the service api-sand
and the pod.
Alternatively there is way to achieve ingress in one namespace and service in another namespace via externalName
.Checkout Kubernetes Cross Namespace Ingress Network
Here is an example referred from here.
kind: Service
apiVersion: v1
metadata:
name: my-service
spec:
type: ExternalName
externalName: test-service.namespacename.svc.cluster.local
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: example.com
http:
paths:
- path: /
backend:
serviceName: my-service
servicePort: 80
Solution 2
I would like to simplify the answer a bit further for those who are reletively new to Kubernetes and its ingress options in particular. There are 2 separate things that need to be present for ingress to work:
- Ingress Controller(essentially a separate Pod/Deployment along with a Service that can be used to utilize routing and proxying. Based on nginx container for example);
-
Ingress rules(a separate Kubernetes
resourse with
kind: Ingress
. Will only take effect if Ingress Controller is already deployed)
Now, Ingress Controller can be deployed in any namespace and is, in fact, usually deployed in a namespace separate from your app services. It can out-of-the-box see Ingress rules in all namespaces in the cluster and will pick them up.
The Ingress rules, however, must reside in the namespace where the app that they configure reside.
There are some workarounds for that, but this is the most common approach.
Solution 3
It's possible actually, you can define ingress and a service with ExternalName type in namespace A, while the ExternalName points to DNS of the service in namespace B. For further details, please refer to this answer: https://stackoverflow.com/a/51899301/2995449
Solution 4
Outside traffic comes through ingress controller service
that is responsible for routing the traffic based on the defined routing rules or what we call ingress rules
in k8s world.
In other words, ingress
resources are just routing rules (think of it in away that's similar to DNS records) so when you define an ingress
resource you just defined a rule for ingress controller
to work on and route traffic based on such defined rules.
Solution:
Since
Ingress
are nothing but routing rules, you could define such rules anywhere in the cluster (in anynamespace
) andcontroller
should pick them up as it monitors creation of such resources and react accordingly.Here's how to create ingress easily using
kubectl
kubectl create ingress <name> -n namespaceName --rule="host/prefix=serviceName:portNumber"
Note: Add
--dry-run=client -oyaml
to generateyaml
manifest fileOr you may create a service of type
ExternalName
in the samenamespace
where you have defined youringress
. such external service can point to any URL (a service that lives outsidenamespace
or even k8s cluster)Here's an example that shows how to create an
ExternalName
service using kubectl:kubectl create service externalname ingress-ns -n namespaceName --external-name=serviceName.namespace.svc.cluster.local --tcp=80:80 --dry-run=client -oyaml
this should generate something similar to the following:
kind: Service
apiVersion: v1
metadata:
name: nginx
namespace: ingress-ns
spec:
type: ExternalName
externalName: serviceName.namespace.svc.cluster.local #or any external svc
ports:
- port: 80 #specify the port of service you want to expose
targetPort: 80 #port of external service
As described above, create an ingress as below:
kubectl create ingress <name> -n namespaceName --rule="host/prefix=serviceName:portNumber"
Note: Add --dry-run=client -oyaml
to generate yaml
manifest file
Related videos on Youtube
camel
Updated on July 08, 2022Comments
-
camel almost 2 years
I need to configure Ingress Nginx on azure k8s, and my question is if is possible to have ingress configured in one namespace et. ingress-nginx and some serivces in other namespace eg. resources? My files looks like so:
# ingress-nginx.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx spec: replicas: 3 selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.12.0 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --annotations-prefix=nginx.ingress.kubernetes.io - --publish-service=$(POD_NAMESPACE)/ingress-nginx env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 - name: https containerPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1
# configmap.yaml kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx --- # default-backend.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: default-http-backend labels: app: default-http-backend namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app: default-http-backend template: metadata: labels: app: default-http-backend spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissible as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: gcr.io/google_containers/defaultbackend:1.4 livenessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi --- apiVersion: v1 kind: Service metadata: name: default-http-backend namespace: ingress-nginx labels: app: default-http-backend spec: ports: - port: 80 targetPort: 8080 selector: app: default-http-backend
kind: Service apiVersion: v1 metadata: name: ingress-nginx namespace: ingress-nginx labels: app: ingress-nginx spec: externalTrafficPolicy: Local type: LoadBalancer selector: app: ingress-nginx ports: - name: http port: 80 targetPort: http - name: https port: 443 targetPort: https
# app-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: app-ingress namespace: ingress-nginx annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / spec: tls: - hosts: - api-sand.fake.com rules: - host: api-sand.fake.com http: paths: - backend: serviceName: api-sand servicePort: 80 path: /
And then I have some app running in the resources namespace, and problem is that I am getting the following error
error obtaining service endpoints: error getting service resources/api-sand from the cache: service resources/api-sand was not found
If I deploy api-sand in the same namespace where ingress is then this service works fine.
-
Choco over 3 yearsnice answer: Can I ask if when creating the Ingress rules, do they all need to be defined in a single Ingress rule, or can there be multiple Ingress rules within each namespace, each mapping to a service within that namespace? tx
-
yuranos over 3 yearsWell, it's not as simple as with Network Policies. Not that NPs are simple, but there's a relatively straightforward way to merge them. With Ingress it can't be so simple because it's harder to standardize underlying Ingress Controller implementation, especially since those are built on top of completely independent components that existed long before Kubernetes inception, like Nginx. Having said that you are not the first one, @Choco, who entertain that thought: github.com/kubernetes/ingress-nginx/issues/1539
-
Lukas over 3 yearsHow did you get it working? I keep getting
service "default/my-service" is type "ExternalName", expected "NodePort" or "LoadBalancer";
-
metasim over 3 yearsI couldn't get it to work either. I'm wondering if it's a provider and/or version issue? I'm on EKS 1.14...
-
Damith Udayanga about 3 years(<error: endpoints "scc-worker-service" not found>) when describing the ingress, show this error. any clue to resolve?
-
Marc over 2 yearsSo if I have 100 Microservices which should be accessible via 1 hostname (and separate paths) I have to put all those services in the same namespace so that they can use the same ingress resource??
-
yuranos over 2 yearsIt's such scenarios, you'd need to come up with a more advances networking architecture. Maybe it will be several levels of ingresses/proxies/gateway. Maybe ExternalName services. Is it a real or a hypothetical example?
-
YoShade over 2 yearsif you read in the example link they speak only on ngnix plus
-
Andy Lorenz about 2 yearshey @yuranos your answer looks great, and I do appreciate how you've addressed it to "those who are reletively new to Kubernetes" - but could you please expand on what you mean by "app" when you say "must reside in the namespace where the app that they configure reside" ? There's no such thing called an "app", so does that mean the "service", the "deployment", or all of them? thanks (from a relative newbie!)
-
yuranos about 2 yearsHey @AndyLorenz. Thanks. Yes, it means any workload resource(deployment, job, replicaset) that configures/controls your pods. They all reside in the same namespace.
-
Suge about 2 yearsShould I put the
LoadBalancer
service beside Ingress Controller, or besideingress
? -
yuranos about 2 yearsIf you mean k8s service, then it's a separate level of abstraction, below your ingress. Ingress usually contains rules to point to k8s services.