Eureka and Kubernetes
Solution 1
How can I setup something like eureka.client.serviceUri?
You have to have a Kubernetes service on top of the eureka pods/deployments which then will provide you a referable IP address and port number. And then use that referable address to look up the Eureka service, instead of "8761".
To address further question about HA configuration of Eureka
You shouldn't have more than one pod/replica of Eureka per k8s service (remember, pods are ephemeral, you need a referable IP address/domain name for eureka service registry). To achieve high availability (HA), spin up more k8s services with one pod in each.
- Eureka service 1 --> a single pod
- Eureka Service 2 --> another single pod
- ..
- ..
- Eureka Service n --> another single pod
So, now you have referable IP/Domain name (IP of the k8s service) for each of your Eureka.. now it can register each other.
Feeling like it's an overkill? If all your services are in same kubernetes namespace you can achieve everything (well, almost everything, except client side load balancing) that eureka offers though k8s service + KubeDNS add-On. Read this article by Christian Posta
Edit
Instead of Services with one pod each, you can make use of StatefulSets as Stefan Ocke pointed out.
Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.
Solution 2
Regarding HA configuration of Eureka in Kubernetes: You can (meanwhile) use a StatefulSet for this instead of creating a service for each instance. The StatefulSet guarantees stable network identity for each instance you create. For example, the deployment could look like the following yaml (StatefulSet + headless Service). There are two Eureka instances here, according to the DNS naming rules for StatefulSets (assuming namespace is "default"):
eureka-0.eureka.default.svc.cluster.local and
eureka-1.eureka.default.svc.cluster.local
As long as your pods are in the same namespace, they can reach Eureka also as:
- eureka-0.eureka
- eureka-1.eureka
Note: The docker image used in the example is from https://github.com/stefanocke/eureka. You might want to chose or build your own one.
---
apiVersion: v1
kind: Service
metadata:
name: eureka
labels:
app: eureka
spec:
ports:
- port: 8761
name: eureka
clusterIP: None
selector:
app: eureka
---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
name: eureka
spec:
serviceName: "eureka"
replicas: 2
selector:
matchLabels:
app: eureka
template:
metadata:
labels:
app: eureka
spec:
containers:
- name: eureka
image: stoc/eureka
ports:
- containerPort: 8761
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
# Due to camelcase issues with "defaultZone" and "preferIpAddress", _JAVA_OPTIONS is used here
- name: _JAVA_OPTIONS
value: -Deureka.instance.preferIpAddress=false -Deureka.client.serviceUrl.defaultZone=http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/
- name: EUREKA_CLIENT_REGISTERWITHEUREKA
value: "true"
- name: EUREKA_CLIENT_FETCHREGISTRY
value: "true"
# The hostnames must match with the the eureka serviceUrls, otherwise the replicas are reported as unavailable in the eureka dashboard
- name: EUREKA_INSTANCE_HOSTNAME
value: ${MY_POD_NAME}.eureka
# No need to start the pods in order. We just need the stable network identity
podManagementPolicy: "Parallel"
Solution 3
@Stefan Ocke i'm trying to the same setup the same, but with my own image of eureka server. but i keep getting this error
Request execution failed with message: java.net.ConnectException: Connection refused (Connection refused)
2019-09-27 06:27:03.363 ERROR 1 --- [ main] c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://eureka-1.eureka:8761/eureka/}
Here are configurations:
Eureka Spring Properties:
server.port=${EUREKA_PORT}
spring.security.user.name=${EUREKA_USERNAME}
spring.security.user.password=${EUREKA_PASSWORD}
eureka.client.register-with-eureka=true
eureka.client.fetch-registry=true
eureka.instance.prefer-ip-address=false
eureka.server.wait-time-in-ms-when-sync-empty=0
eureka.server.eviction-interval-timer-in-ms=15000
eureka.instance.leaseRenewalIntervalInSeconds=30
eureka.instance.leaseExpirationDurationInSeconds=30
eureka.instance.hostname=${EUREKA_INSTANCE_HOSTNAME}
eureka.client.serviceUrl.defaultZone=http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/
StatefulSet Config:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: eureka
spec:
serviceName: "eureka"
podManagementPolicy: "Parallel"
replicas: 2
selector:
matchLabels:
app: eureka
template:
metadata:
labels:
app: eureka
spec:
containers:
- name: eureka
image: "my-image"
command: ["java", "-Djava.security.egd=file:/dev/./urandom", "-jar","/app/eureka-service.jar"]
ports:
- containerPort: 8761
env:
- name: EUREKA_PORT
value: "8761"
- name: EUREKA_USERNAME
value: "theusername"
- name: EUREKA_PASSWORD
value: "thepassword"
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: EUREKA_INSTANCE_HOSTNAME
value: ${MY_POD_NAME}.eureka
Service Config:
apiVersion: v1
kind: Service
metadata:
name: eureka
labels:
app: eureka
spec:
clusterIP: None
selector:
app: eureka
ports:
- port: 8761
targetPort: 8761
Ingress Controller:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: eureka
servicePort: 8761
Solution 4
I got exactly this problem and it got resoled by adding an environment variable for pod. This has the answer. Sample env variable for my pod is shown below,
Solution 5
You have to install a kubernetes kube-dns server to resolve names with their IPs, and then expose your eureka pods as a service. (see kubernetes docs) for more infos to how to create dns and services. @random_dude, what will be the case if i used to create 2 or 3 replicas of eureka? it turned out that when i mount a micro-service 'X' i will be registred in all eureka replicas, but when it becomes down, only one replicas gets the update ! the others still consider the micro-service instance as running
Andrew Rutter
I am a software architect and developer with almost 20 years experience in Enterprise, Mobile and Web Application Development. As president of Creative Clarity Inc, I have the pleasure of working with companies large and small to help develop innovative IT solutions.
Updated on July 09, 2022Comments
-
Andrew Rutter almost 2 years
I am putting together a proof of concept to help identify gotchas using Spring Boot/Netflix OSS and Kubernetes together. This is also to prove out related technologies such as Prometheus and Graphana.
I have a Eureka service setup which is starting with no trouble within my Kubernetes cluster. This is named discovery and has been given the name "discovery-1551420162-iyz2c" when added to K8 using
For my config server, I am trying to use Eureka based on a logical URL so in my bootstrap.yml I have
server: port: 8889 eureka: instance: hostname: configserver client: registerWithEureka: true fetchRegistry: true serviceUrl: defaultZone: http://discovery:8761/eureka/ spring: cloud: config: server: git: uri: https://github.com/xyz/microservice-config
and I am starting this using
kubectl run configserver --image=xyz/config-microservice --replicas=1 --port=8889
This service ends up running named as configserver-3481062421-tmv4d. I then see exceptions in the config server logs as it tries to locate the eureka instance and cannot.
I have the same setup for this using docker-compose locally with links and it starts the various containers with no trouble.
discovery: image: xyz/discovery-microservice ports: - "8761:8761" configserver: image: xyz/config-microservice ports: - "8888:8888" links: - discovery
How can I setup something like eureka.client.serviceUri so my microservices can locate their peers without knowing fixed IP addresses within the K8 cluster?
-
so-random-dude over 7 yearsI know I am a bit late to answer this, I missed the notification You shouldn't have more than one pod/replica of Eureka per k8s service (remember, pods are ephemeral, you need a referable IP address/domain name for eureka service registry).. To achieve high availability (HA), spin up more k8s services with one pod in each.
-
Govind Kailas almost 6 yearsWould this work?
-Deureka.client.serviceUrl.defaultZone=http://eureka-0.eureka.${CURRENT_NAME_SPACE}.svc.cluster.local:8761/eureka/,http://eureka-1.eureka.${CURRENT_NAME_SPACE}.svc.cluster.local:8761/eureka/
Is there any default variables available for getting the current namespace? -
Stefan Ocke almost 6 yearsGood question. I think the best way would be to omit the namespace:
-Deureka.client.serviceUrl.defaultZone=http://eureka-0.eureka:8761/eureka/,http://eureka-1.eureka:8761/eureka/
. However, I can remember that did not work for me. But I did not really search for the reason. Maybe you can give it a try. -
Stefan Ocke almost 6 years@GovindKailas , I updated and tested the example. I now uses the short versions of the hostnames. The trick is not only to change defaultZone, but also change EUREKA_INSTANCE_HOSTNAME accordingly to match the values in default zone. Otherwise, Eureka will report "unavailable replicas". Note that I also changed the docker image, since the one I used before does not exist anymore and was quite outdated, too.
-
Govind Kailas almost 6 yearsThis is a very good improvement and neat too. I just tried this and the only problem I could think is people may get confused about the service URL of Eureka. Thanks for the help.
-
inix over 5 yearsThank you for this solution. I have a question: How can I expose this statefulset so that I can check whether those pods a registered correctly?
-
Stefan Ocke over 5 yearsYou can create a second service with clusterIp that also points to your eureka pods. This service can then be exposed by loadbalancer, ingress or other means.
-
rajesh023 over 4 years@StefanOcke can you help me out with my current setup mentioned below ?
-
rajesh023 over 4 yearsfigured it out myself, I was using authentication on the eureka server and forgot to pass the credentials when registering with it.
-
SRK over 3 yearsHow to add an entry to JAVA_OPTIONS and -Deureka.client.serviceUrl.defaultZone above whenever HPA adds one more eureka instance? Will it always have to be manual? Is it possible to add an entry to the default zone list automatically with HPA
-
Stefan Ocke over 3 years@SRK, I don't know how to do this. But if you even have to autoscale your service discovery, you have likely reached the point to finally get rid of Eureka and use Kubernetes services and DNS instead. I have always considered Eureka in Kubernetes as some valid approach during migration of Spring Cloud applications but not as a permanent solution.
-
akub almost 3 yearsThanks, next time add the env as a code (so it can be copied/pasted)