date and time synchronization among the pods and host in kubernetes

Question

I am having issue with date and time in kubernetes cluster. Cluster is setup in the data center using the kubeadm. My host server time is sync using the NTP, though i have synced it after configuring the cluster. Now all the pods created within my cluster will have wrong time. So the cause for it seems to be the docker taking the UTC timezone. For the temporary solution, i use volume mount /etc/localtime with the hostmeachine in the pods which we create but it seems not feasible for the application i install using helm from helm repo. Is there any way to fix this issue? I don't want every pods have the volume mounts for the correct time. Is there any way through which the docker gets timezone from the host machine.

FYI the k8s cluster is setup upon the CentOS 7. They are VM created over the EXSi. Thank You

Solution

It's not broken. It's working as designed.

Clock in a container is the same as on the host machine because it’s controlled by the kernel of that machine.

Timezone is controlled by the OS layer so it may be different inside the container. The way around it is using specific timezone config and hostPath volume to set specific timezone.

apiVersion: v1
kind: Pod
metadata:
  name: busybox-sleep
spec:
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    volumeMounts:
    - name: tz-config
      mountPath: /etc/localtime
  volumes:
    - name: tz-config
      hostPath:
        path: /usr/share/zoneinfo/Europe/Prague
        type: File

Because you are using helm, you should check the documentation for the image you are using and look for a timezone variable that you could change so you can put that in your value.yaml or use --set option when deploying.

I recommend reading Kubernetes Container Timezone Management.

Comments

[Crou]

What do you mean when saying "pods ... will have wrong time"? Can you provide an example of time on the cluster and inside the pod?

[Susanta Gautam]

Thank you for your comment. I have edited the post so that you can better understand it. The issue seems to be with the docker runtime engine which is taking the UTC timezone form the EXSi. I talked to EXSi guy but he told me timezone cannot be changed on the exsi host but time there is synced. Is there any solution for it?

[Susanta Gautam]

thank you for your answer. I am getting issue cause the docker is not taking the host timezone while the pods are created but the timezone they are taking is from the EXSi host over which the CentOS 7 box are used to create the cluster. Is there any way to force the docker to take the CentOS box timezone?