kubectl : Unable to connect to the server : dial tcp 192.168.214.136:6443: connect: no route to host

Question

I recently installed kubernetes on VMware and also configured few pods , while configuring those pods , it automatically used IP of the VMware and configured. I was able to access the application during that time but then recently i rebooted VM and machine which hosts the VM, during this - IP of the VM got changed i guess and now - I am getting below error when using command :

kubectl get pod -n

userX@ubuntu:~$ kubectl get pod -n NameSpaceX
Unable to connect to the server: dial tcp 192.168.214.136:6443: connect: no route to host```

userX@ubuntu:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: dial tcp 192.168.214.136:6443: connect: no route to host

kubectl cluster-info as well as other related commands gives same output. in VMware workstation settings, we are using network adapter which is sharing host IP address setting. We are not sure if it has any impact.

We also tried to add below entry in /etc/hosts , it is not working.

127.0.0.1 localhost \n 192.168.214.136 localhost \n 127.0.1.1 ubuntu

I expect to run the pods back again to access the application.Instead of reinstalling all pods which is time consuming - we are looking for quick workaround so that pods will get back to running state.

Solution 1

If you use minikube sometimes everything you need to do is just to restart minikube.

Run: minikube start

Solution 2

I encountered the same issue - the problem was that the master node didn't expose port 6443 outside.

Below are the steps I took to fix it.

1 ) Check IP of api-server.
This can be verified via the .kube/config file (under server field) or with:
kubectl describe pod/kube-apiserver-<master-node-name> -n kube-system.

2 ) Run curl https://<kube-apiserver-IP>:6443 and see if port 6443 is open.

3 ) If port 6443 you should get something related to the certificate like:

curl: (60) SSL certificate problem: unable to get local issuer certificate

4 ) If port 6443 is not open:
4.A ) SSH into master node.
4.B ) Run sudo firewall-cmd --add-port=6443/tcp --permanent (I'm assuming firewalld is installed).

4.C ) Run sudo firewall-cmd --reload.

4.D ) Run sudo firewall-cmd --list-all and you should see port 6443 is updated:

public
  target: default
  icmp-block-inversion: no
  interfaces: 
  sources: 
  services: dhcpv6-client ssh
  ports: 6443/tcp <---- Here
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

Solution 3

The common practice is to copy config file to the home directory

sudo cp /etc/kubernetes/admin.conf ~/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/config

Also, make sure that api-server address is valid.

server: https://<master-node-ip>:6443

If not, you can manually edit it using any text editor.

Solution 4

You need to export the admin.conf file as kubeconfig before running the kubectl commands. You may put this as your env variable

export kubeconfig=<path>/admin.conf

after this you should be able to run the kubectl command. I am hoping that your setup of K8S cluster is proper.

View more solutions

Comments

[Vicky]

Tried , no luck , i see even admin.conf has same old IP present. Does updating it directly using vi editor works ( replace IP with localhost ) ?

[Navneet Nandan Jha]

I am not sure if that will work. You can give it a try!!

[A_Suh]

@Vicky you can not use localhost as api-server address, because all your generated certificates will be not valid.

[ryanwebjackson]

Why does this solve the issue?

[jayaprakash R]

such a silly mistake that i didn't check whether the minkube started or not