"x509: certificate signed by unknown authority" when running kubelet
Solution 1
in general the solution was to create another etcd2 port that attaches to loopback device of each machine and works on http
instead of https
. more information at calico-policy-controller requests etcd2 certificates of a different coreos server
Solution 2
The kubelet documentation says that the --tls-cert-file
flag needs the CA be concatenated after the certificate. In you case it is the apiserver.pem
:
--tls-cert-file File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir.
If I read you certificate generation correctly, the apiserver.pem
doesn't contain the root ca.
Related videos on Youtube
Comments
-
ufk over 1 year
I'm trying to install kubernetes with kubelet 1.4.5 on CoreOS beta (1192.2.0).
I'm using a slightly modified version of the controller and worker install scripts from https://github.com/coreos/coreos-kubernetes/tree/master/multi-node/generic
so in general I created the licenses on Gentoo Linux using the following bash script:
#!/bin/bash export MASTER_HOST=coreos-2.tux-in.com export K8S_SERVICE_IP=10.3.0.1 export WORKER_IP=10.79.218.3 export WORKER_FQDN=coreos-3.tux-in.com openssl genrsa -out ca-key.pem 2048 openssl req -x509 -new -nodes -key ca-key.pem -days 10000 -out ca.pem -subj "/CN=kube-ca" openssl genrsa -out apiserver-key.pem 2048 openssl req -new -key apiserver-key.pem -out apiserver.csr -subj "/CN=kube-apiserver" -config openssl.cnf openssl x509 -req -in apiserver.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out apiserver.pem -days 365 -extensions v3_req -extfile openssl.cnf openssl genrsa -out ${WORKER_FQDN}-worker-key.pem 2048 openssl req -new -key ${WORKER_FQDN}-worker-key.pem -out ${WORKER_FQDN}-worker.csr -subj "/CN=${WORKER_FQDN}" -config worker-openssl.cnf openssl x509 -req -in ${WORKER_FQDN}-worker.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out ${WORKER_FQDN}-worker.pem -days 365 -extensions v3_req -extfile worker-openssl.cnf openssl genrsa -out admin-key.pem 2048 openssl req -new -key admin-key.pem -out admin.csr -subj "/CN=kube-admin" openssl x509 -req -in admin.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out admin.pem -days 365 echo done
and this is
openssl.cnf
[req] req_extensions = v3_req distinguished_name = req_distinguished_name [req_distinguished_name] [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment subjectAltName = @alt_names [alt_names] DNS.1 = coreos-2.tux-in.com DNS.2 = coreos-3.tux-in.com IP.1 = 10.3.0.1 IP.2 = 10.79.218.2 IP.3 = 10.79.218.3
and this is my
worker-openssl.cnf
[req] req_extensions = v3_req distinguished_name = req_distinguished_name [req_distinguished_name] [ v3_req ] basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment subjectAltName = @alt_names [alt_names] IP.1 = 10.79.218.3 DNS.1 = coreos-3.tux-in.com
My controller machine is
coreos-2.tux-in.com
which resolves to the lan ip10.79.218.2
my worker machine is
coreos-3.tux-in.com
which resolves to lan ip10.79.218.3
it created the licenses just fine. but when I use them and install the controller script on the main machine, i see that when I run
journalctl -xef -u kubelet
and I noticed the following messages:Nov 08 21:24:06 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:06.805868 2018 event.go:208] Unable to write event: 'x509: certificate signed by unknown authority' (may retry after sleeping) Nov 08 21:24:06 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:06.950827 2018 reflector.go:203] pkg/kubelet/kubelet.go:384: Failed to list *api.Service: Get https://coreos-2.tux-in.com:443/api/v1/services?resourceVersion=0: x509: certificate signed by unknown authority Nov 08 21:24:07 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:07.461042 2018 reflector.go:203] pkg/kubelet/config/apiserver.go:43: Failed to list *api.Pod: Get https://coreos-2.tux-in.com:443/api/v1/pods?fieldSelector=spec.nodeName%3D10.79.218.2&resourceVersion=0: x509: certificate signed by unknown authority Nov 08 21:24:07 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:07.461340 2018 reflector.go:203] pkg/kubelet/kubelet.go:403: Failed to list *api.Node: Get https://coreos-2.tux-in.com:443/api/v1/nodes?fieldSelector=metadata.name%3D10.79.218.2&resourceVersion=0: x509: certificate signed by unknown authority Nov 08 21:24:08 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:08.024366 2018 reflector.go:203] pkg/kubelet/kubelet.go:384: Failed to list *api.Service: Get https://coreos-2.tux-in.com:443/api/v1/services?resourceVersion=0: x509: certificate signed by unknown authority Nov 08 21:24:08 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:08.171170 2018 eviction_manager.go:162] eviction manager: unexpected err: failed GetNode: node '10.79.218.2' not found Nov 08 21:24:08 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:08.543619 2018 reflector.go:203] pkg/kubelet/kubelet.go:403: Failed to list *api.Node: Get https://coreos-2.tux-in.com:443/api/v1/nodes?fieldSelector=metadata.name%3D10.79.218.2&resourceVersion=0: x509: certificate signed by unknown authority Nov 08 21:24:08 coreos-2.tux-in.com kubelet-wrapper[2018]: E1108 21:24:08.543926 2018 reflector.go:203] pkg/kubelet/config/apiserver.go:43: Failed to list *api.Pod: Get https://coreos-2.tux-in.com:443/api/v1/pods?fieldSelector=spec.nodeName%3D10.79.218.2&resourceVersion=0: x509: certificate signed by unknown authority
-
ufk almost 7 yearsthanks for your answer. i appended the ca to the apiserver.pem file but the results are exactly the same.
-
ufk almost 7 yearshi. thanks for your answer. i closed the worker server and I still get the same error messages. it means that it doesn't really relate to the worker configuration no ?
-
Yves Blusseau almost 7 yearsI think you have invert the worker/master dns name and the worker/master ip. You have written that the worker is coreos-3.tux-in.com with ip 10.79.218.3 (in the bash script to generate the certificates) but after you have written: "my worker machine is coreos-2.tux-in.com which resolves to lan ip 10.79.218.3"
-
ufk almost 7 yearshi. thanks for your answer. so the problem is that they're self signed? if i want to keep using self signed certificates, what do i do? how do i get rid of this error ?
-
stephanlindauer almost 7 yearswhat do you mean by append?