Docker pull: TLS handshake timeout

Question

I get this consistenly (Ubuntu 16.04 LTS):

$ docker pull nginx
Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout

However curl TLS works fine (apart from the auth error):

$ curl https://registry-1.docker.io/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}

And even a small golang program (to mimic docker) works fine:

package main
import (
    "fmt"
    "io/ioutil"
    "net/http"
)
func main() {
    resp, err := http.Get("https://registry-1.docker.io/v2/")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }
    fmt.Println("Got: ", string(body))
}

The pcap for the docker TLS timeout request:

reading from file docker-timeout.pcap, link-type LINUX_SLL (Linux cooked)
00:38:54.782452 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [S], seq 26945613, win 29200, options [mss 1460,sackOK,TS val 1609360 ecr 0,nop,wscale 7], length 0
00:38:54.878630 IP registry-1.docker.io.https > my-ubuntu.52036: Flags [S.], seq 2700732154, ack 26945614, win 26847, options [mss 1460,sackOK,TS val 947941366 ecr 1609360,nop,wscale 8], length 0
00:38:54.878691 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [.], ack 1, win 229, options [nop,nop,TS val 1609384 ecr 947941366], length 0
00:38:54.878892 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609384 ecr 947941366], length 155
00:38:55.175931 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609459 ecr 947941366], length 155
00:38:55.475954 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609534 ecr 947941366], length 155
00:38:56.076327 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609684 ecr 947941366], length 155
00:38:57.280103 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1609985 ecr 947941366], length 155
00:38:59.684095 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1610586 ecr 947941366], length 155
00:39:04.492102 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611788 ecr 947941366], length 155
00:39:04.879468 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [F.], seq 156, ack 1, win 229, options [nop,nop,TS val 1611884 ecr 947941366], length 0
00:39:04.976015 IP registry-1.docker.io.https > my-ubuntu.52036: Flags [.], ack 1, win 105, options [nop,nop,TS val 947943890 ecr 1609384,nop,nop,sack 1 {156:157}], length 0
00:39:04.976073 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611909 ecr 947943890], length 155
00:39:05.275922 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1611984 ecr 947943890], length 155
00:39:05.876104 IP my-ubuntu.52036 > registry-1.docker.io.https: Flags [P.], seq 1:156, ack 1, win 229, options [nop,nop,TS val 1612134 ecr 947943890], length 155

What could possibly be going wrong?

Solution 1

net/http: TLS handshake timeout means that you have slow internet connection. Default value of connection timeout is too small for your environment. Unfortunately docker don't have any settings that allows you change connection timeout. You may try to create your own registry cache somewhere else and pull images from it.

Solution 2

I experience the same problem. Then answer of Azamat Hackimov pointed me in the right direction. My machine is somewhat slow, especially at boot time, when I want to launch the service. Therefore the short timeout kicks in and kills my request.

This is my workaround:

docker pull $IMAGE || docker pull $IMAGE ||  docker pull $IMAGE || docker pull $IMAGE

Simply hammer the server with request. Usually the second one is successful for me.

Solution 3

If you are using a private registry, you need to place the certificate for that under:

/etc/docker/certs.d/registryname/ca.crt

registryname will change accordingly.

Also, please change your MTU size to 1300, this was also one thing I did to resolve the error. Registry one I believe you might have already done.

Command for MTU change:

ip link set dev eth0 mtu 1300

MTU size is important to check to avoid this error if your internet speed is really good.

Solution 4

I had an equal Problem, by using docker run hello-world 1st time, which results in downloading an image using https://registry-1.docker.io/v2/, which end to

docker: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: net/http: TLS handshake timeout.

Searching the web for hours and found out, that this happens at some users with ubuntu 18.04 and the current docker release, behind an proxy. A workaround is to remove all the https-proxy configuration in order to leave only the http-proxy configuration, to force an http (not https) download.

Don't know, what the real reason is.

(by the way: I had an equal "TLS handshake" problem with composer and packagist. This was because of an missing cacert.pem file, which was not provided by ubuntu by default. Maybe this docker-problem is going into the same direction?)

Solution 5

In my case my server was behind the nat and proxy and set to auto detect proxy what i have done on current terminal i have export proxy settings

root@k8master:~/runner# export http_proxy="http://192.168.10.208:3128"
root@k8master:~/runner# docker pull gitlab/gitlab-runner:latest
latest: Pulling from gitlab/gitlab-runner
7b722c1070cd: Pull complete 
5fbf74db61f1: Pull complete 
ed41cb72e5c9: Pull complete 
7ea47a67709e: Pull complete 
ae336ceeca88: Pull complete 
f9f79780e6cf: Pull complete 
67e622273f37: Pull complete 
bc84c40af701: Pull complete 
69e36092e9de: Pull complete 
Digest: sha256:b1f5387942aaaf8c220f6613a1e96ba2cbcb6c58a5e47ca0df8ae3216720a15e
Status: Downloaded newer image for gitlab/gitlab-runner:latest

View more solutions

Comments

[Willem]

I swapped my dsl modem and the problem was gone... I suspect it was a mtu problem.

[Nikhil Chilwant]

Well, speedtest.net and fast.com show my internet speed is 90 Mbit/s. Is that slow? I am pulling python:2.7-slim image. I am able to pull hello-world from hub but not the python one. It gives me same TLS handshake timeout error.

[Barafu Albino]

Before people start doing something dramatic I want to remark: having a typo in image name also produces the same error. Very descriptive.

[The Bndr]

An TLS handshake timeout mostly does not mean, the internet connection is to slow. This message will also appear, if the TLS handshake stops for different reasons. For example, if one side don't like to talk with an specific TLS version or because of an certificate-problem.

[Gonzalo Cao]

Not a definitive solution but good as a temporary workaround

[wisbucky]

That is a good tip, but not having the certificate would result in a x509: certificate signed by unknown authority error, not TLS handshake timeout.

[Pamungkas Jayuda]

Thanks its work.

[andybuckley]

Simply restarting the daemon can solve this in some circumstances, where the basic configuration and network are otherwise fine. Not sure why: perhaps stuck in an inconsistent state due to updates of system (networking?) packages that didn't restart Docker. Good ol' "turn it off, then back on again" ;-)

[eemilk]

I'am using WSL2 ubuntu 20.04 and I can say this was just the problem I had. Changing my https_proxy to our corporate http_proxy resolved my problem. Now I have in both proxies our http_proxy configured. In addition I was using buildah and podman

[Arief Karfianto]

Tricky but works

[DerShodan]

I feel personally offended that this actually worked :D

[Admin]

This is proper IT engineering and should be the accepted solution, as well as the solution to every problem in life ever

[Admin]

unbelievable but this worked for me too