AWS ECR PULL no basic auth credentials

Question

I'm deploying Azure K8s cluster with Terraform, and the image is hosted in Amazon ECR. The deployment fails at the image pull from the ECR with the following error:

Failed to pull image "tooot.eu-west-1.amazonaws.com/app-t:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://tooot.eu-west-1.amazonaws.com/v2/app-t/manifests/latest: no basic auth credentials

the following is my kuberentes resource in the terraform template

  metadata {
    name = "terraform-app-deployment-example"
    labels {
      test = "app-deployment"
    }
  }

  spec {
    replicas = 6

    selector {
      match_labels {
        test = "app-deployment"
      }
    }

    template {
      metadata {
        labels {
          test = "app-deployment"
        }
      }

      spec {
        container {
          image = "toot.eu-west-1.amazonaws.com/app-t:latest"
          name  = "app"
        }
      }
    }
  }
}`

Solution 1

Basically you are lacking credentials to pull images from AWS.

You need to create a regcred, which contains the login credentials:

https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/

After that you need to add the regcred on your terraform configuration. I have not worked with templates, but in a deploy specification you would add a field called imagePullSecrets.

https://www.terraform.io/docs/providers/kubernetes/r/deployment.html

The imagePullSecrets description:

image_pull_secrets - (Optional) ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images used by this PodSpec. If specified, these secrets will be passed to individual puller implementations for them to use. For example, in the case of docker, only DockerConfig type secrets are honored

Solution 2

in kubernetes cluster you have to add secret which will used to login into ECR at the time of pulling image

ECR managed the token for pushing and pulling images. Token is valid for 12 hour

so kindly check for token in ECR

i have written shell script for that you can also check it out

it is getting token from aws ECR deleting old secret in kubernetes cluster and creating again new secret in kubernetes cluster. which secret will be used for to pull the image from the aws ecr.

as i am checking there is no secret in container spec option

you can check more at here :

https://github.com/harsh4870/ECR-Token-automation/blob/master/aws-token.sh

Comments

[Renm]

Wow many thanks for the script Harsh !, how should I run it though? I have tried to authenticate my self by running $(aws ecr get-login --no-include-email --region eu-west-1) and the output of that command, however I still have the same problem when i run kubectl get pods --watch i get ImagePullBackOff and ErrImagePull kubelet, aks-default-32086448-1 Failed to pull image "toot.dkr.ecr.eu-west-1.amazonaws.com/app-t:latest": rpc error: code = Unknown desc = Error response from daemon: Get toot.dkr.ecr.eu-west-1.amazonaws.com/v2/app-t/manifests/late‌​st: no basic auth credentials

[Harsh Manvar]

after running the aws ecr get-login --no-include-email --region eu-west-1 you will get one docker command as output with big token ...in your case it is issue with generating the token in kubernetes cluster...

[Harsh Manvar]

have you edited the manbifest file ??? where spec container config there ??? you have to add there secret name..secret which will store the token of ECR

[Renm]

Thanks, i'll read more about it today, i think these would take me to the right direction

[arcseldon]

Your script is a copy of this one - stackoverflow.com/a/50502171/1882064

[Harsh Manvar]

@arcseldon i used it long before also did some changes. i am sorry i have not checked answer avilable or not on So already.

[arcseldon]

@HarshManvar - not at all, good of you to share! At the end of the day, we're all copying from each other all the time. The link answers the question that Renm had too.