Error registering: NoCredentialProviders: no valid providers in chain ECS agent error

Question

Im trying to use EC2 Container service. Im using terraform for creating it. I have defined a ecs cluster, autoscaling group, launch configuration. All seems to work. Except one thing. The ec2 instances are creating, but they are not register in the cluster, cluster just says no instances available.

In ecs agent log on created instance i found logs flooded with one error:

Error registering: NoCredentialProviders: no valid providers in chain

The ec2 instances are created with a proper role ecs_role. This role has two policies, one of them is following, like docs required:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecs:CreateCluster",
        "ecs:DeregisterContainerInstance",
        "ecs:DiscoverPollEndpoint",
        "ecs:Poll",
        "ecs:RegisterContainerInstance",
        "ecs:StartTelemetrySession",
        "ecs:Submit*",
        "ecs:StartTask"
      ],
      "Resource": "*"
    }
  ]
}

Im using ami ami-6ff4bd05. Latest terraform.

Solution 1

It was a problem with trust relationships in the role as the role should include ec2. Unfortunately the error message was not all that helpful.

Example of trust relationship:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": ["ecs.amazonaws.com", "ec2.amazonaws.com"]
      },
      "Effect": "Allow"
    }
  ]
}

Solution 2

Make sure you select the correct ECS role in the launch configuration.

Solution 3

You might want to add AmazonEC2RoleforSSM (or AmazonSSMFullAccess) to your EC2's role.