AWS CLI throws "Unable to locate credentials", the second time it's run

Question

I'm trying to download some files from S3 to an EC2 instance using the AWS CLI (1.8.7) on startup using a User-Data Script. This instance has a IAM Role & Instance Profile with the appropriate permissions. Sometimes, the CLI will fail with the error "Unable to locate credentials". It happens often, (not every time), but always after a separate S3 command, which worked. Here's the output of the script:

++ date +%r
03:24:10 AM
++ aws s3 cp s3://non-public-bucket-1/15mb-zip-file.zip ./15mb-zip-file.zip
Completed 1 of 1 part(s) with 1 file(s) remaining^Mdownload: s3://non-public-bucket-1/15mb-zip-file.zip to 15mb-zip-file.zip
++ date +%r
03:24:14 AM
++ unzip 15mb-zip-file.zip

# Snip

++ date +%r
03:26:01 AM
++ curl http://169.254.169.254/latest/meta-data/instance-id
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    10  100    10    0     0  13831      0 --:--:-- --:--:-- --:--:-- 10000
i-12345678++ date +%r
03:26:01 AM
++ aws s3 cp s3://non-public-bucket-2/my-small-text-file.json ./output.json
download failed: s3://non-public-bucket-2/my-small-text-file.json to output.json Unable to locate credentials

++ date +%r
03:26:34 AM

Any idea what could cause this?

Edit: This is an Amazon Linux server.

Solution

I had the same issue, even contacted support and they could not provide an answer. Exporting credentials to environmental variables does not work for aws s3 ... commands. You have to either use the aws configure util or create a ~/.aws/config file like this one:

[default]
aws_access_key_id=foo
aws_secret_access_key=bar
region=us-west-2

Please reference AWS documentation at http://docs.aws.amazon.com/cli/latest/topic/config-vars.html

Not only did I find it failing to execute at all with the environment exports on EC2 launch via the User Data but also post launch from the command line on a fully deployed running instance.

Comments

[EEAA]

Have you tried putting a 10 second delay before trying to fetch the files? Perhaps the IAM credentials aren't in place yet the first time the command is run.

[sj26]

I'm having exactly the same issue with Ubuntu 16.04 LTS.

[davidrmcharles]

What is ~ in this case? I tried echo $USER and echo $HOME in my User Data script and both of these variables come out blank.