Error while configuring Terraform S3 Backend

Question

I am configuring S3 backend through terraform for AWS.

terraform {
  backend "s3" {}
}

On providing the values for (S3 backend) bucket name, key & region on running "terraform init" command, getting following error

"Error configuring the backend "s3": No valid credential sources found for AWS Provider. Please see https://terraform.io/docs/providers/aws/index.html for more information on providing credentials for the AWS Provider Please update the configuration in your Terraform files to fix this error then run this command again."

I have declared access & secret keys as variables in providers.tf. While running "terraform init" command it didn't prompt any access key or secret key.

How to resolve this issue?

Solution 1

When running the terraform init you have to add -backend-config options for your credentials (aws keys). So your command should look like:

terraform init -backend-config="access_key=<your access key>" -backend-config="secret_key=<your secret key>"

Solution 2

I also had the same issue, the easiest and the secure way is to fix this issue is that configure the AWS profile. Even if you properly mentioned the AWS_PROFILE in your project, you have to mention it again in your backend.tf.

my problem was, I have already set up the AWS provider in the project as below and it is working properly.

provider "aws" {
region = "${var.AWS_REGION}"
profile = "${var.AWS_PROFILE}"
}

but end of the project I was trying to configure the S3 backend configuration file. therefore I have run the command terraform init and I also got the same error message.

Error: error configuring S3 Backend: no valid credential sources for S3 Backend found.

Note that is not enough for the terraform backend configuration. you have to mention the AWS_PROFILE in the backend file as well.

• Full Solution

I'm using the terraform latest version at this moment. it's v0.13.5.

please see the provider.tf

provider "aws" {
region = "${var.AWS_REGION}"
profile = "${var.AWS_PROFILE}" # lets say profile is my-profile
}

for example your AWS_PROFILE is my-profile then your backend.tf should be as below.

terraform {
    backend "s3" {
    bucket = "my-terraform--bucket"
    encrypt = true
    key = "state.tfstate"
    region = "ap-southeast-2"
    profile = "my-profile" # you have to give the profile name here. not the variable("${var.AWS_PROFILE}")
  }
}

then run the terraform init

Solution 3

I've faced a similar problem when renamed profile in AWS credentials file. Deleting .terraform folder, and running terraform init again resolved the problem.

Solution 4

Don't - add variables for secrets. It's a really really bad practice and unnecessary.

Terraform will pick up your default AWS profile, or use whatever AWS profile you set AWS_PROFILE too. If this in AWS you should be using an instance profile. Roles can be done too.

If you hardcode the profile into your tf code then you have to have the same profile names where-ever you want to run this script and change it for every different account its run against.

Don't - do all this cmdline stuff, unless you like wrapper scripts or typing. Do - Add yourself a remote_state.tf that looks like

terraform {
  backend "s3" {
    bucket         = "WHAT-YOU-CALLED-YOUR-STATEBUCKET"
    key            = "mykey/terraform.tfstate"
    region         = "eu-west-1"
  }
}

now when your terraform init:

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes.

The values in the provider aren't relevant to the perms for the remote_state and could even be different AWS accounts (or even another cloud provider).

View more solutions

Comments

[MKM]

I have the provider block in my code. Please find the code below provider "aws" { access_key = "${var.access_key}" secret_key = "${var.secret_key}" region = "${var.region}" } terraform { backend "s3" {} }

[codinghaus]

Sorry for pointing in the wrong direction. I think I know what the problem is now and edited my answer accordingly.

[MKM]

thanks for the help. terraform init is working fine now. On executing terraform plan command it is throwing a new error "Failed to load backend: Error configuring the backend "s3": Not a valid region:" should I provide -backend-config while running terraform plan command as well?

[codinghaus]

Nice! No, backend-config is absolutely only needed during terraform init. Regarding the new error: Make sure that you have region=<region> in your backend "s3" {} block.

[MKM]

I am trying to execute terraform plan -var-file command and getting error. What configuration details must be added in the file given here? I have created a sample .tfvars file with access & secret keys added to it. Is this correct or anything else is required?

[codinghaus]

Please give more information. What is the exact error you get? Normally if any variables are missing terraform explicitly asks for the value of those variables during terraform plan and terraform apply. Are you talking about variables which you defined as variable "<variable_name>" {} ?

[MKM]

just by executing terraform plan command I am getting the following error "Error: Error loading state: InvalidParameter: 1 validation error(s) found. - minimum field size of 1, GetObjectInput.Key." Hence, I tried using terraform plan -var-file= command. But, I am not sure what configurations to be mentioned in that file.

[codinghaus]

I don't think that is has to do something with your var-file. The error sounds more like you are missing the key attribute inside your backend block (which should look something like terraform { backend "s3" { bucket = "<name of your existing backend bucket>" key = "terraform.tfstate" region = "<region>" } } ). If this does not solve your problem I think it is better to ask a separate question here on StackOverflow if needed.

[Zucchini]

This was the exact answer i was looking for. Thank you.

[Willemoes]

Thanks a lot, this was the correct answer to me

[Yayotrón]

This right here is the correct answer, thanks a lot man!