Terraform AWS S3 to Lambda Notification Trigger

Question

I am trying to codify lambda function to be triggered using S3. Creation of Lambda function is successful, but the "aws_s3_bucket_notification" resource fails at terraform:apply with MethodNotAllowed error with status code 405 :

Error: Error applying plan:
20-Sep-2018 15:23:53    1 error(s) occurred:
20-Sep-2018 15:23:53    * aws_s3_bucket_notification.my-trigger: 1 error(s) occurred:
20-Sep-2018 15:23:53    * aws_s3_bucket_notification.my-trigger: Error putting S3 notification configuration: MethodNotAllowed: The specified method is not allowed against this resource.
20-Sep-2018 15:23:53            status code: 405, request id:<hidden>, host id:<hidden>

Here's my code to setup the notification trigger:

resource "aws_s3_bucket_notification" "my-trigger" {
  bucket = "my-bucket"

  lambda_function {
    lambda_function_arn = "${aws_lambda_function.my-function.arn}"
    events              = ["s3:ObjectCreated:*"]
    filter_prefix       = "file-prefix"
    filter_suffix       = "file-extension"
  }
}

Here's the permission to trigger Lambda from S3:

resource "aws_lambda_permission" "s3-lambda-permission" {
  statement_id  = "AllowExecutionFromS3Bucket"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.my-function.arn}"
  principal     = "s3.amazonaws.com"
  source_arn    = "arn:aws:s3:::my-bucket"
}

Thanks in advance!

Solution

So I tried the same setting as yours,

...
// lambda resource

resource "aws_s3_bucket_notification" "my-trigger" {
    bucket = "my-bucket"

    lambda_function {
        lambda_function_arn = "${aws_lambda_function.my-function.arn}"
        events              = ["s3:ObjectCreated:*"]
        filter_prefix       = "AWSLogs/"
        filter_suffix       = ".txt"
    }
}

resource "aws_lambda_permission" "test" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.my-function.arn}"
  principal = "s3.amazonaws.com"
  source_arn = "arn:aws:s3:::my-bucket"
}

...

My S3 bucket policy is like this:

{
    "Version": "2012-10-17",
    "Id": "AWSConsole-AccessLogs-Policy-1534800162725",
    "Statement": [
        {
            "Sid": "AWSConsoleStmt-1534800162725",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my-bucket/AWSLogs/123456789012/*"
        }
    ]
}

And the s3 notification got successfully created.

So I think now you should check 2 things:

1. Make sure your aws_lambda_permission is actually created before your s3 notification. I am using terraform 0.10.2, but there might be a bug in previous versions so that this lambda permission is not created before the s3 notification.

2. Check your s3 bucket policy. It might be explicitly denying lambda access. If there is any, make sure to get rid of those.

Comments

[ydaetskcoR]

This answer is bordering on being a comment. If you could flesh it out with a worked example and remove the questioning paragraph to state what is happening instead this would be a useful answer.