Can't get AWS Lambda function to log (text output) to CloudWatch

Question

I'm trying to set up a Lambda function that will process a file when it's uploaded to an S3 bucket. I need a way to see the output of console.log when I upload a file, but I can't figure out how to link my Lambda function to CloudWatch.

I figured about by looking at the context object that my log group is /aws/lambda/wavToMp3 and the log stream is 2016/05/23/[$LATEST]hex_code_redacted. So I created that group and stream in CloudWatch, yet nothing is being logged to it.

Solution 1

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions.

{
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                 "logs:CreateLogStream",
                 "logs:PutLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
} 

Please refer to the following AWS documentation for more details http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html#lambda-intro-execution-role

Solution 2

After you update your policy, it seems that you have to update your function's settings to refresh all job instances to read new policies.

So if you just click 'test' button from Lambda console after you update your role policy in IAM, the cached Lambda instances will still have old role permissions, so you will still see no logs being written to Cloudwatch logs.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

Solution 3

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions

I already had these permissions yet it did not work.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

I changed the timeout, saved and logs still did not work.

I assigned another role and logs still did not work.

What ended up working for me was clicking "Create a custom role", then "Allow". This was it and logs started being generated but since I did not want to use a new role but my existing role, I simply assigned my existing role afterwards and it worked. So technically I should have returned back to original configuration that did not work but now it works. Go figure.

Solution 4

Apparently another necessity for logging to happen is the Lambda function must indicate completion; for instance in the Python context, the handler must return something other than None.

Solution 5

July 2020 Update !!

Logs may not be in us-east-1, try looking for lambda edge logs in different regions !!

View more solutions

Comments

[reggie3]

Thanks! I was stuck on this for an hour.

[brycem]

This is my vote for the correct answer. Nothing in the documentation will help if you're running into this AWS bug.

[alecxe]

Same here, spent like 2 hours trying to solve this. Disappointed with how buggy and absolutely not transparent and not intuitive the lambda-to-cloudwatch setup is. Thanks!

[Steve Smith]

Note the answer below by hoonoh, I needed to update settings before this new policy actually applied.

[hansaplast]

I came here because I faced the same problem, in my case the error was Log group not found. Adding it here so anyone else googling for it will find this answer

[John Chrysostom]

Thanks for this... this is where I was stuck as well.

[Cameron A. Ellis]

This is awful. Someone should seriously be competing with AWS Lambda. It's like they have no incentive to improve the service.

[Martin]

Thanks ! They should at least add a 'reload role/policy' in the lambda role editor so you don't need to do a dummy edit. Or even an option to 'propagate' a policy update.

[Michael Berry]

It'd be great if this answer could be merged into the accepted one - they're part of the same process necessary to solve the issue.

[Anton v B]

This solved it for me as well!! Had all the right IAM policies but without this, no logs were showing. Thanks a lot for this tip

[Christophe Blin]

As said by @SteveSmith, look at hoonoh answer below (which has more votes than this accepted answer) : it only works if you update the lambda after you update its policy (modifying the timeout, adding a dummy environment var, etc...)

[user2719094]

Happened to me, too, and I'm pretty sure it was because I changed the execution role to an existing role which did not have explicit permissions to create/write to the Cloudwatch log stream for my lambda.

[Krishna]

Thanks! It Worked!

[George Smith]

Awesome, thank you. It worked for me. My guess is that there is something related to the container in which the Lambda is running - because of that, the function probably does not reload the role and its policy(s).

[coderboi]

what do you mean by "allow"? Just a generic role?

[afsinka]

Thanks! In my case when I send a request with Test button on AWS UI logs are created in us-east-1, but when I send a request via my application logs are created in some other region.

[Alexander Santos]

This answer made me try the "Test" button and then the log appeared, thank you.

[C.M.]

Adding the AWSLambdaExecute group solved my issue. What's weird is it didn't need that group initially to log, but only after redeploying the lambda.