Specify log group for an AWS lambda?

Question

Is there a way to specify the CloudWatch log group that an AWS lambda logs to? It seems to be generated directly from the lambda name; however, it would be especially convenient to, for example, aggregate multiple lambdas to a single log group. We are especially interested in specifying the log group when the lambda is created by a CloudFormation template.

Solution 1

I don't think that is possible.

Even if it were possible, each AWS Lambda instance would still write to its own log-stream. And though different invocations of the same lambda can write to the same log-stream (when the lambda instance is reused), this will definitely not be the case for different lambdas (since they must use different lambda instances).

As a result, you must have a tool that aggregates multiple log-stream. If so, what's the problem with making it a little more generic, so that it can aggregate log-streams from different log groups?

Solution 2

Actually, maybe you can, to an extent at least. I too was in search of the answer and gave this a try. Here's a snippet of two resources; the lambda function and the log group:

"MyLambdaFunction": {
    "Type": "AWS::Lambda::Function",
    "DependsOn": "ReadWriteRole",
    "Properties": {
        //snip
    }
},

"MyLambdaFunctionLogGroup": {
    "Type": "AWS::Logs::LogGroup",
    "DependsOn": "MyLambdaFunction",
    "Properties": {
        "LogGroupName": {"Fn::Join": ["", ["/aws/lambda/", {"Ref": "MyLambdaFunction"}]]},
        "RetentionInDays": 14
    }
},

I found that the log group was created with a retention of 14 days as indicated. When the lambda function runs, it does create log streams in this group. When I deleted the stack, however, it seems that the log groups is not deleted, and the retention is now set to never expire. Perhaps that's good enough so the streams don't get too out of hand...

Solution 3

Creating the log group as mentioned as one of the answers works. To keep the retention policy after the stack is deleted, just add a DeletionPolicy.

"MyLambdaFunctionLogGroup": {
  "Type": "AWS::Logs::LogGroup",
  "DependsOn": "MyLambdaFunction",
  "DeletionPolicy": "Retain",
  "Properties": {
    "LogGroupName": {"Fn::Join": ["", ["/aws/lambda/", {"Ref": "MyLambdaFunction"}]]},
    "RetentionInDays": 14
  }
}

Solution 4

I find that @lingrlongr's answer is partially correct.

First, to answer the original question, you cannot specify a custom log group name for the lambda function to write to.

The lambda log group name always follows this pattern:

/aws/lambda/<function-name>

The lambda function will first check if a log group exists with this name.

• If it exists, then it will use that.
• If it does not exist, it will create a log group with that pattern.

Hence, if you want to add settings, such as RetentionInDays and SubscriptionFilter, make sure your CloudFormation or SAM template creates the LogGroup before the lambda function. If your lambda function is created first, an error will be thrown when creating the LogGroup saying that the LogGroup already exists. So, the lambda function should have DependsOn: LogGroup instead of the other way round.

Also, make sure you are not using Ref or GetAtt to reference the lambda function inside the LogGroup because that creates an implicit dependency on the lambda function causing the lambda function being created before the LogGroup.

Solution 5

I attached LogGroup to Serverless function in SAM template as follows:

    MyFuncLogGroup:
        Type: AWS::Logs::LogGroup
        Properties:
            LogGroupName: '/aws/lambda/stackName-env-myFunc-v1'
            RetentionInDays: 30

    MyFunc:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: 'stackName-env-myFunc-v1'
            ...

Some users mentioned the requirement of DependsOn property but it is not required in my experience when using SAM. The only requirement is that the LogGroupName must be /aws/lambda/<FunctionName>. SAM will create the log group before the lambda function as long as your are not referencing the function's logical ID in the LogGroup

Also, if you are adding LogGroup to exisiting function, you can do so by simply updating your template.yaml with the LogGroup resource like above and add FunctionName property to the function resource. Obviously, the FunctionName should be different from your existing lambda function name that appears in your lambda's ARN.

WARNING: FunctionName property requires replacement. Your existing lambda will be deleted and new one will be created.

View more solutions