How do I debug AWS Api Gateway & Lambda's "AWS/ApiGateway 5XXError"

10,641

Solution 1

How to Debug:

  1. Create an IAM role to allow API Gateway to push logs to CloudWatch. The role must have the following policy attach:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:PutLogEvents",
                "logs:GetLogEvents",
                "logs:FilterLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

With the following trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "apigateway.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
  1. In API Gateway console for your API's region: Go to settings >>> Enter in the ARN of the API Gateway-CloudWatch logging role >>> click 'Save'

  2. Go to the stage of your API. Under 'CloudWatch Settings', select 'Enable CloudWatch Logs'. Set 'Log level' to 'INFO'. Select 'Log full requests/responses data'.

enabling cloudwatch logs

  1. Redeploy your API to that stage: Go to the 'Resources' tab for your API. Select Actions >>> Deploy API.

  2. Make requests, wait a few minutes, and see what the logs say (in CloudWatch).

The Error:

The error

The Cause:

Once I enabled "Invoke with caller credentials" using Credentials: 'arn:aws:iam::*:user/*', the caller's IAM role didn't have access to invoke the lambda function. This resulted in the 500 error. Once I gave the caller's IAM role access, everything started to work properly.

Solution 2

What does the API Gateway Log show? does it show "Invalid permissions on Lambda function"? I think you'll need to include the permission creation (resource) in the CloudFormation Template. Here is one of mine:

"PERMISSIONGET": {
        "Type": "AWS::Lambda::Permission",
        "Properties": {
            "FunctionName": "createCabinet",
            "Action": "lambda:InvokeFunction",
            "Principal": "apigateway.amazonaws.com",
            "SourceArn": {
                "Fn::Join": [
                    "",
                    [
                        "arn:aws:execute-api:us-east-1:87875636623:",
                        {
                            "Ref": "APIGATEWAY"
                        },
                        "/*/GET/*"
                    ]
                ]
            }
        },
        "DependsOn": "APIDEPLOYMENT"
    }
Share:
10,641
CamHart
Author by

CamHart

Updated on June 07, 2022

Comments

  • CamHart
    CamHart almost 2 years

    I have an API Gateway resource that runs a lambda function. I'm calling the API Gateway resource using the AWS Generated SDK from my Api Gateway.

    Here's the portion of the stacktrace from my client that seems relevant:

    Caused by: com.amazonaws.mobileconnectors.apigateway.ApiClientException: {"message": "Internal server error"} (Service: DevnetcountableClient; Status Code: 500; Error Code: null; Request ID: 348e8f98-6f55-11e6-97f6-098c2caf220f)
    at com.amazonaws.mobileconnectors.apigateway.ApiClientHandler.handleResponse(ApiClientHandler.java:255) at com.amazonaws.mobileconnectors.apigateway.ApiClientHandler.invoke(ApiClientHandler.java:88)
        at java.lang.reflect.Proxy.invoke(Proxy.java:393)
        at $Proxy1.accountCreatePost(Unknown Source)
    

    Now looking at the AWS console, within my Api Gateway Dashboard I see the requests coming in and resulting in "AWS/ApiGateway 5XXError". However there are 0 logs (that I can find). My lambda function does NOT appear to be called, and no lambda logs are shown.

    Right now this is what my lambda looks like:

    module.exports.createAccount = function(event, context, cb) {
        console.log('createAccount');
        console.log(event);
        console.log(context);
        console.log(cb);
        cb(null, {status: 'SUCCESS', message: 'I ran!'});
    };
    

    What can I do to debug this?

    Edit: Okay here is the cloudformation script that's throwing everything together.

    {
       "AWSTemplateFormatVersion":"2010-09-09",
       "Description":"The AWS CloudFormation template for this Serverless application",
       "Resources":{
          "ServerlessDeploymentBucket":{
             "Type":"AWS::S3::Bucket"
          },
          "IamRoleLambda":{
             "Type":"AWS::IAM::Role",
             "Properties":{
                "AssumeRolePolicyDocument":{
                   "Version":"2012-10-17",
                   "Statement":[
                      {
                         "Effect":"Allow",
                         "Principal":{
                            "Service":[
                               "lambda.amazonaws.com"
                            ]
                         },
                         "Action":[
                            "sts:AssumeRole"
                         ]
                      }
                   ]
                },
                "Path":"/"
             }
          },
          "IamPolicyLambda":{
             "Type":"AWS::IAM::Policy",
             "Properties":{
                "PolicyName":"dev-coolsoftware-lambda",
                "PolicyDocument":{
                   "Version":"2012-10-17",
                   "Statement":[
                      {
                         "Effect":"Allow",
                         "Action":[
                            "logs:CreateLogGroup",
                            "logs:CreateLogStream",
                            "logs:PutLogEvents"
                         ],
                         "Resource":"arn:aws:logs:us-west-2:*:*"
                      }
                   ]
                },
                "Roles":[
                   {
                      "Ref":"IamRoleLambda"
                   }
                ]
             }
          },
          "createAccount":{
             "Type":"AWS::Lambda::Function",
             "Properties":{
                "Code":{
                   "S3Bucket":{
                      "Ref":"ServerlessDeploymentBucket"
                   },
                   "S3Key":"coolsoftware-1472853507538.zip"
                },
                "FunctionName":"coolsoftware-dev-createAccount",
                "Handler":"handler.createAccount",
                "MemorySize":128,
                "Role":{
                   "Fn::GetAtt":[
                      "IamRoleLambda",
                      "Arn"
                   ]
                },
                "Runtime":"nodejs4.3",
                "Timeout":30
             }
          },
          "RestApiApigEvent":{
             "Type":"AWS::ApiGateway::RestApi",
             "Properties":{
                "Name":"dev-coolsoftware"
             }
          },
          "ResourceApigEventCreateaccountAccount":{
             "Type":"AWS::ApiGateway::Resource",
             "Properties":{
                "ParentId":{
                   "Fn::GetAtt":[
                      "RestApiApigEvent",
                      "RootResourceId"
                   ]
                },
                "PathPart":"account",
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                }
             }
          },
          "PutMethodApigEventCreateaccountAccount":{
             "Type":"AWS::ApiGateway::Method",
             "Properties":{
                "AuthorizationType":"AWS_IAM",
                "HttpMethod":"PUT",
                "MethodResponses":[
                   {
                      "ResponseModels":{
                         "application/json":"AccountCreationResponseModel"
                      },
                      "ResponseParameters":{
    
                      },
                      "StatusCode":"200"
                   }
                ],
                "RequestParameters":{
    
                },
                "Integration":{
                   "IntegrationHttpMethod":"POST",
                   "Type":"AWS",
                   "Uri":{
                      "Fn::Join":[
                         "",
                         [
                            "arn:aws:apigateway:",
                            {
                               "Ref":"AWS::Region"
                            },
                            ":lambda:path/2015-03-31/functions/",
                            {
                               "Fn::GetAtt":[
                                  "createAccount",
                                  "Arn"
                               ]
                            },
                            "/invocations"
                         ]
                      ]
                   },
                   "RequestTemplates":{
                      "application/json":"\n            #define( $loop )\n              {\n              #foreach($key in $map.keySet())\n                  \"$util.escapeJavaScript($key)\":\n                    \"$util.escapeJavaScript($map.get($key))\"\n                    #if( $foreach.hasNext ) , #end\n              #end\n              }\n            #end\n            {\n              \"body\": $input.json(\"$\"),\n              \"method\": \"$context.httpMethod\",\n              \"principalId\": \"$context.authorizer.principalId\",\n              \"stage\": \"$context.stage\",\n\n              #set( $map = $input.params().header )\n              \"headers\": $loop,\n\n              #set( $map = $input.params().querystring )\n              \"query\": $loop,\n\n              #set( $map = $input.params().path )\n              \"path\": $loop,\n\n              #set( $map = $context.identity )\n              \"identity\": $loop,\n\n              #set( $map = $stageVariables )\n              \"stageVariables\": $loop\n            }\n          "
                   },
                   "IntegrationResponses":[
                      {
                         "StatusCode":"200",
                         "ResponseParameters":{
    
                         },
                         "ResponseTemplates":{
                            "application/json":""
                         }
                      }
                   ]
                },
                "ResourceId":{
                   "Ref":"ResourceApigEventCreateaccountAccount"
                },
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                },
                "RequestModels":{
                   "application/json":"AccountCreationRequestModel"
                }
             }
          },
          "DeploymentApigEvent1472853508283":{
             "Type":"AWS::ApiGateway::Deployment",
             "Properties":{
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                },
                "StageName":"dev"
             },
             "DependsOn":[
                "PutMethodApigEventCreateaccountAccount"
             ]
          },
          "createAccountApigPermission":{
             "Type":"AWS::Lambda::Permission",
             "Properties":{
                "FunctionName":{
                   "Fn::GetAtt":[
                      "createAccount",
                      "Arn"
                   ]
                },
                "Action":"lambda:InvokeFunction",
                "Principal":"apigateway.amazonaws.com"
             }
          },
          "DynamoDBTableAccounts":{
             "Type":"AWS::DynamoDB::Table",
             "DeletionPolicy":"Retain",
             "Properties":{
                "TableName":"dev-coolsoftware-accounts",
                "ProvisionedThroughput":{
                   "ReadCapacityUnits":1,
                   "WriteCapacityUnits":1
                },
                "AttributeDefinitions":[
                   {
                      "AttributeName":"accountid",
                      "AttributeType":"S"
                   }
                ],
                "KeySchema":[
                   {
                      "AttributeName":"accountid",
                      "KeyType":"HASH"
                   }
                ]
             }
          },
          "AccountCreationRequestModel":{
             "Type":"AWS::ApiGateway::Model",
             "Properties":{
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                },
                "ContentType":"application/json",
                "Description":"Schema for AccountCreationRequestModel",
                "Name":"AccountCreationRequestModel",
                "Schema":{
                   "$schema":"http://json-schema.org/draft-04/schema#",
                   "title":"AccountCreationRequestModel",
                   "type":"object",
                   "properties":{
                      "publickey":{
                         "type":"string"
                      },
                      "deviceid":{
                         "type":"string"
                      }
                   }
                }
             }
          },
          "AccountCreationResponseModel":{
             "Type":"AWS::ApiGateway::Model",
             "Properties":{
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                },
                "ContentType":"application/json",
                "Description":"Schema for AccountCreationResponseModel",
                "Name":"AccountCreationResponseModel",
                "Schema":{
                   "$schema":"http://json-schema.org/draft-04/schema#",
                   "title":"AccountCreationResponseModel",
                   "type":"object",
                   "properties":{
                      "status":{
                         "type":"string"
                      },
                      "message":{
                         "type":"string"
                      }
                   }
                }
             }
          },
          "FailureResponseModel":{
             "Type":"AWS::ApiGateway::Model",
             "Properties":{
                "RestApiId":{
                   "Ref":"RestApiApigEvent"
                },
                "ContentType":"application/json",
                "Description":"Schema for FailureResponseModel",
                "Name":"FailureResponseModel",
                "Schema":{
                   "$schema":"http://json-schema.org/draft-04/schema#",
                   "title":"FailureResponseModel",
                   "type":"object",
                   "properties":{
                      "status":{
                         "type":"string"
                      },
                      "message":{
                         "type":"string"
                      }
                   }
                }
             }
          }
       },
       "Outputs":{
          "ServerlessDeploymentBucketName":{
             "Value":{
                "Ref":"ServerlessDeploymentBucket"
             }
          },
          "Function1Arn":{
             "Description":"Lambda function info",
             "Value":{
                "Fn::GetAtt":[
                   "createAccount",
                   "Arn"
                ]
             }
          },
          "ServiceEndpoint":{
             "Description":"URL of the service endpoint",
             "Value":{
                "Fn::Join":[
                   "",
                   [
                      "https://",
                      {
                         "Ref":"RestApiApigEvent"
                      },
                      ".execute-api.us-west-2.amazonaws.com/dev"
                   ]
                ]
             }
          }
       }
    }
    

    Edit 2: When I test the endpoint using API Gateway's test feature in the AWS Console everything works great :/

    Edit 3: Updated the cloudformation script again--still not working.