AWS Lambda to run in background even after sending response to API Gateway

Question

I have searched all through the net but didn't find a solution of how to make this functionality get succeeded. Require help.

My requirement is: I want a that if I trigger an aws lambda function written in node.js and uses an aws-serverless-express module must send back response quickly to API gateway but still should not exit and still run in the backend and we could see cloud watch logs. It must be asynchronous.

The code snippet is:

    app.get('/check', function(req, res){
     method.invoke(req)
     res.status(200).send('success')
   })

I did and checked like this but the lambda function gets stopped and returns the response to api gateway it didn't even runs the method.invoke() function in backend.

Please correct me if anything I am understanding or doing wrong. I checked with this link: Invoke AWS Lambda and return response to API Gateway asyncronously

Is it the only way to do this problem. Creating two lambda functions.

Solution 1

You can achieve this by using AWS Lambda Step functions, connected to API Gateway, having parallel execution of branches with two lambda functions, where one returns a response to API Gateway and other executes asynchronously.

Solution 2

Besides Step Functions, you could just invoke another Lambda function using the SDK built-in to the Lambda environment.

I'm no expert in express or NodeJS but I would also think there should be a way to send the HTTP response back and still continue code execution.

Solution 3

Can't find a link to the documentation of AWS, but normally it is not possible to continue processing after the Lambda function has returned the response. That's just not how the available runtimes (for the different programming languages) are constructed.

Next to invoking separate asynchronous processes (e.g., other Lambda function requests, or putting work on a queue) or using AWS Step functions as mentioned here, there's a third method that I know that works: supply a special custom runtime for the AWS Lambda functions that addresses this need.

Next to the standard runtimes, you can create and specify a custom runtime to be used for your AWS Lambda functions. In the standard runtimes, the response of your handler is being posted to the Lambda execution context, after which no activities are possible in your handler because the handler is being terminated (or at least: paused).

So, the trick to make additional processing possible after sending the response is to move the responsibility of posting the response to the Lambda operating content from the bootstrap script to the Lambda function handler itself... and continue to do your processing in your Lambda function handler after already having sent the response. Using your custom runtime, processing in the Lambda functions will then not be terminated after having sent the response, since it's not how your custom runtime is constructed.

It's not the architecturally-best solution, as it messes with the responsibilities between the Lambda operating context and your Lambda functions handlers... but it makes it possible to do processing in your Lambda function handlers after having sent the response.

Solution 4

Step function seems the best solution here. See @Ashan's reply. Apart from that you can use the new invoke method in lambda nodejs sdk. Note that invokeAsync is now deprecated. You can set InvocationType to Event See the example below. which is taken from here

var params = {
  ClientContext: "MyApp", 
  FunctionName: "MyFunction", 
  InvocationType: "Event", 
  LogType: "Tail", 
  Payload: <Binary String>, 
  Qualifier: "1"
 };
 lambda.invoke(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);           // successful response
   /*
   data = {
    FunctionError: "", 
    LogResult: "", 
    Payload: <Binary String>, 
    StatusCode: 123
   }
   */
 }); 

one example use case is, first function would return an immediate response, and it would trigger another lambda function which would execute the tasks and eventually may be call a webhook.

View more solutions