How can I list all my Amazon EC2 instances using Node.js in AWS Lambda?

Question

I'm on AWS and using AWS SDK for JavaScript in Node.js. I'm trying to build an AWS Lambda function and inside I want to get a list of all my Amazon EC2 instances, but I just can't seem to get it working. Can anyone spot what I'm doing wrong?

Here is my Lambda function code:

var AWS = require('aws-sdk');
AWS.config.region = 'us-west-1';

exports.handler = function(event, context) {
    console.log("\n\nLoading handler\n\n");
    var ec2 = new AWS.EC2();
    ec2.describeInstances( function(err, data) {
        console.log("\nIn describe instances:\n");
      if (err) console.log(err, err.stack); // an error occurred
      else     console.log("\n\n" + data + "\n\n"); // successful response
    });
    context.done(null, 'Function Finished!');  
};

And this is my policy (I think it's correct?)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:*"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
    "Effect": "Allow",
    "Action": [
      "ec2:*"
    ],
    "Resource": "arn:aws:ec2:*"
  },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    }
  ]
}

And if I do a console.log on 'ec2' I get:

{ config: 
   { credentials: 
      { expired: false,
        expireTime: null,
        accessKeyId: 'XXXXXXXXXXXXXXXXXX',
        sessionToken: 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
        envPrefix: 'AWS' },
     credentialProvider: { providers: [Object] },
     region: 'us-west-1',
     logger: null,
     apiVersions: {},
     apiVersion: null,
     endpoint: 'ec2.us-west-1.amazonaws.com',
     httpOptions: { timeout: 120000 },
     maxRetries: undefined,
     maxRedirects: 10,
     paramValidation: true,
     sslEnabled: true,
     s3ForcePathStyle: false,
     s3BucketEndpoint: false,
     computeChecksums: true,
     convertResponseTypes: true,
     dynamoDbCrc32: true,
     systemClockOffset: 0,
     signatureVersion: 'v4' },
  isGlobalEndpoint: false,
  endpoint: 
   { protocol: 'https:',
     host: 'ec2.us-west-1.amazonaws.com',
     port: 443,
     hostname: 'ec2.us-west-1.amazonaws.com',
     pathname: '/',
     path: '/',
     href: 'https://ec2.us-west-1.amazonaws.com/' } }

Solution 1

The most likely cause is that you are explicitly terminating your Lambda function before it completes the call to EC2 DescribeInstances API.

The reason is that Lambda assumes your code has finished executing as soon as you call context.done(...). And this is happening before the console.log(... data ...) call.

This weird ordering happens because of how NodeJS works and how the AWS SDK for JavaScript works. In NodeJS you should never block the execution. A call to a webservice (such as EC2) would block the execution. Therefore the AWS SDK for JavaScript (as well as most NodeJS libraries) works by making an asynchronous call.

Most often, when you have an asynchronous call, you pass a callback function to that call. When the results are ready, NodeJS will execute the callback function.

In your code, that function(err, data) {...} is the callback function. This is not executed immediately, but will be scheduled for execution when NodeJS sees that the ec2.describeInstances call has received the its results.

As soon as you schedule the execution of your call back, you are calling context.done(...), which tells Lambda: I'm done, you can kill me. And it's happily obeying and interrupting your function, before the EC2 DescribeInstances call receives its data and passes it to your callback function.

How to solve the problem?

The answer should be clear by now: just move your context.done(...) call to inside your callback function, right after the if/else block containing the console.log(...data...) call:

ec2.describeInstances( function(err, data) {
  console.log("\nIn describe instances:\n");
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log("\n\n" + data + "\n\n"); // successful response
  context.done(null, 'Function Finished!');  
});

Solution 2

As of 2019 (October), the given answer didn't help, after digging I found now it is promise based

exports.handler = async function(event) {
  const promise = new Promise(function(resolve, reject) {
      //your logic, goes here at the end just call resolve 
      resolve("data you want to return"); // here lamda exits 
    })
  return promise;// lamda does not  exits here, it waits for to resolve
}

Comments

[Naveen Vijay]

Thank you @Bruno. How do you recommend to accomplishing the same inside a loop - like creating 10 SQS queues, inserting 20 messages [ batch api exists, besides that ], if I put context.done() inside the loop, doesn't that mean the iteration would stop after single execution ( 1 then a partial execution )