reading a packaged file in aws lambda package

Question

I have a very simple node lambda function which reads the contents of packaged file in it. I upload the code as zip file. The directory structure is as follows.

index.js
readme.txt

Then have in my index.js file:

fs.readFile('/var/task/readme.txt', function (err, data) {
if (err) throw err;
});

I keep getting the following error NOENT: no such file or directory, open '/var/task/readme.txt'.

I tried ./readme.txt also.

What am I missing ?

Solution 1

Try this, it works for me:

'use strict'

let fs = require("fs");
let path = require("path");

exports.handler = (event, context, callback) => {
        // To debug your problem
        console.log(path.resolve("./readme.txt"));

        // Solution is to use absolute path using `__dirname`
        fs.readFile(__dirname +'/readme.txt', function (err, data) {
            if (err) throw err;
        });
};

to debug why your code is not working, add below link in your handler

console.log(path.resolve("./readme.txt"));

On AWS Lambda node process might be running from some other folder and it looks for readme.txt file from that folder as you have provided relative path, solution is to use absolute path.

Solution 2

What worked for me was the comment by Vadorrequest to use process.env.LAMBDA_TASK_ROOT. I wrote a function to get a template file in a /templates directory when I'm running it locally on my machine with __dirname or with the process.env.LAMBDA_TASK_ROOT variable when running on Lambda:

function loadTemplateFile(templateName) {
  const fileName = `./templates/${templateName}`
  let resolved
  if (process.env.LAMBDA_TASK_ROOT) {
    resolved = path.resolve(process.env.LAMBDA_TASK_ROOT, fileName)
  } else {
    resolved = path.resolve(__dirname, fileName)
  }
  console.log(`Loading template at: ${resolved}`)
  try {
    const data = fs.readFileSync(resolved, 'utf8')
    return data
  } catch (error) {
    const message = `Could not load template at: ${resolved}, error: ${JSON.stringify(error, null, 2)}`
    console.error(message)
    throw new Error(message)
  }
}

Solution 3

This is an oldish question but comes up first when attempting to sort out whats going on with file paths on Lambda.

Additional Steps for Serverless Framework

For anyone using Serverless framework to deploy (which probably uses webpack to build) you will also need to add the following to your webpack config file (just after target: node):

  // assume target: 'node', is here

  node: {
    __dirname: false,
  },

Without this piece using __dirname with Serverless will STILL not get you the desired absolute directory path.

Solution 4

I went through this using serverless framework and it really was the file that was not sent in the compression. Just add the following line in serverless.yml:

package:
  individually: false
  include:
    - src/**

View more solutions

Comments

[commandt]

It sounds like the text file might not be getting included in your zip, so I would first double check that by downloading the zip file from the console. Otherwise, would you mind adding a bit more context? What does the rest of your handler function look like?

[rOrlig]

actually i'm using serverless framework to package and upload the zip file to S3. I checked the S3 File and it does contain the file. What I am trying to do is read kms encrypted data keys so that my lambda can get encryption keys from kms to encrypt outbound http calls to 3rd party.

[commandt]

Does the S3 file contain the txt file at the location you expect? Also, using a relative path like './readme.txt' might not work if serverless changes the cwd of your function. Maybe try __dirname + '/readme.txt' or you could try logging the cwd of your lambda function with process.cwd()

[Vadorequest]

Using __dirname when using Webpack tends to fail and resolve wrong folder. In such case, prefer to use process.env.LAMBDA_TASK_ROOT and rebuild the real path based on this env variable.

[Wibble]

This would be a better answer if you include context for the two code lines you'e provided. Where should they go? How should fileData2 be used? Why does this work?

[NARGIS PARWEEN]

@jaym If I am using the same process it is throwing me the error that the file not found. What is the base path to get the file in serverless Lambda?

[zya]

If you are using webpack to build, look no further. this is your answer.