How do I run PhantomJS on AWS Lambda with NodeJS

Question

After not finding a working answer anywhere else on the internet, I am submitting this ask-and-answer-myself tutorial

How can I get a simple PhantomJS process running from a NodeJS script on AWS Lambda? My code works fine on my local machine, but I run into different problems trying to run it on Lambda.

Solution 1

EDIT: This no longer works. This is an apparent solution.

---

Here is a complete code sample of a simple PhantomJS process, which is launched as a NodeJS child_process. It is also available on github.

---

index.js

var childProcess = require('child_process');
var path = require('path');

exports.handler = function(event, context) {

    // Set the path as described here: https://aws.amazon.com/blogs/compute/running-executables-in-aws-lambda/
    process.env['PATH'] = process.env['PATH'] + ':' + process.env['LAMBDA_TASK_ROOT'];

    // Set the path to the phantomjs binary
    var phantomPath = path.join(__dirname, 'phantomjs_linux-x86_64');

    // Arguments for the phantom script
    var processArgs = [
        path.join(__dirname, 'phantom-script.js'),
       'my arg'
    ];

    // Launc the child process
    childProcess.execFile(phantomPath, processArgs, function(error, stdout, stderr) {
        if (error) {
            context.fail(error);
            return;
        }
        if (stderr) {
            context.fail(error);
            return;
        }
        context.succeed(stdout);
    });
}

---

phantom-script.js

var system = require('system');
var args = system.args;

// Example of how to get arguments passed from node script
// args[0] would be this file's name: phantom-script.js
var unusedArg = args[1];

// Send some info node's childProcess' stdout
system.stdout.write('hello from phantom!')

phantom.exit();

---

To get a PhantomJS binary that works with Amazon's Linux machine's, go to the PhantomJS Bitbucket Page and download phantomjs-1.9.8-linux-x86_64.tar.bz2.

Solution 2

A generic solution is to use an actual AWS Linux machine to install the npm modules and transfer them to your lambda executable. Here are the steps:

1. spin up an EC2 instance
2. ssh into EC2
3. install Node + npm
4. install the required npm modules
5. zip them up
6. fetch them to your local machine with scp
7. unzip and copy to the npm_modules folder of your lambda function (don't npm install locally!)
8. upload your code to Lambda

Here is a tutorial with links to further resources: Compile node module libraries for AWS Lambda

This also works in such cases when PhantomJS is a dependency of another node module, eg. node-webshot and you have less influence on what is being installed.

Comments

[Dave Maple]

Fantastic answer @Tyler.

[Vikas]

@Tyler In the same I implemented my code. Everything is going fine except that it is showing me process exited before completion. Can i know what is causing this issue??

[geotheory]

Thank you. Much more helpful than the accepted answer.

[Rana Ghosh]

This doesn't work anymore. Here's a more up to date answer: stackoverflow.com/a/56843029/61624