CasperJS passing data back to PHP

Question

CasperJS is being called by PHP using an exec() command. After CasperJS does its work such as retrieving parts of a webpage, how can the retrieved data be returned back to PHP?

Solution 1

You can redirect output from stdout to an array.

On this page it says you can do:

string exec ( string $command [, array &$output [, int &$return_var ]] )

It goes on to say:

If the output argument is present, then the specified array will be filled with every line of output from the command.

So basically you can do exec('casperjs command here, $array_here);

Solution 2

I think the best way to transfer data from CasperJS to another language such as PHP is running CasperJS script as a service. Because CasperJS has been written over PhantomJS, CasperJS can use an embedded web server module of PhantomJS called Mongoose.

For information about how works the embedded web server see here

Here an example about how a CasperJS script can start a web server.

//define ip and port to web service
var ip_server = '127.0.0.1:8585';

//includes web server modules
var server = require('webserver').create();

//start web server
var service = server.listen(ip_server, function(request, response) {

    var links = [];
    var casper = require('casper').create();

    function getLinks() {
        var links = document.querySelectorAll('h3.r a');
        return Array.prototype.map.call(links, function(e) {
            return e.getAttribute('href')
        });
    }

    casper.start('http://google.fr/', function() {
        // search for 'casperjs' from google form
        this.fill('form[action="/search"]', { q: 'casperjs' }, true);
    });

    casper.then(function() {
        // aggregate results for the 'casperjs' search
        links = this.evaluate(getLinks);
        // now search for 'phantomjs' by filling the form again
        this.fill('form[action="/search"]', { q: 'phantomjs' }, true);
    });

    casper.then(function() {
        // aggregate results for the 'phantomjs' search
        links = links.concat(this.evaluate(getLinks));
    });

    //
    casper.run(function() {
            response.statusCode = 200;
            //sends results as JSON object
            response.write(JSON.stringify(links, null, null));
            response.close();              
    });

});
console.log('Server running at http://' + ip_server+'/');

Comments

[danmux]

Whilst I agree that this is a good technique in theory, in practice casper does not free up phantomjs resources, there appears to be linearly increasing memory use with each request. I have tried adding casper.clear() and even hacking clear() to call this.page.close() (from within clear()) page. Whilst calling close appears to help it doesn't solve the problem.