How to invoke external scripts/programs from node.js

Question

I have a C++ program and a Python script that I want to incorporate into my node.js web app.

I want to use them to parse the files that are uploaded to my site; it may take a few seconds to process, so I would avoid to block the app as well.

How can I just accept the file then just run the C++ program and script in a sub-process from a node.js controller?

Solution 1

see child_process. here is an example using spawn, which allows you to write to stdin and read from stderr/stdout as data is output. If you have no need to write to stdin and you can handle all output when the process completes, child_process.exec offers a slightly shorter syntax to execute a command.

// with express 3.x
var express = require('express'); 
var app = express();
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(app.router);
app.post('/upload', function(req, res){
   if(req.files.myUpload){
     var python = require('child_process').spawn(
     'python',
     // second argument is array of parameters, e.g.:
     ["/home/me/pythonScript.py"
     , req.files.myUpload.path
     , req.files.myUpload.type]
     );
     var output = "";
     python.stdout.on('data', function(data){ output += data });
     python.on('close', function(code){ 
       if (code !== 0) {  
           return res.send(500, code); 
       }
       return res.send(200, output);
     });
   } else { res.send(500, 'No file found') }
});

require('http').createServer(app).listen(3000, function(){
  console.log('Listening on 3000');
});

Solution 2

Might be a old question but some of these references will provide more details and different ways of including python in NodeJS.

There are multiple ways of doing this.

• first way is by doing npm install python-shell

and here's the code

var PythonShell = require('python-shell');
//you can use error handling to see if there are any errors
PythonShell.run('my_script.py', options, function (err, results) { 
//your code

you can send a message to python shell using pyshell.send('hello');

you can find the API reference here- https://github.com/extrabacon/python-shell

• second way - another package you can refer to is node python , you have to do npm install node-python

• third way - you can refer to this question where you can find an example of using a child process- How to invoke external scripts/programs from node.js

a few more references - https://www.npmjs.com/package/python

if you want to use service-oriented architecture - http://ianhinsdale.com/code/2013/12/08/communicating-between-nodejs-and-python/

Comments

[Steve]

Is your C++ program being called from within Python or are these two separate calls you will be making? 1) To Python and 2) to C++ app? Or does Python code load up and call your C++ app?

[Joe]

Node's child_process stuff will run processes async. If all you want to do is launch a program from within node, that will do it.

[alh]

@StevenLeggett The python script and C++ app do not interact at all; I want to call them both separately (they don't need to be in any particular order either).

[Plato]

if you can execute the c++ app from the command line then my solution should work... you may want to look at the async library to help with control flow e.g. calling python then c++ in series

[Spencer]

How are types handled with this format? If I send 0.123 as a parameter, how will python treat it since I think it is coming in as a string?

[Plato]

@spencer AFAIK URL components, as well as unix/windows shell commands, always use strings for everything so I consider it the python app's responsibility to cast it to a number. how would you tell python it's a number in a normal command line invocation?

[Spencer]

you're right and that makes a lot of sense. I should have thought about it some more haha. Thanks!

[Plato]

@spencer One thing that might help is to have the node app create a JSON object with your data, that will distinguish number vs. string. Then send it to the python app's stdin and parse it

[PirateApp]

this works if I can spawn a python process from node, I already have a node server running and a python script that runs in daemon thread mode, I want to access a variable in python from node, how would ya do that