Executing an R script in python via subprocess.Popen

Question

When I execute the script in R, it is:

$ R --vanilla --args test_matrix.csv < hierarchical_clustering.R > out.txt

In Python, it works if I use:

process = subprocess.call("R --vanilla --args "+output_filename+"_DM_Instances_R.csv < /home/kevin/AV-labels/Results/R/hierarchical_clustering.R > "+output_filename+"_out.txt", shell=True)

But this method doesn't provide the process.wait() function.

So, I would like to use the subprocess.Popen, I tried:

process = subprocess.Popen(['R', '--vanilla', '--args', "\'"+output_filename+"_DM_Instances_R.csv\'",  '<', '/home/kevin/AV-labels/Results/R/hierarchical_clustering.R'])

But it didn't work, Python just opened R but didn't execute my script.

Solution 1

Instead of 'R', give it the path to Rscript. I had the same problem. Opens up R but doesn't execute my script. You need to call Rscript (instead of R) to actually execute the script.

retcode = subprocess.call("/Pathto/Rscript --vanilla /Pathto/test.R", shell=True)

This works for me.

Cheers!

Solution 2

A couple of ideas:

1. You might want to consider using the Rscript frontend, which makes running scripts easier; you can pass the script filename directly as a parameter, and do not need to read the script in through standard input.
2. You don't need the shell for just redirecting standard output to a file, you can do that directly with subprocess.Popen.

Example:

import subprocess

output_name = 'something'
script_filename = 'hierarchical_clustering.R'
param_filename = '%s_DM_Instances_R.csv' % output_name
result_filename = '%s_out.txt' % output_name
with open(result_filename, 'wb') as result:
    process = subprocess.Popen(['Rscript', script_filename, param_filename],
                               stdout=result);
    process.wait()

Solution 3

I've solved this problem by putting everything into the brackets..

process = subprocess.Popen(["R --vanilla --args "+output_filename+"_DM_Instances_R.csv < /home/kevin/AV-labels/Results/R/hierarchical_clustering.R > "+output_filename+"_out.txt"], shell=True)
process.wait()