Python subprocess.Popen.wait() returns 0 even though an error occured

Question

I am running a command line utility via Python's subprocess module. I create a subprocess.Popen() object with the command line arguments and stdout=subprocess.PIPE and then I use subprocess.wait() to wait until the task is complete and to return the return code which should indicate whether the task completed successfully.

translate = subprocess.Popen(['gdal_translate', '-of', 'HFA', 'inDataset', 'outDataset'], stdout=subprocess.PIPE)
if translate.wait() == 0: print "Success"
else: print "Fail - %s" % translate.stdout.read()

If I run my script in the windows command prompt I can see the messages provided by gdal_translate and they say there were errors that cause a process to fail, but my script still prints "Success".

How can I check for errors reported by the command line utility if the return code is always 0?

Solution

It is possible that you get errors being outputted and still have a return code of zero. In your code you only capture standard out and not standard error. In the run function below it will run a command, wait for execution to end, then read the returncode standard error and standard output. If there is anything on standard error or if returncode is not zero then it will treat that as a failure. You can see the four example calls in the code. The first one is a normal successful call, the second has return code 0 but has output on error and standard output. The third has a non-zero return code and error output, while the last example has a none-zero return code with no output at all.

code

from subprocess import Popen, PIPE

def run(cmd):
    print '-'*40
    print 'running:', cmd
    p = Popen(cmd, stderr=PIPE, stdout=PIPE, shell=True)
    output, errors = p.communicate()
    print [p.returncode, errors, output]
    if p.returncode or errors:
        print 'something went wrong...'

run("echo all is well")
run("echo out;echo error 1>&2")
run("this-will-fail")
run("exit 1")

output

----------------------------------------
running: echo all is well
[0, '', 'all is well\n']
----------------------------------------
running: echo out;echo error 1>&2
[0, 'error\n', 'out\n']
something went wrong...
----------------------------------------
running: this-will-fail
[127, '/bin/sh: this-will-fail: not found\n', '']
something went wrong...
----------------------------------------
running: exit 1
[1, '', '']
something went wrong...