Python: get output from a command line which exits with nonzero exit code

Question

I am Using Python 2.7.1 on a Windows Server 2008 R2 x64 box.

I'm trying to get the output of a command line process which gives a nonzero exit status after outputting the information I need.

I was initially using subprocess.check_output, and catching the CalledProcessError which occurs with nonzero exit status, but while the returncode was stored in the error, no output revealed this.

Running this against cases which give output but have an exit status of 0 works properly and I can get the output using subprocess.check_output.

My assumption was that the output was being written to STDOUT but the exception pulls its 'output' from STDERR. I've tried to re implement the functionality of check_output, but I still get nothing on the output when I believe I should be seeing output to STDOUT and STDERR. My current code is below (where 'command' is the full text, including parameters, of command I am running:

process = subprocess.Popen(command, stdout=subprocess.PIPE, 
stderr=subprocess.STDOUT, universal_newlines=True)
output = process.communicate()
retcode = process.poll()
if retcode:
    raise subprocess.CalledProcessError(retcode, image_check, output=output)
    return output 

This gives me the following in the variable output: [('', None)]

Is my subprocess.Popen code correct?

Solution 1

You code works fine. Turns out that the process that you are calling is probably outputing to CON. See the following example

import subprocess

def check_output(command):
    process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
    output = process.communicate()
    retcode = process.poll()
    if retcode:
            raise subprocess.CalledProcessError(retcode, command, output=output[0])
    return output 

command = "echo this>CON"

print "subprocess -> " + subprocess.check_output(command, shell=True)
print "native -> " + str(check_output(command))

try:
    subprocess.check_output("python output.py", shell=True)
except subprocess.CalledProcessError, e:
    print "subproces CalledProcessError.output = " + e.output

try:
    check_output("python output.py")
except subprocess.CalledProcessError, e:
    print "native CalledProcessError.output = " + e.output

Output

subprocess -> 
native -> ('', None)
stderr subproces CalledProcessError.output = stdout
native CalledProcessError.output = stderr stdout

Sadly, I do not know how to resolve the issue. Notice that subprocess.check_output results contains only the output from stdout. Your check_output replacement would output both stderr and stdout.

After inspecting subprocess.check_output, it does indeed generate a CalledProcessError with the output containing only stdout.

Solution 2

Have you tried stderr=subprocess.STDOUT as mentioned in the python doc page:

To also capture standard error in the result, use stderr=subprocess.STDOUT:

Here is a test code:

import subprocess

try:
    subprocess.check_output('>&2 echo "errrrr"; exit 1', shell=True)
except subprocess.CalledProcessError as e:
    print 'e.output: ', e.output


try:
    subprocess.check_output('>&2 echo "errrrr"; exit 1', shell=True, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
    print 'e.output: ', e.output

output:

errrrr
e.output:  
e.output:  errrrr

Solution 3

There is an issue here that might be hitting you- http://bugs.python.org/issue9905

Comments

[bplattenburg]

I'm pretty sure this is where it is going. I was able to get in touch with the vendor and found out abotu some additional undocumented parameters that strop the nonzero exit statuses from occurring, so my scripting is working now, although I would still like to know how to capture output from CON.