subprocess.Popen() IO redirect

Question

Trying to redirect a subprocess' output to a file.

server.py:

while 1:
    print "Count " + str(count)
    sys.stdout.flush()
    count = count + 1
    time.sleep(1)

Laucher:

cmd = './server.py >temp.txt'
args = shlex.split(cmd)
server = subprocess.Popen( args )

The output appear on screen, temp.txt remains empty. What am I doing wrong?

As background I am trying to capture the output of a program that has already been written.

I cannot use:

server = subprocess.Popen(
                [exe_name],
                stdin=subprocess.PIPE, stdout=subprocess.PIPE)

as the program may not flush. Instead I was going to redirect output through a fifo. This works fine if I manually launch server.py but obviously not if I Popen() cause redirect doesnt work. ps -aux shows that server.py was launched correctly.

Solution 1

Altenatively, you can use the stdout parameter with a file object:

with open('temp.txt', 'w') as output:
    server = subprocess.Popen('./server.py', stdout=output)
    server.communicate()

As explained in the documentation:

stdin, stdout and stderr specify the executed program’s standard input, standard output and standard error file handles, respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and None.

Solution 2

Output redirection with ">" is a feature of shells - by default, subprocess.Popen doesn't instantiate one. This should work:

server = subprocess.Popen(args, shell=True)

Comments

[wim]

Using shell=True is not the preferred way, unless it's necessary (and here it is not)

[Pete Roberts]

That works. Communicate would block reader until the server terminates but now that I can redirect I can send server output to a fifo.

[jfs]

no need to call .communicate() here. subprocess.check_call('command', stdout=file) works.