Python popen command. Wait until the command is finished

Question

I have a script where I launch with popen a shell command. The problem is that the script doesn't wait until that popen command is finished and go continues right away.

om_points = os.popen(command, "w")
.....

How can I tell to my Python script to wait until the shell command has finished?

Solution 1

Depending on how you want to work your script you have two options. If you want the commands to block and not do anything while it is executing, you can just use subprocess.call.

#start and block until done
subprocess.call([data["om_points"], ">", diz['d']+"/points.xml"])

If you want to do things while it is executing or feed things into stdin, you can use communicate after the popen call.

#start and process things, then wait
p = subprocess.Popen([data["om_points"], ">", diz['d']+"/points.xml"])
print "Happens while running"
p.communicate() #now wait plus that you can send commands to process

As stated in the documentation, wait can deadlock, so communicate is advisable.

Solution 2

You can you use subprocess to achieve this.

import subprocess

#This command could have multiple commands separated by a new line \n
some_command = "export PATH=$PATH://server.sample.mo/app/bin \n customupload abc.txt"

p = subprocess.Popen(some_command, stdout=subprocess.PIPE, shell=True)

(output, err) = p.communicate()  

#This makes the wait possible
p_status = p.wait()

#This will give you the output of the command being executed
print "Command output: " + output

Solution 3

Force popen to not continue until all output is read by doing:

os.popen(command).read()

Solution 4

Let the command you are trying to pass be

os.system('x')

then you covert it to a statement

t = os.system('x')

now the python will be waiting for the output from the commandline so that it could be assigned to the variable t.

Solution 5

What you are looking for is the wait method.

View more solutions

Comments

[michele]

But if I type: om_points = os.popen(data["om_points"]+" > "+diz['d']+"/points.xml", "w").wait() I receive this error: Traceback (most recent call last): File "./model_job.py", line 77, in <module> om_points = os.popen(data["om_points"]+" > "+diz['d']+"/points.xml", "w").wait() AttributeError: 'file' object has no attribute 'wait' What is the problem? Thanks again.

[thornomad]

Check out the docs on subprocess.call

[Olivier Verdier]

You did not click on the link I provided. wait is a method of the subprocess class.

[ansgri]

wait can deadlock if the process writes to stdout and nobody reads it

[Chang]

While subprocess is preferred in many answers, it cannot handle space and quota within command very well. The above answer does not directly solve the os.popen question.

[jdmcnair]

I'm no python expert, but this seems to be the simplest solution that makes the fewest modifications to the original code. Any reason why this wouldn't be a good solution?

[MonsieurBeilto]

subprocess can be up to 2x slower than os system - bugs.python.org/issue37790

[MonsieurBeilto]

subprocess can be up to 2x slower than os system - bugs.python.org/issue37790

[MonsieurBeilto]

subprocess can be up to 2x slower than os system - bugs.python.org/issue37790

[MonsieurBeilto]

subprocess can be up to 2x slower than os system - bugs.python.org/issue37790

[LoMaPh]

subprocess.run() was added in Python 3.5 and is "The recommended approach to invoking subprocesses"

[Marc Maxmeister]

@jdmcnair my guess is that if the command threw an error, this .read() method would break the parent process. But some other methods decouple errors in the child process from affecting the parent process. I'm guessing that if the child process hangs forever, .read() will wait forever to a result.