Running two executable in parallel with os.system() in Python?

Question

I wish to run two executable a.exe and b.exe in parallel, invoked one after another.

When I tried,

os.system('a.exe')
#some code
os.system('b.exe')

b.exe is getting started only after I killed a.exe? Why does it happen? How can I run both simultaneously? (Do I need to do multithreading?) Note: I'm in Windows platform

Solution 1

If we ignore exceptions then it is simple to run several programs concurrently:

#!/usr/bin/env python
import subprocess

# start all programs
processes = [subprocess.Popen(program) for program in ['a', 'b']]
# wait
for process in processes:
    process.wait()

See Python threading multiple bash subprocesses?

If you want to stop previously started processes if any of the programs fails to start:

#!/usr/bin/env python3
from contextlib import ExitStack
from subprocess import Popen


def kill(process):
    if process.poll() is None:  # still running
        process.kill()

with ExitStack() as stack:  # to clean up properly in case of exceptions
    processes = []
    for program in ['a', 'b']:
        processes.append(stack.enter_context(Popen(program)))  # start program
        stack.callback(kill, processes[-1])
    for process in processes:
        process.wait()

Solution 2

Try running each one as a separate thread:

import thread

thread.start_new_thread(os.system, ('a.exe',))
thread.start_new_thread(os.system, ('b.exe',))

Solution 3

You probably want to try subprocess.Popen, this allows a process to execute but doesn't block. However you have to think about zombie processes in this instance.

Solution 4

An old question and I'm new to python myself but if you are trying to invoke in parallel with multiple/different executables, I've found 'Popen' to be suitable.

from subprocess import Popen

Popen('a.exe', shell=False)
Popen('b.exe', shell=False)

I have found it to be more useful in my use cases than 'Threading' (@Lalo Ramírez example above) as Popen (although tricky at first, particularly the shell argument) seems easier to manage, check on and terminate a process once started (by interacting with p1 or p2 as in the example below). It may also be useful to use an alias for readability.

from subprocess import Popen as new
from time import sleep

p1 = new('a.exe', shell=False)
p2 = new('b.exe', shell=False)

sleep(20)

p1.terminate()
p1.wait()

Interestingly 'Threading' is emulating the threading functionality of Java which may be more appropriate for those with experience in multithreading in Java. The 'Popen' approach seems the simpler option to me.

Solution 5

You might use a specific way to run two or more commands or programms, such as the threading library of Python. Here you have a wide example about how it works.

import threading
import time

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print "Starting " + self.name
        print_time(self.name, self.counter, 5)
        print "Exiting " + self.name

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print "%s: %s" % (threadName, time.ctime(time.time()))
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

print "Exiting Main Thread"

Then, your code could be something like this:

import threading


class myThread (threading.Thread):
    def __init__(self, command):
        threading.Thread.__init__(self)
        self.cmd = command

    def run(self):
        print "Starting " + self.cmd
        os.system(self.cmd)
        print "Exiting " + self.cmd

lstCmd=["a.exe","b.exe","ping 192.168.0.10","some command"]

# Create new threads
thread1 = myThread(lstCmd[0])
thread2 = myThread(lstCmd[1])
thread3 = myThread(lstCmd[2])
thread4 = myThread(lstCmd[3])

# Start new Threads
thread1.start()
thread2.start()
thread3.start()
thread4.start()

View more solutions

Comments

[robm]

This is the best solution I've found, for its simplicity. Other solutions show good ways to spawn n processes, this is good for spawning one asynchronous process without blocking the rest of the program.

[partida]

COOL. I use Popen(cmd,shell=True)