python multiprocessing.Process.terminate - How to kill child processes
EDIT: As @svalorzen pointed out in comments this doesn't really work since the reference to self.subprocess
is lost.
Finally came to a clean, acceptable solution. Since mp.Process.terminate is a method, we can override it.
class WorkerProcess(mp.Process):
def run(self):
# Simulate long running task
self.subprocess = subprocess.Popen(['python', '-c', 'import time; time.sleep(1000)'])
self.code = self.subprocess.wait()
# HERE
def terminate(self):
self.subprocess.terminate()
super(WorkerProcess, self).terminate()
redacted
Updated on July 28, 2022Comments
-
redacted over 1 year
This code:
import multiprocessing as mp from threading import Thread import subprocess import time class WorkerProcess(mp.Process): def run(self): # Simulate long running task self.subprocess = subprocess.Popen(['python', '-c', 'import time; time.sleep(1000)']) self.code = self.subprocess.wait() class ControlThread(Thread): def run(): jobs = [] for _ in range(2): job = WorkerProcess() jobs.append(job) job.start() # wait for a while and then kill jobs time.sleep(2) for job in jobs: job.terminate() if __name__ == "__main__": controller = ControlThread() controller.start()
When I terminate the spawned WorkerProcess instances. They die just fine, however the subprocesses
python -c 'import time; time.sleep(1000)
runs until completition. This is well documented in the official docs, but how do I kill the child processes of a killed process?A possbile soultion might be:
Wrap WorkerProcess.run() method inside try/except block catching SIGTERM, and terminating the
subprocess.call call
. But I am not sure how to catch the SIGTERM in the WorkerProcessI also tried setting
signal.signal(signal.SIGINT, handler)
in the WorkerProcess, but I am getting ValuError, because it is allowed to be set only in the main thread.
What do I do now?