Sending ^C to Python subprocess objects on Windows

Question

I have a test harness (written in Python) that needs to shut down the program under test (written in C) by sending it ^C. On Unix,

proc.send_signal(signal.SIGINT)

works perfectly. On Windows, that throws an error ("signal 2 is not supported" or something like that). I am using Python 2.7 for Windows, so I have the impression that I should be able to do instead

proc.send_signal(signal.CTRL_C_EVENT)

but this doesn't do anything at all. What do I have to do? This is the code that creates the subprocess:

# Windows needs an extra argument passed to subprocess.Popen,
# but the constant isn't defined on Unix.
try: kwargs['creationflags'] = subprocess.CREATE_NEW_PROCESS_GROUP
except AttributeError: pass
proc = subprocess.Popen(argv,
                        stdin=open(os.path.devnull, "r"),
                        stdout=subprocess.PIPE,
                        stderr=subprocess.PIPE,
                        **kwargs)

Solution 1

There is a solution by using a wrapper (as described in the link Vinay provided) which is started in a new console window with the Windows start command.

Code of the wrapper:

#wrapper.py
import subprocess, time, signal, sys, os

def signal_handler(signal, frame):
  time.sleep(1)
  print 'Ctrl+C received in wrapper.py'

signal.signal(signal.SIGINT, signal_handler)
print "wrapper.py started"
subprocess.Popen("python demo.py")
time.sleep(3) #Replace with your IPC code here, which waits on a fire CTRL-C request
os.kill(signal.CTRL_C_EVENT, 0)

Code of the program catching CTRL-C:

#demo.py

import signal, sys, time

def signal_handler(signal, frame):
  print 'Ctrl+C received in demo.py'
  time.sleep(1)
  sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)
print 'demo.py started'
#signal.pause() # does not work under Windows
while(True):
  time.sleep(1)

Launch the wrapper like e.g.:

PythonPrompt> import subprocess
PythonPrompt> subprocess.Popen("start python wrapper.py", shell=True)

You need to add some IPC code which allows you to control the wrapper firing the os.kill(signal.CTRL_C_EVENT, 0) command. I used sockets for this purpose in my application.

Explanation:

Preinformation

• send_signal(CTRL_C_EVENT) does not work because CTRL_C_EVENT is only for os.kill. [REF1]
• os.kill(CTRL_C_EVENT) sends the signal to all processes running in the current cmd window [REF2]
• Popen(..., creationflags=CREATE_NEW_PROCESS_GROUP) does not work because CTRL_C_EVENT is ignored for process groups. [REF2] This is a bug in the python documentation [REF3]

Implemented solution

1. Let your program run in a different cmd window with the Windows shell command start.
2. Add a CTRL-C request wrapper between your control application and the application which should get the CTRL-C signal. The wrapper will run in the same cmd window as the application which should get the CTRL-C signal.
3. The wrapper will shutdown itself and the program which should get the CTRL-C signal by sending all processes in the cmd window the CTRL_C_EVENT.
4. The control program should be able to request the wrapper to send the CTRL-C signal. This might be implemnted trough IPC means, e.g. sockets.

Helpful posts were:

I had to remove the http in front of the links because I'm a new user and are not allowed to post more than two links.

• http://social.msdn.microsoft.com/Forums/en-US/windowsgeneraldevelopmentissues/thread/dc9586ab-1ee8-41aa-a775-cf4828ac1239/#6589714f-12a7-447e-b214-27372f31ca11
• Can I send a ctrl-C (SIGINT) to an application on Windows?
• Sending SIGINT to a subprocess of python
• http://bugs.python.org/issue9524
• http://ss64.com/nt/start.html
• http://objectmix.com/python/387639-sending-cntrl-c.html#post1443948

Update: IPC based CTRL-C Wrapper

Here you can find a selfwritten python module providing a CTRL-C wrapping including a socket based IPC. The syntax is quite similiar to the subprocess module.

Usage:

>>> import winctrlc
>>> p1 = winctrlc.Popen("python demo.py")
>>> p2 = winctrlc.Popen("python demo.py")
>>> p3 = winctrlc.Popen("python demo.py")
>>> p2.send_ctrl_c()
>>> p1.send_ctrl_c()
>>> p3.send_ctrl_c()

Code

import socket
import subprocess
import time
import random
import signal, os, sys


class Popen:
  _port = random.randint(10000, 50000)
  _connection = ''

  def _start_ctrl_c_wrapper(self, cmd):
    cmd_str = "start \"\" python winctrlc.py "+"\""+cmd+"\""+" "+str(self._port)
    subprocess.Popen(cmd_str, shell=True)

  def _create_connection(self):
    self._connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    self._connection.connect(('localhost', self._port))

  def send_ctrl_c(self):
    self._connection.send(Wrapper.TERMINATION_REQ)
    self._connection.close()

  def __init__(self, cmd):
    self._start_ctrl_c_wrapper(cmd)
    self._create_connection()


class Wrapper:
  TERMINATION_REQ = "Terminate with CTRL-C"

  def _create_connection(self, port):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind(('localhost', port))
    s.listen(1)
    conn, addr = s.accept()
    return conn

  def _wait_on_ctrl_c_request(self, conn):
    while True:
      data = conn.recv(1024)
      if data == self.TERMINATION_REQ:
        ctrl_c_received = True
        break
      else:
        ctrl_c_received = False
    return ctrl_c_received

  def _cleanup_and_fire_ctrl_c(self, conn):
    conn.close()
    os.kill(signal.CTRL_C_EVENT, 0)

  def _signal_handler(self, signal, frame):
    time.sleep(1)
    sys.exit(0)

  def __init__(self, cmd, port):
    signal.signal(signal.SIGINT, self._signal_handler)
    subprocess.Popen(cmd)
    conn = self._create_connection(port)
    ctrl_c_req_received = self._wait_on_ctrl_c_request(conn)
    if ctrl_c_req_received:
      self._cleanup_and_fire_ctrl_c(conn)
    else:
      sys.exit(0)


if __name__ == "__main__":
  command_string = sys.argv[1]
  port_no = int(sys.argv[2])
  Wrapper(command_string, port_no)

Solution 2

My solution also involves a wrapper script, but it does not need IPC, so it is far simpler to use.

The wrapper script first detaches itself from any existing console, then attach to the target console, then files the Ctrl-C event.

import ctypes
import sys

kernel = ctypes.windll.kernel32

pid = int(sys.argv[1])
kernel.FreeConsole()
kernel.AttachConsole(pid)
kernel.SetConsoleCtrlHandler(None, 1)
kernel.GenerateConsoleCtrlEvent(0, 0)
sys.exit(0)

The initial process must be launched in a separate console so that the Ctrl-C event will not leak. Example

p = subprocess.Popen(['some_command'], creationflags=subprocess.CREATE_NEW_CONSOLE)

# Do something else

subprocess.check_call([sys.executable, 'ctrl_c.py', str(p.pid)]) # Send Ctrl-C

where I named the wrapper script as ctrl_c.py.

Solution 3

Try calling the GenerateConsoleCtrlEvent function using ctypes. As you are creating a new process group, the process group ID should be the same as the pid. So, something like

import ctypes

ctypes.windll.kernel32.GenerateConsoleCtrlEvent(0, proc.pid) # 0 => Ctrl-C

should work.

Update: You're right, I missed that part of the detail. Here's a post which suggests a possible solution, though it's a bit kludgy. More details are in this answer.

Solution 4

Here is a fully working example which doesn't need any modification in the target script.

This overrides the sitecustomize module so it might no be suitable for every scenario. However, in this case you could use a *.pth file in site-packages to execute code at the subprocess startup (see https://nedbatchelder.com/blog/201001/running_code_at_python_startup.html).

Edit This works only out of the box for subprocesses in Python. Other processes have to manually call SetConsoleCtrlHandler(NULL, FALSE).

main.py

import os
import signal
import subprocess
import sys
import time


def main():
    env = os.environ.copy()
    env['PYTHONPATH'] = '%s%s%s' % ('custom-site', os.pathsep,
                                    env.get('PYTHONPATH', ''))
    proc = subprocess.Popen(
        [sys.executable, 'sub.py'],
        env=env,
        creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
        )
    time.sleep(1)
    proc.send_signal(signal.CTRL_C_EVENT)
    proc.wait()


if __name__ == '__main__':
    main()

custom-site\sitecustomize.py

import ctypes
import sys
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)

if not kernel32.SetConsoleCtrlHandler(None, False):
    print('SetConsoleCtrlHandler Error: ', ctypes.get_last_error(),
          file=sys.stderr)

sub.py

import atexit
import time


def cleanup():
    print ('cleanup')

atexit.register(cleanup)


while True:
    time.sleep(1)

Solution 5

For those interested in a "quick fix", I've made a console-ctrl package based on Siyuan Ren's answer to make it even easier to use.

Simply run pip install console-ctrl, and in your code:

import console_ctrl
import subprocess

# Start some command IN A SEPARATE CONSOLE
p = subprocess.Popen(['some_command'], creationflags=subprocess.CREATE_NEW_CONSOLE)
# ...

# Stop the target process
console_ctrl.send_ctrl_c(p.pid)

View more solutions

Comments

[zwol]

This didn't work either, so I reread the MSDN page and realized that it specifically says "you cannot send CTRL_C_EVENT to a process group, it has no effect". Sending CTRL_BREAK_EVENT instead does work (without ctypes even), and does precisely what I want in a toy test program, but when I use it on my real program-under-test I get the "has encountered a problem and needs to close" dialog box over and over again. Any ideas?

[aknuds1]

This technique (of the parent sending Ctrl+C to itself and its related processes) really works! I arrived at it independently, but I found that one should wait in the parent until SIGINT is handled, to avoid the signal interrupting e.g. system calls once it arrives.

[jfs]

the order of arguments is wrong. It should be os.kill(pid, sig) instead of os.kill(sig, pid). Though os.kill(0, signal.CTRL_C_EVENT) doesn't interrupt input() call on Python 3.5 in vm with Windows 7 (the intent is to send Ctrl+C to all processes that share the console)

[jfs]

@zwol: the docs say "This signal cannot be generated for process groups." but os.kill(0, signal.CTRL_C_EVENT) generates KeyboardInterrupt in ipython in Windows console for me as if I've pressed Ctrl+C manually i.e., you can use CTRL_C_EVENT with 0 ("the signal is generated in all processes that share the console of the calling process.").

[Eryk Sun]

MSDN is wrong that process groups can't receive Ctrl+C. It's a peculiar claim because every process is either the lead of a new process group or inherits the group of its parent. By default the first process in a new group has Ctrl+C disabled in its ProcessParameters->ConsoleFlags, which is inherited by child processes. The console itself is oblivious to this flag; it's handled within process by the control-thread startup function, CtrlRoutine in kernelbase.dll. A process that needs Ctrl+C support should override the inherited/initial flag value via SetConsoleCtrlHandler(NULL, FALSE).

[Eryk Sun]

This isn't checking whether SetConsoleCtrlHandler failed. ctypes doesn't raise Python exceptions for a failed function call. That has to be done manually, or automated with a ctypes errcheck function. Anyway, we don't need to raise an exception in this case since there isn't anything to be done about it, but we should check for and log failure. Use kernel32 = ctypes.WinDLL('kernel32', use_last_error=True). Then, for example, if not kernel32.SetConsoleCtrlHandler(None, False): print('SetConsoleCtrlHandler Error: ', ctypes.get_last_error(), file=sys.stderr).

[Eryk Sun]

That said, if we're in control of the child process, we should also have a C SIGBREAK handler for a clean shutdown. This will be invoked for both CTRL_BREAK_EVENT and CTRL_CLOSE_EVENT (i.e. Ctrl+Break or closing the console window). Unfortunately the Python interpreter prevents handling a SIGBREAK that's generated from closing the console, since its C signal handler returns immediately and thus csrss.exe terminates the process before the Python handler executes. As a workaround, we can set a ctypes callback handler via SetConsoleCtrlHandler, which bypasses Python's C handler.

[Eryk Sun]

send_signal(CTRL_C_EVENT) works fine provided the child process is the leader of a process group and manually enables Ctrl+C via SetConsoleCtrlHandler(NULL, FALSE), which will be inherited by its own child processes. The documentation's claim that "CTRL_C_EVENT is ignored for process groups" is nonsense. Every process is in a process group. A new group initially has Ctrl+C disabled.

[Eryk Sun]

os.kill(0, CTRL_C_EVENT) or GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0) is ham handed. It sends the event to all processes attached to the console, including ancestors. Use a process group, and use Ctrl+Break if Ctrl+C is disabled. If it doesn't handle Ctrl+Break, then it likely also doesn't handle Ctrl+Close from closing the console window, since both events are mapped to C SIGBREAK. That's an oversight that should be submitted as a bug report if possible. All console applications that need special handling for a clean shutdown should handle Ctrl+Break and Ctrl+Close.

[Daniel Z.]

Very useful example! Thanks! Instead of using ctrl_c.py as file, I wrote procedure and run it using multiprocessing.Process() Work perfectly.

[Andrey Moiseev]

Important: the process to be killed has to have a console, so it should be started with creationflags=CREATE_NEW_CONSOLE, startupinfo=STARTUPINFO(dwFlags=STARTF_USESHOWWINDOW, wShowWindow=SW_HIDE)