Run interactive Bash with popen and a dedicated TTY Python

Question

I need to run an interactive Bash instance in a separated process in Python with it's own dedicated TTY (I can't use pexpect). I used this code snippet I commonly see used in similar programs:

master, slave = pty.openpty()

p = subprocess.Popen(["/bin/bash", "-i"], stdin=slave, stdout=slave, stderr=slave)

os.close(slave)

x = os.read(master, 1026)

print x

subprocess.Popen.kill(p)
os.close(master)

But when I run it I get the following output:

$ ./pty_try.py
bash: cannot set terminal process group (10790): Inappropriate ioctl for device
bash: no job control in this shell

Strace of the run shows some errors:

...
readlink("/usr/bin/python2.7", 0x7ffc8db02510, 4096) = -1 EINVAL (Invalid argument)
...
ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffc8db03590) = -1 ENOTTY (Inappropriate ioctl for device)
...
readlink("./pty_try.py", 0x7ffc8db00610, 4096) = -1 EINVAL (Invalid argument)

The code snippet seems pretty straightforward, is Bash not getting something it needs? what could be the problem here?

Solution 1

This is the solution that worked for me at the end (as suggested by qarma) :

libc = ctypes.CDLL('libc.so.6')

master, slave = pty.openpty()
p = subprocess.Popen(["/bin/bash", "-i"], preexec_fn=libc.setsid, stdin=slave, stdout=slave, stderr=slave)
os.close(slave)

... do stuff here ...

x = os.read(master, 1026)
print x

Solution 2

This is a solution to run an interactive command in subprocess. It uses pseudo-terminal to make stdout non-blocking(also some command needs a tty device, eg. bash). it uses select to handle input and ouput to the subprocess.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen

command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()

# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())

# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()


try:
    # use os.setsid() make it run in a new process group, or bash job control will not be enabled
    p = Popen(command,
              preexec_fn=os.setsid,
              stdin=slave_fd,
              stdout=slave_fd,
              stderr=slave_fd,
              universal_newlines=True)

    while p.poll() is None:
        r, w, e = select.select([sys.stdin, master_fd], [], [])
        if sys.stdin in r:
            d = os.read(sys.stdin.fileno(), 10240)
            os.write(master_fd, d)
        elif master_fd in r:
            o = os.read(master_fd, 10240)
            if o:
                os.write(sys.stdout.fileno(), o)
finally:
    # restore tty settings back
    termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)

Comments

[Dima Tisnek]

That's quite normal — you got an interactive shell without job control.

[Dima Tisnek]

If you want job control too, you need your shell to become a process leader — that is start new "session", it's achieved with start_new_session=True keyword argument to Popen (since Python 3.2). If you need more control, use preexec_fn=...

[TKKS]

Ok, that sound reasonable. I understand that the start_new_session=True is only relevant to >3.2. Is there an equivalent in 2.7? Sorry, probably should have mentioned the python version in the question.

[Dima Tisnek]

You can do that by hand by calling setsid() in preexec_fn via ctypes

[Dima Tisnek]

I think this question is about same fundamentals as stackoverflow.com/questions/23826695/… stackoverflow.com/questions/33119213/… stackoverflow.com/questions/37737649/… stackoverflow.com/questions/13243807/… it could be considered a duplicate.

[TKKS]

I don't think any of them is really about using the pseudo-terminal with Popen like this. I don't think I could have solved this issue with any of these other questions. I will publish my solution code.

[K.Mulier]

How to do this on Windows, using the cmd shell instead of bash?

[Dima Tisnek]

I doubt you need -i any more with all the other things, bash should detect it's environment.

[balki]

From docs, pty.spawn seem to do similar stuff but simpler interface. docs.python.org/3/library/pty.html#example Are they doing the same?

[moi]

@balki pty.spawn is blocking: "It is expected that the process spawned behind the pty will eventually terminate, and when it does spawn will return."

[Paco]

@balki I've looked at the source code on github.com/python/cpython/blob/master/Lib/pty.py#L151. I think pty.spawn is doing basically the same thing as this code snippet does. So yes, use pty.spawn as it would make application code simpler.

[anthony sottile]

if the process exits without output this hangs waiting for input (setting a timeout on select.select or setting it to nonblocking "fixes" this). this also needs a little work for window sizing (SIGWINCH, etc.)

[Noortheen Raja]

Is it possible to separate stdout/stderr ?

[Trevor Boyd Smith]

in my case the subprocess required an interactive tty ... so when i tried doing bash app.py & it error'd out. then i tried passing stdin=subprocess.DEVNULL and that didn't work. then i tried passing an open file descriptor and that didn't work. finally i opened the pty.openpty() and passed in to the subprocess stdin=slave_fd and it worked.