input to C++ executable python subprocess

Question

I have a C++ executable which has the following lines of code in it

/* Do some calculations */
.
.
for (int i=0; i<someNumber; i++){
   int inputData;
   std::cin >> inputData;
   std::cout<<"The data sent from Python is :: "<<inputData<<std::endl;
   .
   .
   /* Do some more calculations with inputData */
}

and this is called in a loop. I want to call this executable in python subprocess like

p = Popen(['./executable'], shell=True, stdout=PIPE, stderr=PIPE, stdin=PIPE)

I could get the output from the executable using

p.server.stdout.read()

But I am not able to send data (integers) from python using

p.stdin.write(b'35')

Since cin is called in a loop, the stdin.write should also be called multiple times (in a loop). Is this above possible .. ?

Any hints and suggestion how I could do it ? Thanks in advance.

Solution

Here's minimalistic example of how to call a C++ executable from Python, and communicate with it from Python.

1) Note that you must add \n when writing to input stream (i.e. stdin) of the subprocess (just like you would hit Rtn if running the program manually).

2) Also note the flushing of the streams, so that the receiving program doesn't get stuck waiting for the whole buffer to fill before printing the result.

3) And if running Python 3, be sure to convert the streaming value from string to bytes (see https://stackoverflow.com/a/5471351/1510289).

Python:

from subprocess import Popen, PIPE

p = Popen(['a.out'], shell=True, stdout=PIPE, stdin=PIPE)
for ii in range(10):
    value = str(ii) + '\n'
    #value = bytes(value, 'UTF-8')  # Needed in Python 3.
    p.stdin.write(value)
    p.stdin.flush()
    result = p.stdout.readline().strip()
    print(result)

C++:

#include <iostream>

int main(){
    for( int ii=0; ii<10; ++ii ){
        int input;
        std::cin >> input;
        std::cout << input*2 << std::endl;
        std::cout.flush();
    }
}

Output of running Python:

0
2
4
6
8
10
12
14
16
18

Comments

[AdityaG]

thank you very much for your help. But unfortunately I get error saying TypeError: 'str' does not support the buffer interface ....

[Joran Beasley]

@AdityaG just change to p.stdin.write(b'35\n') ... that use bytes instead of str (@Velimir is using str , maybe different python versions)

[Velimir Mlaker]

@Joran Beasley, you're right. AdityaG, you must be running Python 3. In that case, you must convert the string to bytes. I added a comment that does just that.

[AdityaG]

@Velimir ... one more small question, do you think we can send numpy array like that ..? (I will experiment but am just curious).

[Velimir Mlaker]

Well, using this methodology you can stream anything you want from Python, so long as your C++ executable properly parses standard input. But ideally you'd call your C++ computation routines directly in Python, by wrapping your C++ code (e.g. using SWIG) and generating Python callable functions -- creating subprocesses and streaming bytes is not optimal.