How do I execute a command and get the output of the command within C++ using POSIX?

Question

I am looking for a way to get the output of a command when it is run from within a C++ program. I have looked at using the system() function, but that will just execute a command. Here's an example of what I'm looking for:

std::string result = system("./some_command");

I need to run an arbitrary command and get its output. I've looked at boost.org, but I have not found anything that will give me what I need.

Solution 1

#include <cstdio>
#include <iostream>
#include <memory>
#include <stdexcept>
#include <string>
#include <array>

std::string exec(const char* cmd) {
    std::array<char, 128> buffer;
    std::string result;
    std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
    if (!pipe) {
        throw std::runtime_error("popen() failed!");
    }
    while (fgets(buffer.data(), buffer.size(), pipe.get()) != nullptr) {
        result += buffer.data();
    }
    return result;
}

Pre-C++11 version:

#include <iostream>
#include <stdexcept>
#include <stdio.h>
#include <string>

std::string exec(const char* cmd) {
    char buffer[128];
    std::string result = "";
    FILE* pipe = popen(cmd, "r");
    if (!pipe) throw std::runtime_error("popen() failed!");
    try {
        while (fgets(buffer, sizeof buffer, pipe) != NULL) {
            result += buffer;
        }
    } catch (...) {
        pclose(pipe);
        throw;
    }
    pclose(pipe);
    return result;
}

Replace popen and pclose with _popen and _pclose for Windows.

Solution 2

Getting both stdout and stderr (and also writing to stdin, not shown here) is easy peasy with my pstreams header, which defines iostream classes that work like popen:

#include <pstream.h>
#include <string>
#include <iostream>

int main()
{
  // run a process and create a streambuf that reads its stdout and stderr
  redi::ipstream proc("./some_command", redi::pstreams::pstdout | redi::pstreams::pstderr);
  std::string line;
  // read child's stdout
  while (std::getline(proc.out(), line))
    std::cout << "stdout: " << line << '\n';
  # if reading stdout stopped at EOF then reset the state:
  if (proc.eof() && proc.fail())
    proc.clear();
  // read child's stderr
  while (std::getline(proc.err(), line))
    std::cout << "stderr: " << line << '\n';
} 

Solution 3

For Windows, popen also works, but it opens up a console window - which quickly flashes over your UI application. If you want to be a professional, it's better to disable this "flashing" (especially if the end-user can cancel it).

So here is my own version for Windows:

(This code is partially recombined from ideas written in The Code Project and MSDN samples.)

#include <windows.h>
#include <atlstr.h>
//
// Execute a command and get the results. (Only standard output)
//
CStringA ExecCmd(
    const wchar_t* cmd              // [in] command to execute
)
{
    CStringA strResult;
    HANDLE hPipeRead, hPipeWrite;

    SECURITY_ATTRIBUTES saAttr = {sizeof(SECURITY_ATTRIBUTES)};
    saAttr.bInheritHandle = TRUE; // Pipe handles are inherited by child process.
    saAttr.lpSecurityDescriptor = NULL;

    // Create a pipe to get results from child's stdout.
    if (!CreatePipe(&hPipeRead, &hPipeWrite, &saAttr, 0))
        return strResult;

    STARTUPINFOW si = {sizeof(STARTUPINFOW)};
    si.dwFlags     = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES;
    si.hStdOutput  = hPipeWrite;
    si.hStdError   = hPipeWrite;
    si.wShowWindow = SW_HIDE; // Prevents cmd window from flashing.
                              // Requires STARTF_USESHOWWINDOW in dwFlags.

    PROCESS_INFORMATION pi = { 0 };

    BOOL fSuccess = CreateProcessW(NULL, (LPWSTR)cmd, NULL, NULL, TRUE, CREATE_NEW_CONSOLE, NULL, NULL, &si, &pi);
    if (! fSuccess)
    {
        CloseHandle(hPipeWrite);
        CloseHandle(hPipeRead);
        return strResult;
    }

    bool bProcessEnded = false;
    for (; !bProcessEnded ;)
    {
        // Give some timeslice (50 ms), so we won't waste 100% CPU.
        bProcessEnded = WaitForSingleObject( pi.hProcess, 50) == WAIT_OBJECT_0;

        // Even if process exited - we continue reading, if
        // there is some data available over pipe.
        for (;;)
        {
            char buf[1024];
            DWORD dwRead = 0;
            DWORD dwAvail = 0;

            if (!::PeekNamedPipe(hPipeRead, NULL, 0, NULL, &dwAvail, NULL))
                break;

            if (!dwAvail) // No data available, return
                break;

            if (!::ReadFile(hPipeRead, buf, min(sizeof(buf) - 1, dwAvail), &dwRead, NULL) || !dwRead)
                // Error, the child process might ended
                break;

            buf[dwRead] = 0;
            strResult += buf;
        }
    } //for

    CloseHandle(hPipeWrite);
    CloseHandle(hPipeRead);
    CloseHandle(pi.hProcess);
    CloseHandle(pi.hThread);
    return strResult;
} //ExecCmd

Solution 4

I'd use popen() (++waqas).

But sometimes you need reading and writing...

It seems like nobody does things the hard way any more.

(Assuming a Unix/Linux/Mac environment, or perhaps Windows with a POSIX compatibility layer...)

enum PIPE_FILE_DESCRIPTERS
{
  READ_FD  = 0,
  WRITE_FD = 1
};

enum CONSTANTS
{
  BUFFER_SIZE = 100
};

int
main()
{
  int       parentToChild[2];
  int       childToParent[2];
  pid_t     pid;
  string    dataReadFromChild;
  char      buffer[BUFFER_SIZE + 1];
  ssize_t   readResult;
  int       status;

  ASSERT_IS(0, pipe(parentToChild));
  ASSERT_IS(0, pipe(childToParent));

  switch (pid = fork())
  {
    case -1:
      FAIL("Fork failed");
      exit(-1);

    case 0: /* Child */
      ASSERT_NOT(-1, dup2(parentToChild[READ_FD], STDIN_FILENO));
      ASSERT_NOT(-1, dup2(childToParent[WRITE_FD], STDOUT_FILENO));
      ASSERT_NOT(-1, dup2(childToParent[WRITE_FD], STDERR_FILENO));
      ASSERT_IS(0, close(parentToChild [WRITE_FD]));
      ASSERT_IS(0, close(childToParent [READ_FD]));

      /*     file, arg0, arg1,  arg2 */
      execlp("ls", "ls", "-al", "--color");

      FAIL("This line should never be reached!!!");
      exit(-1);

    default: /* Parent */
      cout << "Child " << pid << " process running..." << endl;

      ASSERT_IS(0, close(parentToChild [READ_FD]));
      ASSERT_IS(0, close(childToParent [WRITE_FD]));

      while (true)
      {
        switch (readResult = read(childToParent[READ_FD],
                                  buffer, BUFFER_SIZE))
        {
          case 0: /* End-of-File, or non-blocking read. */
            cout << "End of file reached..."         << endl
                 << "Data received was ("
                 << dataReadFromChild.size() << "): " << endl
                 << dataReadFromChild                << endl;

            ASSERT_IS(pid, waitpid(pid, & status, 0));

            cout << endl
                 << "Child exit staus is:  " << WEXITSTATUS(status) << endl
                 << endl;

            exit(0);


          case -1:
            if ((errno == EINTR) || (errno == EAGAIN))
            {
              errno = 0;
              break;
            }
            else
            {
              FAIL("read() failed");
              exit(-1);
            }

          default:
            dataReadFromChild . append(buffer, readResult);
            break;
        }
      } /* while (true) */
  } /* switch (pid = fork())*/
}

---

You also might want to play around with select() and non-blocking reads.

fd_set          readfds;
struct timeval  timeout;

timeout.tv_sec  = 0;    /* Seconds */
timeout.tv_usec = 1000; /* Microseconds */

FD_ZERO(&readfds);
FD_SET(childToParent[READ_FD], &readfds);

switch (select (1 + childToParent[READ_FD], &readfds, (fd_set*)NULL, (fd_set*)NULL, & timeout))
{
  case 0: /* Timeout expired */
    break;

  case -1:
    if ((errno == EINTR) || (errno == EAGAIN))
    {
      errno = 0;
      break;
    }
    else
    {
      FAIL("Select() Failed");
      exit(-1);
    }

  case 1:  /* We have input */
    readResult = read(childToParent[READ_FD], buffer, BUFFER_SIZE);
    // However you want to handle it...
    break;

  default:
    FAIL("How did we see input on more than one file descriptor?");
    exit(-1);
}

Solution 5

Two possible approaches:

1. I don't think popen() is part of the C++ standard (it's part of POSIX from memory), but it's available on every UNIX I've worked with (and you seem to be targeting UNIX since your command is ./some_command).

2. On the off-chance that there is no popen(), you can use system("./some_command >/tmp/some_command.out");, then use the normal I/O functions to process the output file.

View more solutions

Comments

[Misha M]

The hard way is right :) I like the idea with select() call, though in this case, I actually need to wait until the task completes. I'll keep this code for another project I have :)

[kalaxy]

Be aware that this will only grab stdout and not stderr.

[Fred Foo]

Also be aware that an exception can occur in result += buffer, so the pipe might not be properly closed.

[DragonLord]

popen is the correct solution to use when either input or output communication is desired from the child process, but not both.

[Jonathan Wakely]

I disagree. popen requires you to use the C stdio API, I prefer the iostreams API. popen requires you to manually clean up the FILE handle, pstreams do that automatically. popen only accepts a const char* for the argument, which requires care to avoid shell injection attacks, pstreams allows you to pass a vector of strings similar to execv, which is safer. popen gives you nothing but a pipe, pstreams tells you the child's PID allowing you to send signals e.g. to kill it if it's blocked or not exiting. All of those are advantages even if you only want unidirectional IO.

[Per Johansson]

...or you could use the existing posix_spawnp function

[jinyong lee]

@larsmans But it is because of a code error out of this function, right, not something that may be expected?

[Kristóf Marussy]

Your execlp call has a bug: the last arg pointer passed must be (char *) NULL to properly terminate the variadic argument list (see execlp(3) for reference).

[Igbanam]

when is this ever going to give "ERROR" as a return value from the function. Under what circumstances?

[TOMKA]

@Yasky: When the program being executed is int main(){ puts("ERROR"); }.

[Mark Lakata]

If this is C++, you should throw an exception, not return "ERROR".

[fnc12]

The answer is good but it would be better if you replace 'char* cmd' with 'const char* cmd'

[mehov]

@fnc12 I think it showed me some error and I actually did that. You might want to edit the answer to include your suggestion

[Jesse Chisholm]

Another issue with this solution is if the child writes to stderr enough to fill the buffers and block before it starts writing to stdout. The parent will block reading stdout, while the child is blocked waiting for stderr to be read. resource deadlock! At least one of those loops would be better as asynchronous (i.e., threaded).

[Jonathan Wakely]

@JesseChisholm, yes, that could be a problem. But you don't need to use threads because pstreams allows an approximation of non-blocking I/O using the iostream interface, specifically using the readsome function, which checks for readiness using pstreambuf::in_avail(), so won't block. That allows demultiplexing on the process' stdout and stderr as each has data available. pstreambuf::in_avail() only works 100% reliably if the OS supports the non-standard FIONREAD ioctl, but that is supported on (at least) GNU/Linux and Solaris.

[Jesse Chisholm]

@JonathanWakely - good to know that pstreams has the necessary support. BTW, FIONREAD is also available in Windows, for those working over there.

[Some programmer dude]

Why use while (!feof(...)) and an if statement in the loop, when all you have to do is while (fgets(...) != NULL)?

[QuantumKarl]

@JoachimPileborg Its actually safer too faq.cprogramming.com/cgi-bin/…

[Czipperz]

unique_ptr is a better fit here, where the actual reference count is never used.

[Jonathan Wakely]

@chiliNUT the new 1.0.1 release uses the Boost licence.

[Wolf]

This is my favourite solution for Windows, I hope you forgive my changes. I'd suggest to make the const-cast more explicit, whereas I consider the explicit usage of wchar_t and CreateProcessW as an unnecessary restriction.

[Wolf]

Simply changing popen and pclose into _pclose and _popen seems not enough for getting this running under Windows (producing, as you'll guess it, "popen() failed!"), whereas TarmoPikaro's solution works.

[TarmoPikaro]

Do you see any problem or potential problem with this cast ? I prefer to keep code at minimum and don't write it without need.

[Wolf]

After reading CreateProcess function (Windows), I see a real danger in doing this: The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation. So it's maybe better to copy the command line into a separate buffer first, to prevent the caller from getting its original input changed.

[Trevor Hickey]

You could use constexpr on the buffer size and avoid specifying a literal twice.

[Admin]

Bless your soul for the pre-C++11 version.

[arturn]

This code is just what I'm looking for! but I have one more problem. With windows programs like notepad.exe the while (!feof(pipe)) never finished. Do you have any idea how I can solve without adding a 'boolean isGui' var? Thanks!

[LachoTomov]

Thank you! This is the best popen implementation for Windows on the Internet! And by passing the CREATE_NO_WINDOW flag one can finally get rid of the annoying cmd prompts that show up.

[ColH]

Is there a danger of the shell'd command never responding/exiting?

[Antti Haapala -- Слава Україні]

while (!feof()) is always wrong, and fgets is equally wrong when you're not really reading lines. Use fread and check the return value.

[Alex Mathew]

@waqas is possible to return only instanceType from result. I'm new to this. Please help. Thanks in advance

[Darren Smith]

C++11 example doesn't compile; unnecessary curly bracket before return statement. Also, I would use fread instead of fgets, no need to split on newlines etc.

[Refael Sheinker]

This answer does not handle stderr properly.

[Refael Sheinker]

Where do you pass the CREATE_NO_WINDOW thingy?

[Refael Sheinker]

@Bill Moore, if you notice, there is a bug in your answer. ListStdErr is never used.

[Aaron Franke]

Is this still the best practice with C++17?

[xception]

it's not best practice even in c++11, as fgets doesn't actually return the number of read bytes so reading a '\0' would break the code @AaronFranke , as @AnttiHaapala suggests fread should be used instead

[255.tar.xz]

Does this also work for Unix systems? Or would I have to use something else for a Unix device?

[A. K.]

@JonathanWakely how can i kill the ipstream after say a 5 second timeout?

[A. K.]

Also string_view isn't supported for basic_ipstream.

[Jonathan Wakely]

@A.K. you'd have to implement the timeout yourself using non-blocking reads (i.e. readsome) and a loop. Adding string_view support is simple, once I get a round tuit.

[Steven Lu]

It is important to understand that this launches a shell which performs the command argument tokenization.

[kittu]

Will this work on unix, linux and windows ? Can you please header files as well?

[kittu]

How can I pass .bat file as argument instead of cmd?

[Ahmed Hussein]

Why buffer size exactly 128 ?

[0x0C4]

But what if you need the return value from pclose() ? :)

[CK.]

@JonathanWakely Is there a way to get the return value from the process?

[Jonathan Wakely]

@CK. read the FAQ: pstreams.sourceforge.net/faq.html#faq_exit_status

[Jonathan Wakely]

@AdityaG15 the process' current working directory has nothing to do with where the executable is installed. It depends where you run it from, not where it's installed. And after changing the working directory, a new process created by a pstreams object will inherit the changed working directory from the parent process. So I think you are mistaken.

[AdityaG15]

@JonathanWakely yes I also meant that, i have my executable in build/ directory, say i run it from a directory inside build/ with ../exec, the working directory should have been the directory where i executed this, but it executed as if in build/ and behaviour of std::system and pstreams was SURELY different (adjacent to each other, same command, in my case it was git rev-parse HEAD) that time. I will check it again though, thanks for your efforts on the library :D

[Jonathan Wakely]

@AdityaG15 you are mistaken.

[Mikolasan]

@0x0C4 use pre-C++11 version or write a special deleter: auto pipe_deleter = [&exit_status](FILE* stream) { exit_status = pclose(stream); };

[Richard Lalancette]

Good solution, but requires custom code, hence my downvote.

[Eric Duminil]

@RefaelSheinker: I think you can replace the 0 in createProcess by CREATE_NO_WINDOW. docs.microsoft.com/en-us/windows/win32/procthread/…

[Eric Duminil]

@RefaelSheinker: Indeed. I think the second ListStdOut += s; should be replaced with ListStdErr += s;, if you want to have two distinct strings. I'd like to have them merged anyway, so I'll simply remove ListStdErr. Finally, List is kind of a weird name for a string.

[Max]

This breaks for me for the output of fping. Apparently this is related to what @xception said, because if I go to debug and check buffer it indeed includes \0. However, I also tried while (fread(buffer, 1, ftell(pipe.get()), pipe.get()) > 0) and this will just get the first line. Can someone fix that for me?

[xception]

As I said you should use fread instead, and account for the fact that fread returns the number of bytes read... I hope you can take it further from there. fread only has 1 extra parameter which in this example is the second parameter and should have the value 1, read the man page for the rest of the info

[Ben Voigt]

Beware of naming confusion with the POSIX standard family of 'exec' functions

[Tim Autin]

Thanks. How can you kill a started redi::ipstream? I tried proc.rdbuf()->kill(15) without success.

[Jonathan Wakely]

@TimAutin that will send SIGTERM to the child process. If that doesn't kill it, that's your child process' fault. kill(9) will be more aggressive. What are you expecting to happen? Maybe the child was killed but is a zombie until you wait for it.

[Tim Autin]

SIGTERM does kill the process (it's an FFmpeg instance). I switched to libexecstream, it works now (with SIGTERM, no need for SIGKILL). Thanks anyway!

[John]

@Wolf Could pclose works well when the popen returns NULL?

[John]

@FredFoo It's seems there is no such problem(i.e. the pipe might not be properly closed)now. Am I right?

[Wolf]

@John That's long ago, I'm not sure what my comment was about (maybe something like this stackoverflow.com/q/40758828/2932052 ?). But pclose should not be attempted if popen has already failed, even though there may be APIs that are tolerant in handling such “null handles”.

[John]

@Wolf If I understand you correctly, the code snippet in this answer needs to be improved. You see, std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose) would call pclose even if popen fails. How do you think about it?

[Wolf]

@John Sorry, I have not the resources to deal with your question. If you fail to search for it, maybe you should ask a new one about this detail of the std::unique_ptr approach. (In that case, it might be beneficial to link the new question here as well.)

[John]

@Wolf I see. Here is the new question.stackoverflow.com/questions/72371379/pclose-with-nu‌​ll-parameter

[infinitezero]

This only works partially. If you want to retrieve the ID of the started process, it won't work.

[Kevin]

@John I don't see why pclose would be called if popen fails. The std::unique_ptr destructor (and also the reset function) doesn't call the deleter if it's not holding a value: "If get() == nullptr there are no effects. Otherwise, the owned object is destroyed via get_deleter()(get())."