Capture stdout from a script?
Solution 1
Setting stdout
is a reasonable way to do it. Another is to run it as another process:
import subprocess
proc = subprocess.Popen(["python", "-c", "import writer; writer.write()"], stdout=subprocess.PIPE)
out = proc.communicate()[0]
print out.upper()
Solution 2
For future visitors: Python 3.4 contextlib provides for this directly (see Python contextlib help) via the redirect_stdout
context manager:
from contextlib import redirect_stdout
import io
f = io.StringIO()
with redirect_stdout(f):
help(pow)
s = f.getvalue()
Solution 3
Here is a context manager version of your code. It yields a list of two values; the first is stdout, the second is stderr.
import contextlib
@contextlib.contextmanager
def capture():
import sys
from cStringIO import StringIO
oldout,olderr = sys.stdout, sys.stderr
try:
out=[StringIO(), StringIO()]
sys.stdout,sys.stderr = out
yield out
finally:
sys.stdout,sys.stderr = oldout, olderr
out[0] = out[0].getvalue()
out[1] = out[1].getvalue()
with capture() as out:
print 'hi'
Solution 4
Starting with Python 3 you can also use sys.stdout.buffer.write()
to write (already) encoded byte strings to stdout (see stdout in Python 3).
When you do that, the simple StringIO
approach doesn't work because neither sys.stdout.encoding
nor sys.stdout.buffer
would be available.
Starting with Python 2.6 you can use the TextIOBase
API, which includes the missing attributes:
import sys
from io import TextIOWrapper, BytesIO
# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)
# do some writing (indirectly)
write("blub")
# get output
sys.stdout.seek(0) # jump to the start
out = sys.stdout.read() # read output
# restore stdout
sys.stdout.close()
sys.stdout = old_stdout
# do stuff with the output
print(out.upper())
This solution works for Python 2 >= 2.6 and Python 3.
Please note that our sys.stdout.write()
only accepts unicode strings and sys.stdout.buffer.write()
only accepts byte strings.
This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes.
If you need to support code that sends byte strings to stdout directly without using stdout.buffer, you can use this variation:
class StdoutBuffer(TextIOWrapper):
def write(self, string):
try:
return super(StdoutBuffer, self).write(string)
except TypeError:
# redirect encoded byte strings directly to buffer
return super(StdoutBuffer, self).buffer.write(string)
You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.
Solution 5
This is the decorator counterpart of my original code.
writer.py
remains the same:
import sys
def write():
sys.stdout.write("foobar")
mymodule.py
sligthly gets modified:
from writer import write as _write
from decorators import capture
@capture
def write():
return _write()
out = write()
# out post processing...
And here is the decorator:
def capture(f):
"""
Decorator to capture standard output
"""
def captured(*args, **kwargs):
import sys
from cStringIO import StringIO
# setup the environment
backup = sys.stdout
try:
sys.stdout = StringIO() # capture output
f(*args, **kwargs)
out = sys.stdout.getvalue() # release output
finally:
sys.stdout.close() # close the stream
sys.stdout = backup # restore original stdout
return out # captured output wrapped in a string
return captured
Related videos on Youtube
Paolo
Enthusiast software developer. Self reminders I'm here to learn. Learning is an experience, everything else is just information. (A.Einstein)
Updated on July 08, 2022Comments
-
Paolo almost 2 years
suppose there is a script doing something like this:
# module writer.py import sys def write(): sys.stdout.write("foobar")
Now suppose I want to capture the output of the
write
function and store it in a variable for further processing. The naive solution was:# module mymodule.py from writer import write out = write() print out.upper()
But this doesn't work. I come up with another solution and it works, but please, let me know if there is a better way to solve the problem. Thanks
import sys from cStringIO import StringIO # setup the environment backup = sys.stdout # #### sys.stdout = StringIO() # capture output write() out = sys.stdout.getvalue() # release output # #### sys.stdout.close() # close the stream sys.stdout = backup # restore original stdout print out.upper() # post processing
-
Joshua Richardson almost 11 yearsLove this solution. I modified, so as not to accidentally lose stuff from a stream on which I'm not expecting output, e.g. unexpected errors. In my case, capture() can accept sys.stderr or sys.stdout as a parameter, indicating to only capture that stream.
-
mafrosis over 10 yearsStringIO doesn't support unicode in any fashion, so you can integrate the answer here to make the above support non-ASCII chars: stackoverflow.com/a/1819009/425050
-
Eric over 10 yearsModifying a yielded value in the finally is really rather wierd -
with capture() as out:
will behave differently towith capture() as out, err:
-
JonnyJD over 10 yearsUnicode / stdout.buffer support can be reached with using the io module. See my answer.
-
Arthur over 9 yearscheck_output directly captures the output of a command run in a subprocess: <br> value = subprocess.check_output( command, shell=True)
-
letmaik over 9 yearsThis solution breaks if you use
subprocess
and redirect output to sys.stdout/stderr. This is becauseStringIO
is not a real file object and misses thefileno()
function. -
weaver over 6 yearsThis does not solve the problem when trying to write to sys.stdout.buffer (as you need to do when writing bytes). StringIO does not have the buffer attribute, while TextIOWrapper does. See the answer from @JonnyJD.
-
Boris Verkhovskiy about 3 yearsOn modern Python versions you can do
capture_output=True
instead ofstdout=subprocess.PIPE
.