Can I redirect the stdout into some sort of string buffer?

Question

I'm using python's ftplib to write a small FTP client, but some of the functions in the package don't return string output, but print to stdout. I want to redirect stdout to an object which I'll be able to read the output from.

I know stdout can be redirected into any regular file with:

stdout = open("file", "a")

But I prefer a method that doesn't uses the local drive.

I'm looking for something like the BufferedReader in Java that can be used to wrap a buffer into a stream.

Solution 1

from cStringIO import StringIO # Python3 use: from io import StringIO
import sys

old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()

# blah blah lots of code ...

sys.stdout = old_stdout

# examine mystdout.getvalue()

Solution 2

There is a contextlib.redirect_stdout() function in Python 3.4+:

import io
from contextlib import redirect_stdout

with io.StringIO() as buf, redirect_stdout(buf):
    print('redirected')
    output = buf.getvalue()

Here's a code example that shows how to implement it on older Python versions.

Solution 3

Just to add to Ned's answer above: you can use this to redirect output to any object that implements a write(str) method.

This can be used to good effect to "catch" stdout output in a GUI application.

Here's a silly example in PyQt:

import sys
from PyQt4 import QtGui

class OutputWindow(QtGui.QPlainTextEdit):
    def write(self, txt):
        self.appendPlainText(str(txt))

app = QtGui.QApplication(sys.argv)
out = OutputWindow()
sys.stdout=out
out.show()
print "hello world !"

Solution 4

A context manager for python3:

import sys
from io import StringIO


class RedirectedStdout:
    def __init__(self):
        self._stdout = None
        self._string_io = None

    def __enter__(self):
        self._stdout = sys.stdout
        sys.stdout = self._string_io = StringIO()
        return self

    def __exit__(self, type, value, traceback):
        sys.stdout = self._stdout

    def __str__(self):
        return self._string_io.getvalue()

use like this:

>>> with RedirectedStdout() as out:
>>>     print('asdf')
>>>     s = str(out)
>>>     print('bsdf')
>>> print(s, out)
'asdf\n' 'asdf\nbsdf\n'

Solution 5

Starting with Python 2.6 you can use anything implementing the TextIOBase API from the io module as a replacement. This solution also enables you to use sys.stdout.buffer.write() in Python 3 to write (already) encoded byte strings to stdout (see stdout in Python 3). Using StringIO wouldn't work then, because neither sys.stdout.encoding nor sys.stdout.buffer would be available.

A solution using TextIOWrapper:

import sys
from io import TextIOWrapper, BytesIO

# setup the environment
old_stdout = sys.stdout
sys.stdout = TextIOWrapper(BytesIO(), sys.stdout.encoding)

# do something that writes to stdout or stdout.buffer

# get output
sys.stdout.seek(0)      # jump to the start
out = sys.stdout.read() # read output

# restore stdout
sys.stdout.close()
sys.stdout = old_stdout

This solution works for Python 2 >= 2.6 and Python 3.

Please note that our new sys.stdout.write() only accepts unicode strings and sys.stdout.buffer.write() only accepts byte strings. This might not be the case for old code, but is often the case for code that is built to run on Python 2 and 3 without changes, which again often makes use of sys.stdout.buffer.

You can build a slight variation that accepts unicode and byte strings for write():

class StdoutBuffer(TextIOWrapper):
    def write(self, string):
        try:
            return super(StdoutBuffer, self).write(string)
        except TypeError:
            # redirect encoded byte strings directly to buffer
            return super(StdoutBuffer, self).buffer.write(string)

You don't have to set the encoding of the buffer the sys.stdout.encoding, but this helps when using this method for testing/comparing script output.

View more solutions

Comments

[Alexey]

I do not think stdout = open("file", "a") by itself will redirect anything.

[Ayman Hourieh]

+1, you don't need to keep a reference to the original stdout object, as it is always available at sys.__stdout__. See docs.python.org/library/sys.html#sys.__stdout__.

[yantrab]

Well, that's an interesting debate. The absolute original stdout is available, but when replacing like this, it's better to use an explicit save as I've done, since someone else could have replaced stdout and if you use stdout, you'd clobber their replacement.

[Nicolas Lefebvre]

Works for me with python 2.6 and PyQT4. Seems strange to down vote working code when you can't tell why it doesn't work !

[Anuvrat Parashar]

would this operation in one thread alter the behavior of other threads? I mean is it threadsafe?

[SourceSeeker]

@AnuvratParashar: I think that would be an excellent question to ask on its own.

[Will]

don't forget to add flush() too!

[PhilMacKay]

@AnuvratParashar: I'm pretty sure it is not thread safe, unless there are details I don't know about (or didn't notice), I used this method to retrieve print calls in other threads.

[JonnyJD]

This doesn't work if sys.stdout.buffer (Python 3) is used. See my answer for a solution working in that case. For old Python-2-only code this is still the best solution.

[Matthias Kuhn]

I highly recommend to reassign the old stdout in a finally: block, so it is also reassigned if an exception is risen in between. try: bkp = sys.stdout ... ... finally: sys.stdout = bkp

[jfs]

@erikb85: subprocess.call() requires that sys.stdout.fileno() is redirected, the solution that just replaces sys.stdout won't work even for functions that store sys.stdout value locally in Python. See stdout_redirected() that redirects sys.stdout.fileno() instead of replacing sys.stdout. Though if it is your code, you could use subprocess.call(stdout=..) to redirect stdout of a subprocess (or just call subprocess.check_output()).

[Anthony Labarre]

If you want to use this in Python 3, replace cStringIO with io .

[CMCDragonkai]

There's also redirect_stderr on the latest Python too!

[toonice]

You could improve the quality of your Answer by explaining how the above code works and how this is an improvement over the Questioner's situation.

[fragorl]

This answer helped me when setting up an Environment object's stdout param for use with Httpie's core.py.

[karansthr]

for python3 use from io import StringIO

[Soner from The Ottoman Empire]

I think there is no need to add try/finally block for this solution.

[jrieke]

+1 for @NedBatchelder comment to store stdout. E.g. in Jupyter notebooks, stdout is altered and resetting to sys.__stdout__ screws up printing in Jupyter.

[Martin]

Replacing built-in library attributes is bad practice (a.o. because of thread safety)