Python testing: using a fake file with mock & io.StringIO

Question

I'm trying to test some code that operates on a file, and I can't seem to get my head around how to replace using a real file with mock and io.StringIO My code is pretty much the following:

class CheckConfig(object):
    def __init__(self, config):
        self.config = self._check_input_data(config)

    def _check_input_data(self, data):
        if isinstance(data, list):
            return self._parse(data)
        elif os.path.isfile(data):
            with open(data) as f:
                return self._parse(f.readlines())

    def _parse(self, data):
        return data

I have a class that can take either a list or a file, if it's a file it opens it and extracts the contents into a list, and then does what it needs to do to the resulting list.

I have a working test as follows:

def test_CheckConfig_with_file():
    config = 'config.txt'
    expected = parsed_file_data
    actual = CheckConfig(config).config
    assert expected == actual

I want to replace the call to the filesystem. I have tried replacing the file with io.StringIO but I get a TypeError from os.path.isfile() as it's expecting either a string, bytes or int. I also tried mocking the isfile method like so:

@mock.patch('mymodule.os.path')
def test_CheckConfig_with_file(mock_path):
    mock_path.isfile.return_value = True
    config = io.StringIO('data')
    expected = parsed_file_data
    actual = CheckConfig(config).config
    assert expected == actual

but I still get the same TypeError as the _io.StringIO type is causing the exception before isfile gets a chance to return something.

How can I get os.path.isfile to return True, when I pass it a fake file? Or is this a suggestion I should change my code?

Solution 1

Just mock out both os.path.isfile and the open() call, and pass in a fake filename (you are not expected to pass in an open file, after all).

The mock library includes a utility for the latter: mock_open():

@mock.patch('os.path.isfile')
def test_CheckConfig_with_file(mock_isfile):
    mock_isfile.return_value = True
    config_data = mock.mock_open(read_data='data')
    with mock.patch('mymodule.open', config_data) as mock_open:
        expected = parsed_file_data
        actual = CheckConfig('mocked/filename').config
        assert expected == actual

This causes the if isinstance(data, list): test to be false (because data is a string instead), followed by the elif os.path.isfile(data): returning True, and the open(data) call to use your mocked data from the mock_open() result.

You can use the mock_open variable to assert that open() was called with the right data (mock_open. assert_called_once_with('mocked/filename') for example).

Demo:

>>> import os.path
>>> from unittest import mock
>>> class CheckConfig(object):
...     def __init__(self, config):
...         self.config = self._check_input_data(config)
...     def _check_input_data(self, data):
...         if isinstance(data, list):
...             return self._parse(data)
...         elif os.path.isfile(data):
...             with open(data) as f:
...                 return self._parse(f.readlines())
...     def _parse(self, data):
...         return data
...
>>> with mock.patch('os.path.isfile') as mock_isfile:
...     mock_isfile.return_value = True
...     config_data = mock.mock_open(read_data='line1\nline2\n')
...     with mock.patch('__main__.open', config_data) as mock_open:
...         actual = CheckConfig('mocked/filename').config
...
>>> actual
['line1\n', 'line2\n']
>>> mock_open.mock_calls
[call('mocked/filename'),
 call().__enter__(),
 call().readlines(),
 call().__exit__(None, None, None)]

Solution 2

In case you end up here wondering how to solve this using the pytest-mock library, here is how you do it:

def test_open(mocker):
    m = mocker.patch('builtins.open', mocker.mock_open(read_data='bibble'))
    with open('foo') as h:
        result = h.read()

    m.assert_called_once_with('foo')
    assert result == 'bibble'

This code example was found (but had to be adjusted) here.

Comments

[bordeltabernacle]

Excellent, thankyou! This is exactly what I needed, and has helped me get a better grasp on mocking. Just out of interest, is there a reason you are using a context manager rather than a @mock.patch decorator. In previous tests I had been using the decorator, but here I couldn't find a way as I was using a pytest.fixture for the content of the open call, which wasn't available until inside the test function.

[Luis Meraz]

for anyone stumbling on this answer. mock_open currently doesn't support iterables so if instead of f.readlines() you used yield you will have to work around the iteration. stackoverflow.com/questions/24779893/…

[John]

Echoing bordeltabernacle comment. Based on this answer I could mock file contents using either a decorator or context manager. However, the decorator did not support taking a parameter as identifier so I was in the unusual position of having a patch decorator and just self parameter.

[Martijn Pieters]

The only reason I used a context manager here is that it is easier to demo with the context manager. The decorator version work exactly the same; the decorator just sets up the context manager and passes in the patch object as an extra argument to the decorated function.

[Rmatt]

Except this works for absolutely everything, including templates for errors and exceptions, which can lead to surprising results! :-)