Python mock builtin 'open' in a class using two different files

13,467

Solution 1

You must use side_effect attribute of your patched open object (mock_open) and don't forget to set the return_value for __exit__ method.

@patch('__builtin__.open', spec=open)
def test_interface_mapping(self, mock_open):
    handle1 = MagicMock()
    handle1.__enter__.return_value.__iter__.return_value = ('aa', 'bb')
    handle1.__exit__.return_value=False
    handle2 = MagicMock()
    handle2.__enter__.return_value.__iter__.return_value = ('AA', 'BB')
    handle2.__exit__.return_value=False
    mock_open.side_effect = (handle1, handle2)
    with open("ppp") as f:
        self.assertListEqual(["aa","bb"],[x for x in f])
    with open("ppp") as f:
        self.assertListEqual(["AA","BB"],[x for x in f])

[EDIT] I found a much more elegant way to do it Mock builtin 'open" function when used in contextlib

So you can rewrote test like

@patch('__builtin__.open', new_callable=mock_open, read_data="aa\nbb")
def test_interface_mapping_new(self, mo):
    handlers = (mo.return_value,mock_open(read_data="AA\nBB").return_value,)
    mo.side_effect = handlers
    with open("ppp") as f:
        self.assertEqual("aa\nbb",f.read())
    with open("ppp") as f:
        self.assertEqual("AA\nBB",f.read())

And from python 3.4 you can use also readline(), readlines() without mocking anything else.

Solution 2

If you need much more control over file content you can use a wrapper function. It substitutes the content of a file according to the filename as the original open does.

import unittest.mock as mock


def my_open(filename):
    if filename == 'file.txt':
        content = "text file\ncontent"
    elif filename == 'second.txt':
        content = 'foobar'
    else:
        raise FileNotFoundError(filename)
    file_object = mock.mock_open(read_data=content).return_value
    file_object.__iter__.return_value = content.splitlines(True)
    return file_object

In the elif chain you set "file contents" for each existing file path.

Tests:

# standalone
open_patch = mock.patch('__main__.open', new=my_open)
open_patch.start()

file = open('file.txt')
assert file.read() == "text file\ncontent"
file.close()

open_patch.stop()

#with statement
with mock.patch('__main__.open', new=my_open):
    with open('second.txt') as file:
        assert file.read() == 'foobar'

    # as iterable
    with open('file.txt') as file:
        assert ['text file\n', 'content'] == list(file)

# function decorator
@mock.patch('__main__.open', new=my_open)
def test_patched_open():
    with open('second.txt') as file:
        assert file.readline() == 'foobar'

test_patched_open()

Solution 3

You'd create two 'file' mocks, and mock open to return these in sequence as open() is called. The side_effect attribute lets you do just that:

@patch('__builtin__.open')
def test_interface_mapping(self, mock_open):
    handle1 = MagicMock('file1').__enter__.return_value
    handle1.__iter__.return_value = ('aa', 'bb')
    handle2 = MagicMock('file2').__enter__.return_value
    handle2.__iter__.return_value = ('foo', 'bar')
    mock_open.return_value.side_effect = (handle1, handle2)

The mocked open() call returns first handle1 when called, then handle2. Either object then responds to __enter__() being called with a mock that returns a given tuple for the __iter__ call.

Share:
13,467

Related videos on Youtube

chromeeagle
Author by

chromeeagle

Updated on June 04, 2022

Comments

  • chromeeagle
    chromeeagle almost 2 years

    I am having trouble figuring out how to mock two file opens in a class when they both use context managers. I know how to do it for one context-managed file using the mock module like this:

    @patch('__builtin__.open')
    def test_interface_mapping(self, mock_config):
            m = MagicMock(spec=file)
            handle = m.return_value.__enter__.return_value
            handle.__iter__.return_value = ('aa', 'bb')
    

    My problem is how to do this when a class opens two different files in the same call. In my case, the class __init__() preloads the files into two maps. This class is used in other classes. I want to mock the loading of these two files to provide my test data so that the other classes that use the IfAddrConfig object can be tested against my preloaded test file content.

    Here's an example of the class I am struggling with that loads two files in __init__(), both of which I want to mock to load my test injected file contents. getInterfaceMap() is the function that is called frequently so I do not want that to be loading and parsing the files every call, hence the reason for preloading the maps in __init__() once.

    class IfAddrConfig(object):
        def __init__(self):
            # Initialize the static maps once since they require file operations
            # that we do not want to be calling every time getInterfaceMap() is used
            self.settings_map = self.loadSettings()
            self.config_map = self.loadConfig()
    
        def loadConfig(self):
            config_map = defaultdict(dict)
            with open(os.path.join('some_path.cfg'), 'r') as stream:
                for line in stream:
                    # Parse line and build up config_map entries
            return config_map
    
        def loadSettings(self):
            settings_map = {}
            with open('another_path.cfg', 'r') as stream:
                for line in stream:
                    # Parse line and build up settings_map entries
            return settings_map
    
        def getInterfaceMap(self, interface):
            # Uses both the settings and config maps to finally create a composite map
            # that is returned to called
            interface_map = {}
            for values in self.config_map.values():
                # Accesss self.settings_map and combine/compare entries with
                # self.config_map values to build new composite mappings that
                # depend on supplied interface value
            return interface_map
    
  • Marius Gedminas
    Marius Gedminas over 9 years
    Thank you! My question was different -- how do I mock with open(filename) as f: for line in f: ... -- and your answer helped me do it. Unfortunately mock_open doesn't do anything about iteration.
  • Michele d'Amico
    Michele d'Amico over 9 years
    @MariusGedminas try to switch to python3.4 (If you didn't already do) and use h.__iter__=h.readlines.side_effect where h are handlers in handlers. It should fix it ... maybe it is bug or they forget to set it. Here hg.python.org/cpython/file/3.4/Lib/unittest/mock.py#l2309 is the source code.
  • Marius Gedminas
    Marius Gedminas over 9 years
    I'm using mock from PyPI with various Pythons (2.6 to 3.4). It's possible mock_open.return_value.__iter__.side_effect = mock_open.return_value.readline.side_effect might work, but it seems more complicated compared to what I already do (which was inspired by the 1st part of your answer).
  • Michele d'Amico
    Michele d'Amico over 9 years
    @MariusGedminas Your solution is OK if you don't use both readlines() and iterator. I don't think there is a best solution for every case and in testing by mocks the first aim should be SIMPLICITY and your way is simple and clear for your case. My answer and comment were generic and focused on the core of the question that was about mocking two different files with different contents. If you would like to test a function that call two open where you need different contents you should use my approach with side_effects.
  • ron rothman
    ron rothman over 6 years
    Why do you patch __builtin__.open, only to throw away the resulting Mock object (by immediately redefining mock_open in the first line of the function)?
  • Martijn Pieters
    Martijn Pieters over 6 years
    @ron.rothmanℝℝ: good question, there is indeed no need to do both. I can't remember why I did that (it's been 3 years almost), so I just removed the second mock.
  • Scott Carpenter
    Scott Carpenter over 4 years
    Iteration mocking was fixed in Python 3.8 and backported to 3.7.
  • maxstrobel
    maxstrobel over 2 years
    I think, you could improve the readability even a bit more, if you create both mocks already inside the patch decorator. @patch('__builtin__.open', side_effect=[mock_open(read_data="aa\bb").return_value, mock_open(read_data="AA\nBB").return_value])