patching a class yields "AttributeError: Mock object has no attribute" when accessing instance attributes

Question

The Problem
Using mock.patch with autospec=True to patch a class is not preserving attributes of instances of that class.

The Details
I am trying to test a class Bar that instantiates an instance of class Foo as a Bar object attribute called foo. The Bar method under test is called bar; it calls method foo of the Foo instance belonging to Bar. In testing this, I am mocking Foo, as I only want to test that Bar is accessing the correct Foo member:

import unittest
from mock import patch

class Foo(object):
    def __init__(self):
        self.foo = 'foo'

class Bar(object):
    def __init__(self):
        self.foo = Foo()

    def bar(self):
        return self.foo.foo

class TestBar(unittest.TestCase):
    @patch('foo.Foo', autospec=True)
    def test_patched(self, mock_Foo):
        Bar().bar()

    def test_unpatched(self):
        assert Bar().bar() == 'foo'

The classes and methods work just fine (test_unpatched passes), but when I try to Foo in a test case (tested using both nosetests and pytest) using autospec=True, I encounter "AttributeError: Mock object has no attribute 'foo'"

19:39 $ nosetests -sv foo.py
test_patched (foo.TestBar) ... ERROR
test_unpatched (foo.TestBar) ... ok

======================================================================
ERROR: test_patched (foo.TestBar)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1201, in patched
    return func(*args, **keywargs)
  File "/home/vagrant/dev/constellation/test/foo.py", line 19, in test_patched
    Bar().bar()
  File "/home/vagrant/dev/constellation/test/foo.py", line 14, in bar
    return self.foo.foo
  File "/usr/local/lib/python2.7/dist-packages/mock.py", line 658, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'foo'

Indeed, when I print out mock_Foo.return_value.__dict__, I can see that foo is not in the list of children or methods:

{'_mock_call_args': None,
 '_mock_call_args_list': [],
 '_mock_call_count': 0,
 '_mock_called': False,
 '_mock_children': {},
 '_mock_delegate': None,
 '_mock_methods': ['__class__',
                   '__delattr__',
                   '__dict__',
                   '__doc__',
                   '__format__',
                   '__getattribute__',
                   '__hash__',
                   '__init__',
                   '__module__',
                   '__new__',
                   '__reduce__',
                   '__reduce_ex__',
                   '__repr__',
                   '__setattr__',
                   '__sizeof__',
                   '__str__',
                   '__subclasshook__',
                   '__weakref__'],
 '_mock_mock_calls': [],
 '_mock_name': '()',
 '_mock_new_name': '()',
 '_mock_new_parent': <MagicMock name='Foo' spec='Foo' id='38485392'>,
 '_mock_parent': <MagicMock name='Foo' spec='Foo' id='38485392'>,
 '_mock_wraps': None,
 '_spec_class': <class 'foo.Foo'>,
 '_spec_set': None,
 'method_calls': []}

My understanding of autospec is that, if True, the patch specs should apply recursively. Since foo is indeed an attribute of Foo instances, should it not be patched? If not, how do I get the Foo mock to preserve the attributes of Foo instances?

NOTE:
This is a trivial example that shows the basic problem. In reality, I am mocking a third party module.Class -- consul.Consul -- whose client I instantiate in a Consul wrapper class that I have. As I don't maintain the consul module, I can't modify the source to suit my tests (I wouldn't really want to do that anyway). For what it's worth, consul.Consul() returns a consul client, which has an attribute kv -- an instance of consul.Consul.KV. kv has a method get, which I am wrapping in an instance method get_key in my Consul class. After patching consul.Consul, the call to get fails because of AttributeError: Mock object has no attribute kv.

Resources Already Checked:

http://mock.readthedocs.org/en/latest/helpers.html#autospeccing http://mock.readthedocs.org/en/latest/patch.html

Solution

No, autospeccing cannot mock out attributes set in the __init__ method of the original class (or in any other method). It can only mock out static attributes, everything that can be found on the class.

Otherwise, the mock would have to create an instance of the class you tried to replace with a mock in the first place, which is not a good idea (think classes that create a lot of real resources when instantiated).

The recursive nature of an auto-specced mock is then limited to those static attributes; if foo is a class attribute, accessing Foo().foo will return an auto-specced mock for that attribute. If you have a class Spam whose eggs attribute is an object of type Ham, then the mock of Spam.eggs will be an auto-specced mock of the Ham class.

The documentation you read explicitly covers this:

A more serious problem is that it is common for instance attributes to be created in the __init__ method and not to exist on the class at all. autospec can’t know about any dynamically created attributes and restricts the api to visible attributes.

You should just set the missing attributes yourself:

@patch('foo.Foo', autospec=Foo)
def test_patched(self, mock_Foo):
    mock_Foo.return_value.foo = 'foo'
    Bar().bar()

or create a subclass of your Foo class for testing purposes that adds the attribute as a class attribute:

class TestFoo(foo.Foo):
    foo = 'foo'  # class attribute

@patch('foo.Foo', autospec=TestFoo)
def test_patched(self, mock_Foo):
    Bar().bar()