When should I mock?

65,330

Solution 1

A unit test should test a single codepath through a single method. When the execution of a method passes outside of that method, into another object, and back again, you have a dependency.

When you test that code path with the actual dependency, you are not unit testing; you are integration testing. While that's good and necessary, it isn't unit testing.

If your dependency is buggy, your test may be affected in such a way to return a false positive. For instance, you may pass the dependency an unexpected null, and the dependency may not throw on null as it is documented to do. Your test does not encounter a null argument exception as it should have, and the test passes.

Also, you may find its hard, if not impossible, to reliably get the dependent object to return exactly what you want during a test. That also includes throwing expected exceptions within tests.

A mock replaces that dependency. You set expectations on calls to the dependent object, set the exact return values it should give you to perform the test you want, and/or what exceptions to throw so that you can test your exception handling code. In this way you can test the unit in question easily.

TL;DR: Mock every dependency your unit test touches.

Solution 2

Mock objects are useful when you want to test interactions between a class under test and a particular interface.

For example, we want to test that method sendInvitations(MailServer mailServer) calls MailServer.createMessage() exactly once, and also calls MailServer.sendMessage(m) exactly once, and no other methods are called on the MailServer interface. This is when we can use mock objects.

With mock objects, instead of passing a real MailServerImpl, or a test TestMailServer, we can pass a mock implementation of the MailServer interface. Before we pass a mock MailServer, we "train" it, so that it knows what method calls to expect and what return values to return. At the end, the mock object asserts, that all expected methods were called as expected.

This sounds good in theory, but there are also some downsides.

Mock shortcomings

If you have a mock framework in place, you are tempted to use mock object every time you need to pass an interface to the class under the test. This way you end up testing interactions even when it is not necessary. Unfortunately, unwanted (accidental) testing of interactions is bad, because then you're testing that a particular requirement is implemented in a particular way, instead of that the implementation produced the required result.

Here's an example in pseudocode. Let's suppose we've created a MySorter class and we want to test it:

// the correct way of testing
testSort() {
    testList = [1, 7, 3, 8, 2] 
    MySorter.sort(testList)

    assert testList equals [1, 2, 3, 7, 8]
}


// incorrect, testing implementation
testSort() {
    testList = [1, 7, 3, 8, 2] 
    MySorter.sort(testList)

    assert that compare(1, 2) was called once 
    assert that compare(1, 3) was not called 
    assert that compare(2, 3) was called once 
    ....
}

(In this example we assume that it's not a particular sorting algorithm, such as quick sort, that we want to test; in that case, the latter test would actually be valid.)

In such an extreme example it's obvious why the latter example is wrong. When we change the implementation of MySorter, the first test does a great job of making sure we still sort correctly, which is the whole point of tests - they allow us to change the code safely. On the other hand, the latter test always breaks and it is actively harmful; it hinders refactoring.

Mocks as stubs

Mock frameworks often allow also less strict usage, where we don't have to specify exactly how many times methods should be called and what parameters are expected; they allow creating mock objects that are used as stubs.

Let's suppose we have a method sendInvitations(PdfFormatter pdfFormatter, MailServer mailServer) that we want to test. The PdfFormatter object can be used to create the invitation. Here's the test:

testInvitations() {
   // train as stub
   pdfFormatter = create mock of PdfFormatter
   let pdfFormatter.getCanvasWidth() returns 100
   let pdfFormatter.getCanvasHeight() returns 300
   let pdfFormatter.addText(x, y, text) returns true 
   let pdfFormatter.drawLine(line) does nothing

   // train as mock
   mailServer = create mock of MailServer
   expect mailServer.sendMail() called exactly once

   // do the test
   sendInvitations(pdfFormatter, mailServer)

   assert that all pdfFormatter expectations are met
   assert that all mailServer expectations are met
}

In this example, we don't really care about the PdfFormatter object so we just train it to quietly accept any call and return some sensible canned return values for all methods that sendInvitation() happens to call at this point. How did we come up with exactly this list of methods to train? We simply ran the test and kept adding the methods until the test passed. Notice, that we trained the stub to respond to a method without having a clue why it needs to call it, we simply added everything that the test complained about. We are happy, the test passes.

But what happens later, when we change sendInvitations(), or some other class that sendInvitations() uses, to create more fancy pdfs? Our test suddenly fails because now more methods of PdfFormatter are called and we didn't train our stub to expect them. And usually it's not only one test that fails in situations like this, it's any test that happens to use, directly or indirectly, the sendInvitations() method. We have to fix all those tests by adding more trainings. Also notice, that we can't remove methods no longer needed, because we don't know which of them are not needed. Again, it hinders refactoring.

Also, the readability of test suffered terribly, there's lots of code there that we didn't write because of we wanted to, but because we had to; it's not us who want that code there. Tests that use mock objects look very complex and are often difficult to read. The tests should help the reader understand, how the class under the test should be used, thus they should be simple and straightforward. If they are not readable, nobody is going to maintain them; in fact, it's easier to delete them than to maintain them.

How to fix that? Easily:

  • Try using real classes instead of mocks whenever possible. Use the real PdfFormatterImpl. If it's not possible, change the real classes to make it possible. Not being able to use a class in tests usually points to some problems with the class. Fixing the problems is a win-win situation - you fixed the class and you have a simpler test. On the other hand, not fixing it and using mocks is a no-win situation - you didn't fix the real class and you have more complex, less readable tests that hinder further refactorings.
  • Try creating a simple test implementation of the interface instead of mocking it in each test, and use this test class in all your tests. Create TestPdfFormatter that does nothing. That way you can change it once for all tests and your tests are not cluttered with lengthy setups where you train your stubs.

All in all, mock objects have their use, but when not used carefully, they often encourage bad practices, testing implementation details, hinder refactoring and produce difficult to read and difficult to maintain tests.

For some more details on shortcomings of mocks see also Mock Objects: Shortcomings and Use Cases.

Solution 3

Rule of thumb:

If the function you are testing needs a complicated object as a parameter, and it would be a pain to simply instantiate this object (if, for example it tries to establish a TCP connection), use a mock.

Solution 4

You should mock an object when you have a dependency in a unit of code you are trying to test that needs to be "just so".

For example, when you are trying to test some logic in your unit of code but you need to get something from another object and what is returned from this dependency might affect what you are trying to test - mock that object.

A great podcast on the topic can be found here

Share:
65,330

Related videos on Youtube

Eduardo Scoz
Author by

Eduardo Scoz

Code makes me happy.

Updated on January 03, 2021

Comments

  • Eduardo Scoz
    Eduardo Scoz about 3 years

    I have a basic understanding of mock and fake objects, but I'm not sure I have a feeling about when/where to use mocking - especially as it would apply to this scenario here.

    • Vladimir
      Vladimir almost 4 years
      I recommend only mocking out-of-process dependencies and only those of them, interactions with which are observable externally (SMTP server, message bus, etc). Don't mock the database, it's an implementation detail. More about it here: enterprisecraftsmanship.com/posts/when-to-mock
  • Rogério
    Rogério over 13 years
    This answer is too radical. Unit tests can and should exercise more than a single method, as long as it all belongs to the same cohesive unit. Doing otherwise would require way too much mocking/faking, leading to complicated and fragile tests. Only the dependencies that don't really belong to the unit under test should be replaced through mocking.
  • Jeff Axelrod
    Jeff Axelrod almost 13 years
    This answer is also too optimistic. It would be better if it incorporated @Jan's shortcomings of mock objects.
  • Draemon
    Draemon about 12 years
    Isn't this more of an argument for injecting dependencies for tests rather than mocks specifically? You could pretty much replace "mock" with "stub" in your answer. I agree that you should either mock or stub the significant dependencies. I've seen a lot of mock-heavy code which basically ends up reimplementing parts of the mocked objects; mocks certainly aren't a silver bullet.
  • Draemon
    Draemon about 12 years
    A well thought out answer, and I mostly agree. I would say that since unit tests are white-box testing, having to change the tests when you change the implementation to send fancier PDFs may not be an unreasonably burden. Sometimes mocks can be useful way to quickly implement stubs instead of having a lot of boiler plate. In practice it does seem that their use is not retricted to these simple cases, however.
  • PositiveGuy
    PositiveGuy almost 11 years
    isn't the whole point of a mock is that your tests are consistent, that you don't have to worry about mocking on objects whose implementations are continually changing possibly by other programmers each time you run your test and you get consistent test results?
  • Kemoda
    Kemoda almost 11 years
    Very good and relevant points (especially about tests fragility). I used to use mocks a lot when i was younger, but now i consider unit test that heavilly depend on mocks as potentially disposable and focus more on integration testing (with actual components)
  • Michael Freidgeim
    Michael Freidgeim over 10 years
    "Not being able to use a class in tests usually points to some problems with the class." If the class is a service(e.g. access to database or proxy to web service), it should be considered as an external dependancy and mocked/stubbed
  • Narendra Pathai
    Narendra Pathai over 9 years
    @MichaelFreidgeim Why mock database? Why not use an embedded database to test the same thing? That way you don't have separate unit and integration tests. Also your tests perform faster as required because the database is embedded.
  • weberc2
    weberc2 over 9 years
    @Rogério Not if you've got a well-designed system: destroyallsoftware.com/blog/2014/…
  • weberc2
    weberc2 over 9 years
    @Rogério Edit: I took you to mean that we should always instantiate dependencies (i.e., every unit test is also a potential integration test); perhaps you meant mocking as opposed to stubbing (some dependencies can be stubbed instead of mocked)?
  • Rogério
    Rogério over 9 years
    @weberc2 I prefer to use "mock" as a general term for both strict mocks and stubs, as there is often confusion and different understandings of them (for example, some think that a "mock" always requires all expectations to be recorded, which is only true for strict mocks). But the real point here is that often we have dependencies that fall inside the "unit" to be tested; personally, I like to create non-public helper classes in the same package (Java) whenever the public class becomes too big or complex; such dependencies are really internal details, and I don't unit test them separately.
  • weberc2
    weberc2 over 9 years
    @Rogério I think you're confusing unit with class. In this case, your private helper class belongs in the same unit as the class it helps. It isn't an external dependency. This example doesn't contradict the answer, it supports it.
  • Rogério
    Rogério over 9 years
    @weberc2 Your answer only says "dependency", without specifying whether it's internal or external to the unit under test. So, I assumed it meant any dependency of the unit. If it in fact only means external dependency, then we are in agreement. But the answer is confusing when it says "a single codepath through a single method", as we will obviously have several codepaths through several methods, all inside the unit.
  • weberc2
    weberc2 over 9 years
    @Rogério it's not my answer. More importantly, as far as I'm concerned, dependencies can only be external to the unit. What you're calling an "internal dependency" isn't a dependency of your unit, it's part of your unit. Thus, the answer doesn't conflict with your scenario.
  • Teoman shipahi
    Teoman shipahi about 9 years
    Mock every dependency your unit test touches. This explains everything.
  • kayleeFrye_onDeck
    kayleeFrye_onDeck about 8 years
    I guess this is one accepted unit testing philosophy... A dubious one (or at least a limited scope), but hey, knock yourself out. What's the worst that could happen? :)
  • vibhu
    vibhu over 7 years
    Very cleanly expressed the dependency a Mock potentially creates with SUT.
  • Wes
    Wes about 7 years
    it's maybe a monumentally annoying task doing unit testing this way, but it's for sure the best possible approach to it
  • Christopher Will
    Christopher Will almost 6 years
    But what happens later, when we change sendInvitations()? If code under test gets modified it does not guarantee the previous contract anymore, hence it has to fail. And usually it's not only one test that fails in situations like this. If this is the case, the code is not clean implemented. The verification of method-calls of the dependency should be tested only once (in the appropriate unit test). All other classes will use the mock instance only. So I don't see any benefits mixing integration- with unit-tests.
  • Praveen Tiwari
    Praveen Tiwari about 5 years
    TL;DR: Mock every dependency your unit test touches. - this is not really a great approach, mockito itself says - don't mock everything. (downvoted)
  • Bob Ray
    Bob Ray about 4 years
    I think the key phrase some people have missed in this answer is: "When the execution of a method passes outside of that method, into another object, and back again, you have a dependency" It's not saying "into another method in the same class." I agree that when another object is involved, it's an integration test, not a unit test.
  • C Perkins
    C Perkins almost 4 years
    The link now routes to the current episode, not the intended episode. Is the intended podcast this hanselminutes.com/32/mock-objects?
  • user1415536
    user1415536 over 3 years
    I do not want to create a new question. I will ask here. Imagine a call to the mocked object's method returns some wrapper class with a basic setter and getter. Also, imagine this wrapper is defined in another library. Will you still create a mock for this wrapper class?
  • Traycho Ivanov
    Traycho Ivanov over 3 years
    Mocking everything around bring again the effect returning false positives. Mocking also get rid of many code interactions where your application could fail runtime in PROD. Cascade mocking is also hard to maintain.
  • ggorlen
    ggorlen about 3 years
    @ChristopherWill I think the author's point is that overusing mocks tends to lure developers into testing overly-rigid or outright false contracts based on irrelevant implementation details. Jan points out that in cases when these details are genuinely important (as in the sort compare test), then that's a situation where mocks make sense.
  • Leponzo
    Leponzo about 3 years
  • Traycho Ivanov
    Traycho Ivanov almost 3 years
    @Leponzo Sounds good but practice tell us different stories. Mocking every depndency means you need to know how the dependency works especially in case it is nested one. Having mocking all your code and relying only on these tests raises a lot of unexpected issues on PROD. There should be a balance and an article could not explain it in details. I am not fan of mocking everything.
  • Leponzo
    Leponzo almost 3 years
    @TraychoIvanov Yes, that's why you need both integration and unit tests. The article just says that your unit tests should mock all dependencies.
  • Traycho Ivanov
    Traycho Ivanov almost 3 years
    @Leponzo that is true, but I had impression you haven't worked on bigger codebase where mocking eveyrhing is impossible as dependencies have multiple levels of nesting and mocking them sounds as very bad thing specisally when they are refactored regulary. Theory and practice are two very different things. There are many articles not supporting mock eveyrhing. I would advise every case to be handled seprately there is no golden rule. techyourchance.com/mocks-in-unit-testing , jtway.co/mock-everything-is-a-good-way-to-sink-b8a1284fb81f