What's the difference between faking, mocking, and stubbing?

Question

I know how I use these terms, but I'm wondering if there are accepted definitions for faking, mocking, and stubbing for unit tests? How do you define these for your tests? Describe situations where you might use each.

Here is how I use them:

Fake: a class that implements an interface but contains fixed data and no logic. Simply returns "good" or "bad" data depending on the implementation.

Mock: a class that implements an interface and allows the ability to dynamically set the values to return/exceptions to throw from particular methods and provides the ability to check if particular methods have been called/not called.

Stub: Like a mock class, except that it doesn't provide the ability to verify that methods have been called/not called.

Mocks and stubs can be hand generated or generated by a mocking framework. Fake classes are generated by hand. I use mocks primarily to verify interactions between my class and dependent classes. I use stubs once I have verified the interactions and am testing alternate paths through my code. I use fake classes primarily to abstract out data dependencies or when mocks/stubs are too tedious to set up each time.

Solution 1

You can get some information :

From Martin Fowler about Mock and Stub

Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.

Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

From xunitpattern:

Fake: We acquire or build a very lightweight implementation of the same functionality as provided by a component that the SUT depends on and instruct the SUT to use it instead of the real.

Stub : This implementation is configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested Code (see Production Bugs on page X) within the SUT. A key indication for using a Test Stub is having Untested Code caused by the inability to control the indirect inputs of the SUT

Mock Object that implements the same interface as an object on which the SUT (System Under Test) depends. We can use a Mock Object as an observation point when we need to do Behavior Verification to avoid having an Untested Requirement (see Production Bugs on page X) caused by an inability to observe side-effects of invoking methods on the SUT.

Personally

I try to simplify by using : Mock and Stub. I use Mock when it's an object that returns a value that is set to the tested class. I use Stub to mimic an Interface or Abstract class to be tested. In fact, it doesn't really matter what you call it, they are all classes that aren't used in production, and are used as utility classes for testing.

Solution 2

Stub - an object that provides predefined answers to method calls.

Mock - an object on which you set expectations.

Fake - an object with limited capabilities (for the purposes of testing), e.g. a fake web service.

Test Double is the general term for stubs, mocks and fakes. But informally, you'll often hear people simply call them mocks.

Solution 3

I am surprised that this question has been around for so long and nobody has as yet provided an answer based on Roy Osherove's "The Art of Unit Testing".

In "3.1 Introducing stubs" defines a stub as:

A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly.

And defines the difference between stubs and mocks as:

The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub.

Fake is just the name used for both stubs and mocks. For example when you don't care about the distinction between stubs and mocks.

The way Osherove's distinguishes between stubs and mocks, means that any class used as a fake for testing can be both a stub or a mock. Which it is for a specific test depends entirely on how you write the checks in your test.

• When your test checks values in the class under test, or actually anywhere but the fake, the fake was used as a stub. It just provided values for the class under test to use, either directly through values returned by calls on it or indirectly through causing side effects (in some state) as a result of calls on it.
• When your test checks values of the fake, it was used as a mock.

Example of a test where class FakeX is used as a stub:

const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);

cut.SquareIt;

Assert.AreEqual(25, cut.SomeProperty);

The fake instance is used as a stub because the Assert doesn't use fake at all.

Example of a test where test class X is used as a mock:

const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);

cut.SquareIt;

Assert.AreEqual(25, fake.SomeProperty);

In this case the Assert checks a value on fake, making that fake a mock.

Now, of course these examples are highly contrived, but I see great merit in this distinction. It makes you aware of how you are testing your stuff and where the dependencies of your test are.

I agree with Osherove's that

from a pure maintainability perspective, in my tests using mocks creates more trouble than not using them. That has been my experience, but I’m always learning something new.

Asserting against the fake is something you really want to avoid as it makes your tests highly dependent upon the implementation of a class that isn't the one under test at all. Which means that the tests for class ActualClassUnderTest can start breaking because the implementation for ClassUsedAsMock changed. And that sends up a foul smell to me. Tests for ActualClassUnderTest should preferably only break when ActualClassUnderTest is changed.

I realize that writing asserts against the fake is a common practice, especially when you are a mockist type of TDD subscriber. I guess I am firmly with Martin Fowler in the classicist camp (See Martin Fowler's "Mocks aren't Stubs") and like Osherove avoid interaction testing (which can only be done by asserting against the fake) as much as possible.

For fun reading on why you should avoid mocks as defined here, google for "fowler mockist classicist". You'll find a plethora of opinions.

Solution 4

As mentioned by the top-voted answer, Martin Fowler discusses these distinctions in Mocks Aren't Stubs, and in particular the subheading The Difference Between Mocks and Stubs, so make sure to read that article.

Rather than focusing on how these things are different, I think it's more enlightening to focus on why these are distinct concepts. Each exists for a different purpose.

Fakes

A fake is an implementation that behaves "naturally", but is not "real". These are fuzzy concepts and so different people have different understandings of what makes things a fake.

One example of a fake is an in-memory database (e.g. using sqlite with the :memory: store). You would never use this for production (since the data is not persisted), but it's perfectly adequate as a database to use in a testing environment. It's also much more lightweight than a "real" database.

As another example, perhaps you use some kind of object store (e.g. Amazon S3) in production, but in a test you can simply save objects to files on disk; then your "save to disk" implementation would be a fake. (Or you could even fake the "save to disk" operation by using an in-memory filesystem instead.)

As a third example, imagine an object that provides a cache API; an object that implements the correct interface but that simply performs no caching at all but always returns a cache miss would be a kind of fake.

The purpose of a fake is not to affect the behavior of the system under test, but rather to simplify the implementation of the test (by removing unnecessary or heavyweight dependencies).

Stubs

A stub is an implementation that behaves "unnaturally". It is preconfigured (usually by the test set-up) to respond to specific inputs with specific outputs.

The purpose of a stub is to get your system under test into a specific state. For example, if you are writing a test for some code that interacts with a REST API, you could stub out the REST API with an API that always returns a canned response, or that responds to an API request with a specific error. This way you could write tests that make assertions about how the system reacts to these states; for example, testing the response your users get if the API returns a 404 error.

A stub is usually implemented to only respond to the exact interactions you've told it to respond to. But the key feature that makes something a stub is its purpose: a stub is all about setting up your test case.

Mocks

A mock is similar to a stub, but with verification added in. The purpose of a mock is to make assertions about how your system under test interacted with the dependency.

For example, if you are writing a test for a system that uploads files to a website, you could build a mock that accepts a file and that you can use to assert that the uploaded file was correct. Or, on a smaller scale, it's common to use a mock of an object to verify that the system under test calls specific methods of the mocked object.

Mocks are tied to interaction testing, which is a specific testing methodology. People who prefer to test system state rather than system interactions will use mocks sparingly if at all.

Test doubles

Fakes, stubs, and mocks all belong to the category of test doubles. A test double is any object or system you use in a test instead of something else. Most automated software testing involves the use of test doubles of some kind or another. Some other kinds of test doubles include dummy values, spies, and I/O blackholes.

Solution 5

The thing that you assert on it is called a mock object.

Everything else that just helped the test run is a stub.

View more solutions

Comments

[Simon Tewsi]

It seems to me the definitions for Stub and Fake are reversed in the xUnitPattern quote compared to Martin Fowler's quote. Also, that Martin Fowler's definitions of Stub and Fake are reversed compared to the definitions in tvanfosson's original question. In reality is there any generally accepted definitions of those two terms or does it just depend on who you're talking to?

[MasterMastic]

Could anybody explain & define to me what is a "canned answer" in this context?

[eugenio calabrese]

An explicit value, rather than a value that is calculated.

[Brad Cupit]

+1 for "I try to simplify by using : Mock and Stub". That's a great idea!

[Hector Ordonez]

Can't see how using only Mock and Stub is a great idea. Every test double has its purposes and, thus, its uses.

[IdontCareAboutReputationPoints]

I can't see the difference between Fake and Mock in MF's definition.

[dbalakirev]

@MusuNaji: In MF's definition there are no "expectations" in regarding the conversation for a Fake, other than it has an implementation for it's interface. On the other hand the Mock will be challenged (was this method called?).

[Mario Garcia]

while other answers have great detail and are really good. this one makes it so clear and easy to make the difference, it's hard to not upvote. gj!

[Brill Pappin]

Personally, i don't encourage the use of Mocks. I've see far too many tests that don't test anything. Mocks allow bad smells in the code, by allowing the tests to circumvent problems, instead of having the developer fix the issue. Not they are not useful, but more often than not, i see them associated with poor tests.

[Shyam]

Really well detailed and crisp defintion.

[Gabriel Staples]

Finally! Some definitions I can understand! Based on these definitions, then, googletest (gtest) / googlemock (gmock) allows mocked objects to also be stubs, as you can create EXPECT_CALL()s on a mocked method which force certain outputs based on certain inputs, by using the .WillOnce(Invoke(my_func_or_lambda_func)) (or with .WillRepeatedly()) type syntax attached to an EXPECT_CALL(). Some examples of using Invoke() can be seen in a different context at the bottom of my long answer here: stackoverflow.com/a/60905880/4561887.

[Gabriel Staples]

Gmock documentation on Invoke() is here: github.com/google/googletest/blob/master/googlemock/docs/…. Anyway, the conclusion is: Google mock (gmock) allows one to easily create both mocks and stubs, though most mocks are not stubs.

[Luke]

@Brill Pappin how do you approach integration tests or e2e without Mocks?

[Luke]

Mocks are a superset of Stubs, they can still return predefined answers but also allow the developer to set expectations. IMO certain libraries out there blur the lines of all of the testing dummies.

[Brill Pappin]

@Luke we're talking Unit tests here, which are different. An integration test should have real system behind it. However, if you can use a Mock in your integration test, then you can use a fake or a stub (depending on your definition). if you have to use a Mock, then you are probably testing the mock, and not the integration. However, one place Mocks really do come in handy, is when you have a poorly written api that you can't easily test. In those cases, Mocks can be a lifesaver. IMO of course.

[IceFire]

I don't like this terminology. This "setting an expectation on mocks" is either a wrong abstraction or too specialised. You could just straight-forwardly create a mock object, exercise some function on it and compare the result to some well-known result. "Setting an expectation" is never explicitly done that way. In fact, in many frameworks, the expectation is not seen as part of the mock

[Basil Musa]

I've read all the answers. I think this is the best clarification so far.

[eugenio calabrese]

@IceFire - perhaps a clearer way of saying it would be "an object on which you verify interactions". My wording/experience was from using site.mockito.org (10 years ago)

[Utkarsh Tiwari]

If anyone is wondering what SUT is, it's "System Under Test".