assertAll vs multiple assertions in JUnit5

Question

Is there any reason to group multiple assertions:

public void shouldTellIfPrime(){
    Assertions.assertAll(
            () -> assertTrue(isPrime(2)),
            () -> assertFalse(isPrime(4))
    );
}

instead of doing this:

public void shouldTellIfPrime(){
    Assertions.assertTrue(isPrime(2));
    Assertions.assertFalse(isPrime(4));
}

Solution 1

The interesting thing about assertAll is that it always checks all of the assertions that are passed to it, no matter how many fail. If all pass, all is fine - if at least one fails you get a detailed result of all that went wrong (and right for that matter).

It is best used for asserting a set of properties that belong together conceptionally. Something where your first instinct would be, "I want to assert this as one".

Example

Your specific example is not an optimal use case for assertAll because checking isPrime with a prime and a non-prime is independent of each other - so much so that I would recommend writing two test methods for that.

But assume you have a simple class like an address with fields city, street, number and would like to assert that those are what you expect them to be:

Address address = unitUnderTest.methodUnderTest();
assertEquals("Redwood Shores", address.getCity());
assertEquals("Oracle Parkway", address.getStreet());
assertEquals("500", address.getNumber());

Now, as soon as the first assertion fails, you will never see the results of the second, which can be quite annoying. There are many ways around this and JUnit Jupiter's assertAll is one of them:

Address address = unitUnderTest.methodUnderTest();
assertAll("Should return address of Oracle's headquarter",
    () -> assertEquals("Redwood Shores", address.getCity()),
    () -> assertEquals("Oracle Parkway", address.getStreet()),
    () -> assertEquals("500", address.getNumber())
);

If the method under test returns the wrong address, this is the error you get:

org.opentest4j.MultipleFailuresError:
    Should return address of Oracle's headquarter (3 failures)
    expected: <Redwood Shores> but was: <Walldorf>
    expected: <Oracle Parkway> but was: <Dietmar-Hopp-Allee>
    expected: <500> but was: <16>

Solution 2

According to documentation here

Asserts that all supplied executables do not throw an AssertionError.

If any supplied Executable throws an AssertionError, all remaining executables will still be executed, and all failures will be aggregated and reported in a MultipleFailuresError. However, if an executable throws an exception that is not an AssertionError, execution will halt immediately, and the exception will be rethrown as is but masked as an unchecked exception.

So main difference is that the assertAll will allow all the asserts to execute without breaking the flow while the others like assertTrue and the lot will stop the test with the AssertionError

So in your first example both assertions will execute regardless of pass to fail, while in the second example test will stop if first assertion fails.

Is there any reason to group multiple assertions

If you want all assertions exercised in the unit test.

Comments

[Timothy Truckle]

But do not abuse it! A single test method should always test only one assumption about the production code. That's the main reason why you usually have only one assert per test method.

[Nicolai Parlog]

I agree with not abusing it and testing only one assumption, but disagree with there being any value in counting assertions. That is a purely syntactical consideration without any relevance. Take my example: Chances are Address:equals tests exactly these properties, in which case I could verify them with one assertion. Logically there would be no difference at all but suddenly it's "only one assert". The same is true if I go through the pain to create a Hamcrest matcher for the class.

[Timothy Truckle]

"but disagree with there being any value in counting assertions" I did not suggest to "count" assertions. one assert per test method is a rule of thumb, nothing more but nothing less... Anyway, if you have multiple asserts should ask yourself if you test really tests a single assumption.

[Fr Jeremy Krieg]

I don't fully agree with the "one test, one assertion" rule of thumb. It assumes that the code-under-test is quick & simple to run. As your tests scale from low-level unit tests to high-level integration tests, this assumption doesn't hold. It is much more efficient to run an expensive piece of code once and run several inexpensive assertions on the results, rather than running the expensive code several times and testing one thing each time. Assertions at intermediate points can also help with debugging. As long as you clearly label assertions there is no problem in using multiple.