Unexpected exceptions in Java Unit tests

Question

I'm really a newbie to JUnit and unit testing in general and I'm struggling to find the right approach. What is the better way to deal with unexpected exceptions, and why?

Method A:

1. first catch the expected ones, fail the test with a message
2. in the last catch block, catch a general Exception and fail the test with some "Unexpected exception has occured" message

Method B:

1. catch only the expected ones, fail with a message
2. mark the test method with throws Exception and let anything unexpected "bubble up" completely out of the test

And to add to the confusion, by saying "unexpected exception", I mean either one of these things:

1. an exception that the method being tested claims to throw, but on in this case. For example, my method throws IllegalArgumentException and NullPointerException, but in this try block, I expect it to throw IllegalArgument; NullPointer is considered unexpected. Catch and fail or bubble up?
2. a general Exception I really cannot anticipate

I'm aware this question comes out a bit confusing, I'm getting lost in it myself, but hopefully someone will give me any kind of hint. Won't blame you for downvotes, it's still worth the risk :)

Solution 1

With a test method of JUnit, basically, if the method manages to end without error, JUnit considers the test passed. So what you should do is to catch all exceptions that should arrives. In that case, here's what you can do :

@Test
public myMethodToTest()
{
    try {
        /* ... */
        /* error should trigger before the following statement : */
        fail("Reason");
    } catch(GoodException e) { } 
   // no need to do anything since you expected that error
   // You can let the test ends without further statement 
}

There is another way to write it :

@Test (expected=IndexOutOfBoundsException.class)
public void elementAt()
{
    int[] intArray = new int[10];

    int i = intArray[20]; // Should throw IndexOutOfBoundsException
}

If you still want, you can still catch exceptions that shouldn't trigger, and do something like that :

@Test
public myMethodToTest()
{
    try {
        /* ... */
    } catch(BadException e) {
        fail("Reason");
    }

}

Solution 2

You can use this syntax see SO question (Asserting exceptions in Java, how?)

@Test(expected=NumberFormatException.class);

to ensure that an expected Exception might be thrown.

Some methods may legitimately throw more than one exception. For example if you are solving a quadratic equation you can expect both divide-by-zero and negative-square-root. You should test each of these by writing two separate unit tests each of which is designed to fail and so test that the exceptions are thrown.

Example:

Pair roots = solveQuadratic(double a, double b, double c) {
    //... 
}

where Pair is the expected pair of solutions.

Here's a test we expect to pass.

@Test
public void testQuadratic() {
    Pair roots = solveQuadratic(1.0, -5.0, 6.0);
    Assert.assertEqual(3.0, roots.largest, 0.000001); 
    Assert.assertEqual(2.0, roots.smallest, 0.000001); 
}

Here is one we expect to fail:

@Test
public void testQuadraticDivideByZero() {
    try {
        Pair roots = solveQuadratic(0.0, -5.0, 6.0);
        Assert.fail("should have failed with ArithmeticException");
    } catch (ArithmeticException e) {
    }
}

in that case you might want to write the code so that you trapped a=0.0 and throw IllegalArgumentException with a message.

In general you should think of all the things that could conceivably go wrong and trap them and test the traps. Because some day someone will pass a=0.0 to this method and needs a clear error message.

Solution 3

I got the feeling, none of the answers so far really got to the point of the question. The OP explicitly asks for the handling of unexpected exceptions. My two cents on this topic are:

It depends on the level of verbosity you want to achieve:

Usually, you should strive for short tests which pinpoint one aspect of the code. Ideally, only a handful of methods need to be called and only one or two of them may raise an unexpected exception. In this case, adding a throws-clause for all checked exceptions should be sufficient to analyze the problem when you test fails. I prefer this solution because it's easier to write, shorter and more comprehensive.

Example:

@Test
public void testPrivateMethod() throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {

    //...

    Method method = MyClass.class.getMethod("privateMethod");
    method.setAccessible(true);
    boolean result = method.invoke(myInstance);
    assertTrue(result);
}

If the test code needs to be more complicated multiple methods may be responsible for raising an exception. Probably, some of them even throw exceptions of the same kind. In this case, try-catch blocks may be beneficial for locating the problem when your test case fails. However, this produces more code and may render the tests less readable.

Example:

@Test
public void testPrivateMethod() {

    //...

    Method method;
    try {
        Method method = MyClass.class.getMethod("privateMethod");
        method.setAccessible(true);
        boolean result = method.invoke(myInstance);
        assertTrue(result);
    } catch (NoSuchMethodException | SecurityException e) {
        fail("Could not access method 'privateMethod'.");
    } catch (IllegalAccessException | IllegalArgumentException | InvocationTargetException e) {
        fail("Call to 'privateMethod' raised an exception.")
    }
}

I hope I got the intention of the question right.

Solution 4

The idea to UNIT test a method is to make sure it is a reliable method. Reliability means that for a given input, output is always the expected one. So it depends what is expected out of the method whether there is an exception or not. If we believe that an unexpected exception has happened and method was expected to do nothing about it, then you can mark the test case as PASSED. But if the method is expected to handle any exception scenario and if it not able to, then mark the test case as FAILED.

Solution 5

The idea of having a test-case is to avoid all unexpected exceptions, by feeding it with the inputs for all the boundary conditions.

If the method throws some Exceptions (the method has throws clause and its trying to notify caller with some checked exception), pass the input which makes your testcase throw that Exception, and catch it in your test-case. This makes one of the test-case PASS.

If the method is throwing unexpected Exceptions, fix the method. A method should be stable enough, not to throw unexpected exceptions, like NullPointerException and should be fixed.

So to answer your question, a better way to deal with unexpected Exception is to find them by enhancing your test-case it fix it.

View more solutions

Comments

[JB Nizet]

The first test would pass even if GoodException is not thrown. It lacks a fail() call at the end of the try block. The second test could simply let the exception propagate and JUnit would mark it failed (in error) is the exception is thrown.

[Tom Anderson]

I strongly prefer the former style, where you catch the expected exception exactly where it is thrown. The problem with using the expected attribute of the @Test annotation is that if the right exception is thrown from the wrong place in the code (in some setup code, say), then the test will pass, even though the code is doing the wrong thing. I have seen this happen several times in real codebases.