Set JUnit timeout in eclipse

Question

Question

When I run all our JUnit tests, using eclipse, can I set a default timeout?

Background

My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete. When I try to run our entire test suite (only about 300 tests) it can take over 30 minutes. I want to put something in place that will stop any test that takes longer than 10 seconds.

I know an individual test can be annotated with:

@Test(timeout=10000)

But doing this would make his long tests always fail. I want them to work when he runs them on his box (if I have to make minor adjustments to the project before checking it in, that's acceptable. However, deleting the timeouts from 40 different test files is not practical).

I also know I can create an ant task to set a default timeout for all tests, along the lines of:

<junit timeout="10000">
  ...
</junit>

The problem with that we typically run our tests from inside eclipse with Right Click > Run As > JUnit Test.

Summary

So is there a relatively painless way to set a timeout for all tests, perhaps using a Run Configuration setting, or project setting, or JUnit preference, or environment variable, or something? I'd even settle for installing some other plugin that lets me right click on particular test folders and run all the tests in some other manner like through ant or something...

Solution 1

Possible solution: Extend all your Test classes from another class: TestBase for example

Add to TestBase global timeout. This timeout will be applied to all extended classes:

public class TestBase {
    @Rule
    public Timeout globalTimeout = new Timeout(10000);
}

Solution 2

If you want to configure the tests to run for a maximum of ten seconds you can try this:

@Test(timeout=10000)

Solution 3

My manager insists on writing Unit tests that sometimes take up to 5 minutes to complete

This almost certainly indicates that those tests are not in fact unit tests. Cut that Gordian knot: try refactoring your testsuite to provide equivalent test coverage without requiring a test-case that runs for that long.

Solution 4

So maybe a combination of using Infinitest with the "Slow test warning" enabled together with the filtering feature would do the trick. You could identify tests that exceed your time-limit and add them to the filter list, this would only affect testing from inside Eclipse. Running the tests via a possible build script via CLI/CI etc would not be affected at all. You can find more on setting this up here: http://improvingworks.com/products/infinitest/infinitest-user-guide/

View more solutions

Comments

[gMale]

thanks for taking the time to answer but you are preaching to the choir ;) I agree with you 100% but I cannot make such changes. So my question is more about neutralizing the effect of crappy unit tests.

[brain]

@gmale: We've all been there mate but if you don't keep on at your boss nothing will change. Had a quick look at the Ant source code and the timeout they apply isn't a JUnit thing. You might want to use @Ignore for the specific tests you don't want to run instead of an arbitrary timeout. One thing you could do is edit the JUnit source code to set a default timeout. You'd have to change which JUnit library you use.

[gMale]

Sadly, they are unit tests. He verifies that a low-level function times out after 5 minutes of failing. :( That function has like 3 lines of code.

[Raedwald]

Using a time-out requires a clock. The real code will use the system clock, but your unit tests need not. Abstract the clock to be an object and use dependency injection to remove the direct dependency on the system clock. That is, instead of directly calling the system clock function, introduce a Clock interface (or abstract base class) passed to the time-out code. For the unit tests use a MockClock (that IS-A Clock) that simulates a time-out. For the real, integrated, code use a SystemClock (that IS-A Clock) that calls the system clock function.

[gMale]

Although he's not using a clock, I see your point. In this case, the timeout is actually based on a failed library call (that ultimately times out during I/O). It would be nice if he mocked that call but he doesn't. I can't really change all his tests everywhere, using mocks where I see fit. So I was looking for a way to just set a timeout for all tests. These days, we're back to using ant so I could just use the timeout built into the junit task, as I mentioned in the original post. Ant is great.

[gMale]

Thanks but this solution won't work for the situation described in the original question: doing this would make his long tests always fail. I want them to work when he runs them on his box . . . deleting the timeouts from 40 different test files [before checkin] is not practical

[gMale]

This is a great suggestion. Something that could could definitely be used in other situations. But in this case, it would require changing all his tests, which wasn't an option.

[gMale]

This looks fantastic! Pretty much exactly what I was looking for and something I could use often, in other situations. I'll try this out and if it works, I'm sold...

[рüффп]

Nice trick, but is there any way to override that value for a specific test (e.g only one) extending that class?

[alaster]

Sure. You can use @Test(timeout=10) on specific method in test class

[byxor]

This would be more appropriate as a comment.