How to run test methods in specific order in JUnit4?

Question

I want to execute test methods which are annotated by @Test in specific order.

For example:

public class MyTest {
    @Test public void test1(){}
    @Test public void test2(){}
}

I want to ensure to run test1() before test2() each time I run MyTest, but I couldn't find annotation like @Test(order=xx).

I think it's quite important feature for JUnit, if author of JUnit doesn't want the order feature, why?

Solution 1

I think it's quite important feature for JUnit, if author of JUnit doesn't want the order feature, why?

I'm not sure there is a clean way to do this with JUnit, to my knowledge JUnit assumes that all tests can be performed in an arbitrary order. From the FAQ:

How do I use a test fixture?

(...) The ordering of test-method invocations is not guaranteed, so testOneItemCollection() might be executed before testEmptyCollection(). (...)

Why is it so? Well, I believe that making tests order dependent is a practice that the authors don't want to promote. Tests should be independent, they shouldn't be coupled and violating this will make things harder to maintain, will break the ability to run tests individually (obviously), etc.

That being said, if you really want to go in this direction, consider using TestNG since it supports running tests methods in any arbitrary order natively (and things like specifying that methods depends on groups of methods). Cedric Beust explains how to do this in order of execution of tests in testng.

Solution 2

If you get rid of your existing instance of Junit, and download JUnit 4.11 or greater in the build path, the following code will execute the test methods in the order of their names, sorted in ascending order:

import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SampleTest {

    @Test
    public void testAcreate() {
        System.out.println("first");
    }
    @Test
    public void testBupdate() {
        System.out.println("second");
    }
    @Test
    public void testCdelete() {
        System.out.println("third");
    }
}

Solution 3

If the order is important, you should make the order yourself.

@Test public void test1() { ... }
@Test public void test2() { test1(); ... }

In particular, you should list some or all possible order permutations to test, if necessary.

For example,

void test1(); 
void test2(); 
void test3(); 


@Test
public void testOrder1() { test1(); test3(); }

@Test(expected = Exception.class)
public void testOrder2() { test2(); test3(); test1(); }

@Test(expected = NullPointerException.class)
public void testOrder3() { test3(); test1(); test2(); }

Or, a full test of all permutations:

@Test
public void testAllOrders() {
    for (Object[] sample: permute(1, 2, 3)) {
        for (Object index: sample) {
            switch (((Integer) index).intValue()) {
                case 1: test1(); break; 
                case 2: test2(); break; 
                case 3: test3(); break; 
            }
        }
    }
}

Here, permute() is a simple function which iterates all possible permuations into a Collection of array.

Solution 4

Migration to TestNG seems the best way, but I see no clear solution here for jUnit. Here is most readable solution / formatting I found for jUnit:

@FixMethodOrder( MethodSorters.NAME_ASCENDING ) // force name ordering
public class SampleTest {
    @Test
    void stage1_prepareAndTest(){};

    @Test
    void stage2_checkSomething(){};

    @Test
    void stage2_checkSomethingElse(){};

    @Test
    void stage3_thisDependsOnStage2(){};

    @Test
    void callTimeDoesntMatter(){}
}

This ensures stage2 methods are called after stage1 ones and before stage3 ones.

P.S. I feel this approach is better that jUnit 5.5 @Order annotation because it provides better notation for reader.

Solution 5

JUnit since 5.5 allows @TestMethodOrder(OrderAnnotation.class) on class and @Order(1) on test-methods.

JUnit old versions allow test methods run ordering using class annotations:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@FixMethodOrder(MethodSorters.JVM)
@FixMethodOrder(MethodSorters.DEFAULT)

By default test methods are run in alphabetical order. So, to set specific methods order you can name them like:

a_TestWorkUnit_WithCertainState_ShouldDoSomething b_TestWorkUnit_WithCertainState_ShouldDoSomething c_TestWorkUnit_WithCertainState_ShouldDoSomething

Or

_1_TestWorkUnit_WithCertainState_ShouldDoSomething _2_TestWorkUnit_WithCertainState_ShouldDoSomething _3_TestWorkUnit_WithCertainState_ShouldDoSomething

You can find examples here.

View more solutions