Test order with espresso

Question

Is there a way to set test running order in android?
I use Espresso framework and need to test a lot of activities and transitions between them. I want to write different test for those activities, but I need a specific order for running those tests.

Solution 1

As @spinster said above, you should write your tests in a way where order doesn't matter.

I believe Junit 3 will run tests in alphabetical order of the fully qualified class name, so in theory you could control the order by naming them ( package name, class name, method name ) alphabetically in the order you would like them executed, but I would not recommend that.

See: How to run test methods in specific order in JUnit4? How to pre-define the running sequences of junit test cases？

Solution 2

espresso set running order of tests

From Junit 4.11 comes with @FixMethodOrder annotation. Instead of using custom solutions just upgrade your junit version and annotate test class with FixMethodOrder(MethodSorters.NAME_ASCENDING). Check the release notes for the details.

Here is a sample:

import org.junit.runners.MethodSorters;

import org.junit.FixMethodOrder;
import org.junit.Test;

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SampleTest {

   @Test
   public void A_firstTest() {
      System.out.println("first");
   }

   @Test
   public void B_secondTest() {
      System.out.println("second");
   }
}

Solution 3

You can add annotation as test runner fixture as shown here:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)

just above class name

Solution 4

Yes You can set order using the order no with the test_name, See the below example-

public class MyEspressoTest
        extends ActivityInstrumentationTestCase2<UserLoginActivity> {

    private UserLoginActivity mActivity;

    public MyEspressoTest() {
        super(UserLoginActivity.class);
    }

    @Before
    public void setUp() throws Exception {
        super.setUp();
        injectInstrumentation(InstrumentationRegistry.getInstrumentation());
        mActivity = getActivity();
    }

    public void test1InvalidPropigerLogin() {
        // Type text and then press the button.

        //setContentView function to see the layout

        onView(withId(R.id.username))
                .perform(typeText("[email protected]"), closeSoftKeyboard());
        onView(withId(R.id.password))
                .perform(typeText("hhhhh"), closeSoftKeyboard());

        onView(withId(R.id.user_login_button)).perform(click());
        // Check that the text was changed.
        onView(withId(R.id.login_status))
                .check(matches(withText("Invalid username or password")));

        //System.out.println("Test pass with invalid user and password");
    }

    public void test2ValidPropigerLogin() {
        // Type text and then press the button.

        onView(withId(R.id.username))
                .perform(typeText("[email protected]"), closeSoftKeyboard());
        onView(withId(R.id.password))
                .perform(typeText("gggggg"), closeSoftKeyboard());

        onView(withId(R.id.user_login_button)).perform(click());

        //System.out.println("Test pass with valid user and password");
    }

    public void test3ForgetPasswordButton() {

        onView(withId(R.id.forgot_pwd_button)).perform(click());

        //onView(isRoot()).perform(ViewActions.pressBack());

        onView(withId(R.id.email_edittext))
                .perform(typeText("[email protected]"), closeSoftKeyboard());
        onView(withId(R.id.reset_password_button)).perform(click());
        // Check that the text was changed.
        onView(withId(R.id.reset_result))
                .check(matches(withText("Email not registered with propiger")));
    }
    public void test4ForgetPasswordButton2() {

        onView(withId(R.id.forgot_pwd_button)).perform(click());

        onView(withId(R.id.email_edittext))
                .perform(typeText("[email protected]"), closeSoftKeyboard());
        onView(withId(R.id.reset_password_button)).perform(click());
        // Check that the text was changed.
        onView(withId(R.id.reset_result))
                .check(matches(withText("Reset password link sent successfully")));
    }
    public void test5RegisterButton() {
        onView(withId(R.id.register_button)).perform(click());

              //onView(isRoot()).perform(ViewActions.pressBack());

        onView(withId(R.id.register_name_edittext))
                .perform(typeText("Hill Hacker"), closeSoftKeyboard());
        onView(withId(R.id.register_email_edittext))
                .perform(typeText("[email protected]"), closeSoftKeyboard());
        onView(withId(R.id.register_mobileno_edittext))
                .perform(typeText("9090909090"), closeSoftKeyboard());
        onView(withId(R.id.register_password_edittext))
                .perform(typeText("password111"), closeSoftKeyboard());
        onView(withId(R.id.register_confirm_password_edittext))
                .perform(typeText("password111"), closeSoftKeyboard());
        //onView(withId(R.id.register_country_spinner)).perform(click());
        //onView(isRoot()).perform(withId(R.id.register_country_spinner, Sampling.SECONDS_15));
        onData(allOf(is(instanceOf(String.class)), is("India")))
                .perform(click());

       onView(withId(R.id.register_country_spinner)).check(matches(withText(containsString("India"))));

        onView(withId(R.id.register_button)).perform(click());

    }


}

Solution 5

You have 3 ways:

---

---

way 1: JUnit 4 and 5 work

@Test
public void testFunctionMain() {
    test1(); 
    test2()
    test3(); 
}

---

way 2: JUnit 4 and 5 work

use @FixMethodOrder

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@RunWith(AndroidJUnit4::class)
class LoginActivityTest {
}

---

way 3: Junit5 work

use @Order

@Test
@Order(2)
public void testFunction(){
}

View more solutions

Comments

[Tim Boland]

wow...that sounds crazy to me...the test suite could take forever if each test method required its own setup...especially if the setup functionality has already been tested by a previously ran test method..with that methodology, you would be retesting scenarios over and over again just to get to the required states for each given test method...if specifying a test order is a bad idea...forcing alphabetical is an even worse one.

[yogurtearl]

Yes, it is a bit "crazy" and I would not recommend it, but it was an answer to the question posed.

[mr. Nutscracker]

you can put some authorization\deauthorization functionality out from the test into specific methods. Then from LoginTest you just call authorization method, from LogoutTest - authorization and then deauthorization. Other tests will then extend some BaseTest which will have authorization in setUp method, and, if needed, deauthorization in tearDown

[zzz]

What if you want to run the test in order (with the class name)?

[pregmatch]

I don't agree, what if you are testing api call that store api token in memory (with singleton) and other test access that token. Do you still think that order does not matter?