testng not running in priority order when dependsOnMethods is specified

Question

Whenever we specify priority and dependsOnMethods on a @Test annotated method, the order of execution of test methods is not according to the priority. why is so? Here is the test class to demonstrate the issue:

package unitTest.TestNGTestCases;

import org.testng.annotations.Test;

public class TestNGTest1 {
    @Test(priority=1)
    public void t1()
    {
        System.out.println("Running 1");
    }
    @Test(priority=2,dependsOnMethods="t1")
    public void t2()
    {
        System.out.println("Running 2");
    }
    @Test(priority=3,dependsOnMethods="t2")
    public void t3()
    {
        System.out.println("Running 3");
    }
    @Test(priority=4)
    public void t4()
    {
        System.out.println("Running 4");
    }
}

Actual Output :

Running 1
Running 4
Running 2
Running 3

===============================================
All Tests Suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================

Expected output :

Running 1
Running 2
Running 3
Running 4

===============================================
All Tests Suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================

The order of test execution should have been t1, t2, t3, t4. why is t4 getting executed after t1, when t2 and t3 have higher priority then t4?

TIA

Solution 1

All independent methods (that do not have @dependsOnMethods dependency) will be executed first. Then methods with dependency will be executed. If there is ambiguity in execution order even after this ordering, priority comes into picture.

This is the ordering scheme:

1. Execute all independent methods (methods without @dependsOnMethods annotation)
2. If there is ambiguity in this ordering use priority to resolve ambiguity for independent methods
3. Execute dependent methods in order of dependency
4. If there is ambiguity in this ordering use priority to resolve ambiguity for dependent methods
5. If there is still ambiguity (due to not using priority or two methods having same priority) order them based on alphabetical order.

Now all ambiguity is resolved since no two methods can have the same name.

Solution 2

I've encountered the same problem today.

At first, I was using only priority for my tests, but then I needed to add dependsOnMethods as well.

Initially i've added the dependsOnMethods only to some of my @Test methods. As a result the execution order of my tests has scrambled.

I've read a lot of articles and discussions related to this topic and it turned out, that the using of priority and dependsOnMethods attributes togeter brings a lot of uncertainty into the whole picture and the behaviour of TestNG will never be predictable and well defined in this situation.

My solution was to add the dependsOnMethods to ALL of my test methods, while I kept the priority also for ALL of the mehtods. Now their execution order is back to normal and at the same time I benefit from the capabilities of the dependsOnMethods. i.e. The first failing test method in the chain, causes all subsequent test methods to be skipped and show correct in the reports.

Here is fragment from my tests class:

   @Test(priority = 2, dependsOnMethods= {"Meganav_Point_C1_and_Click_C3"})
    public void Click_product_in_Category_result_page() throws Throwable {

        Grid.clickProduct(1, 1);

    }


    @Test(priority = 3, dependsOnMethods= {"Click_product_in_Category_result_page"})
    public void PDP_setQty() throws Throwable {

        ProductDetailsPage.setQty(2);

    }


    @Test(priority = 4, dependsOnMethods= {"PDP_setQty"}, alwaysRun= true)
    public void PDP_click_Add_To_Basket() throws Throwable {

        ProductDetailsPage.addToBasket();

    }

Hope this helps.

Regards, Veselin Petrov

Comments

[user1058106]

I believe your example will work same way without priority attribute. The order is not affected by the attribute and will be built based on dependency chain.

[Mrunal Gosar]

What if I had multiple tests in a class and some of them are depended whilst others are not, but the order of execution is important?

[Renjith]

better as as a question instead of posting on an existing thread

[Glenn Lawrence]

This is an excellent summary and agrees with what I see too. Did you work this out by testing @Jerry, or did you find some Testng documentation that covers this?

[Glenn Lawrence]

It might be worth adding that if you don't set a priority it will implicitly have a priority of zero. At least this is what I see in my testing on TestNG version 6.14.3

[Jerry]

I figured it out by testing @GlennLawrence