Test cases getting skipped with Selenium WebDriver using TestNG framework

Question

I am using Selenium WebDriver with TestNG framework for running test suite on Windows and MAC platform on different browsers - Chrome, IE, firefox and Safari. I have around 300 test cases in my test suite.

The problem is that some of the test cases gets skipped in between where I believe driver becomes unresponsive. However the logs failed to capture any details why the test cases are getting skipped.

The reporter class extends TestListenerAdapter and hence the skipped test cases gets listed in the log file with the use of onConfigurationSkip method. It only prints that a particular test case has been skipped.

Below are some code snippets for reference

Code from Reporter Class

@Override
public void onConfigurationSkip(ITestResult testResult) {
    LOGGER.info(String.format("Skipped Configuration for : %s", testResult.getMethod().getMethodName()));
}

Sample Test Class

public class TestClass {
private WebDriver driver;

@Parameters({ "platform", "browser"})
@BeforeClass
public void setUp(String platform, String browser) {
    // Creates a new driver instance based on platform and browser details
    driver = WebDriverFactory.getDriver(platform, browser);
    // Login to web application
    Utils.login(driver, username, password);
}


@Test
public void sampleTestMethod() {
   // scripts for validating Web elements
}

@AfterClass
public void tearDown() {
     driver.quit();;
}
}

Observations:

• driver.quit() doesn't guarantee that driver instance has been closed because I can still see driver instance running in task manager. Again this is intermittent and happens sometimes only.

• This issue is observed on all platform and browser

• This is an intermittent issue and probability of its occurrence increases as the number of test cases increase in test suite

• There is no definite pattern of skipped test cases. The test cases get randomly skipped on some browser and platform

• The probability of occurance of skip test cases increases with subsequent run of test suite. I believe the reason is that more and more driver instances that were not properly closed keep running in the back ground

• Normally a test Class has 5-15 test methods and new driver instance is created every time in @BeforeClass method and is closed in @AfterClass

Any Suggestions? Thanks in Advance

Solution

If you're fine with opening and closing the browsers around every test then you should use @BeforeMethod or @AfterMethod instead of @BeforeClass and @AfterClass

If you follow the following code and its output then you'll find that @BeforeMethod executes before every test annotated methods however, @BeforeClass does only once for all methods in the class.

Since I don't have your full code to analyze then I can just assume that your tests are trying to reuse the wrong driver instances. So the best bet would be to close it down after every test execution finishes.

Code:

package com.autmatika.testng;

import org.testng.annotations.*;

public class FindIt {

@BeforeClass
public void beforeClass(){
    System.out.println("Before Class");
}

@AfterClass
public void afterClass(){
    System.out.println("After Class");
}


@BeforeMethod
public void beforeTest(){
    System.out.println("Before Test");
}

@AfterMethod
public void afterTest(){
    System.out.println("After Test");
}

@Test
public void test1(){
    System.out.println("test 1");
}

@Test
public void test2(){
    System.out.println("test 2");
}
@Test
public void test3(){
    System.out.println("test 3");
}
@Test
public void test4(){
    System.out.println("test 4");
}

}

Output:

Before Class
Before Test
test 1
After Test
Before Test
test 2
After Test
Before Test
test 3
After Test
Before Test
test 4
After Test
After Class

===============================================
Default Suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================