Test cases getting skipped with Selenium WebDriver using TestNG framework - selenium

Question

I am using Selenium WebDriver with TestNG framework for running test suite on Windows and MAC platform on different browsers - Chrome, IE, firefox and Safari. I have around 300 test cases in my test suite.
The problem is that some of the test cases gets skipped in between where I believe driver becomes unresponsive. However the logs failed to capture any details why the test cases are getting skipped.
The reporter class extends TestListenerAdapter and hence the skipped test cases gets listed in the log file with the use of onConfigurationSkip method. It only prints that a particular test case has been skipped.
Below are some code snippets for reference
Code from Reporter Class
#Override
public void onConfigurationSkip(ITestResult testResult) {
LOGGER.info(String.format("Skipped Configuration for : %s", testResult.getMethod().getMethodName()));
}
Sample Test Class
public class TestClass {
private WebDriver driver;
#Parameters({ "platform", "browser"})
#BeforeClass
public void setUp(String platform, String browser) {
// Creates a new driver instance based on platform and browser details
driver = WebDriverFactory.getDriver(platform, browser);
// Login to web application
Utils.login(driver, username, password);
}
#Test
public void sampleTestMethod() {
// scripts for validating Web elements
}
#AfterClass
public void tearDown() {
driver.quit();;
}
}
Observations:
driver.quit() doesn't guarantee that driver instance has been closed because I can still see driver instance running in task manager. Again this is intermittent and happens sometimes only.
This issue is observed on all platform and browser
This is an intermittent issue and probability of its occurrence increases as the number of test cases increase in test suite
There is no definite pattern of skipped test cases. The test cases get randomly skipped on some browser and platform
The probability of occurance of skip test cases increases with subsequent run of test suite. I believe the reason is that more and more driver instances that were not properly closed keep running in the back ground
Normally a test Class has 5-15 test methods and new driver instance is created every time in #BeforeClass method and is closed in #AfterClass
Any Suggestions? Thanks in Advance

Answer 1

If you're fine with opening and closing the browsers around every test then you should use #BeforeMethod or #AfterMethod instead of #BeforeClass and #AfterClass
If you follow the following code and its output then you'll find that #BeforeMethod executes before every test annotated methods however, #BeforeClass does only once for all methods in the class.
Since I don't have your full code to analyze then I can just assume that your tests are trying to reuse the wrong driver instances. So the best bet would be to close it down after every test execution finishes.
Code:
package com.autmatika.testng;
import org.testng.annotations.*;
public class FindIt {
#BeforeClass
public void beforeClass(){
System.out.println("Before Class");
}
#AfterClass
public void afterClass(){
System.out.println("After Class");
}
#BeforeMethod
public void beforeTest(){
System.out.println("Before Test");
}
#AfterMethod
public void afterTest(){
System.out.println("After Test");
}
#Test
public void test1(){
System.out.println("test 1");
}
#Test
public void test2(){
System.out.println("test 2");
}
#Test
public void test3(){
System.out.println("test 3");
}
#Test
public void test4(){
System.out.println("test 4");
}
}
Output:
Before Class
Before Test
test 1
After Test
Before Test
test 2
After Test
Before Test
test 3
After Test
Before Test
test 4
After Test
After Class
===============================================
Default Suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================

Answer 2

The most common reason for the test cases getting skipped with Selenium using TestNG is if your methods are dependent on other method and a method you depend on is failed.

Answer 3

To get information on why the test got skipped you can implement that in after test method as below :-
#AfterMethod
public void afterTest(ItestResult result) {
Throwable t = result.getThrowable();
// with the object of t you can get the stacktrace and log it into your reporter
}
and to avoid tests getting skipped you can have alwaysRun parameter to be true after the #Test annotation
#Test(alwaysRun=true)
to avoid the webdriver getting retarted do the driver cleanups in methods with #BeforeMethod and #AfterMethod annotations, so try changing the annotation of setUp and tearDown methods to #BeforeMethod and #AfterMethod respectively