Details: I'm trying to run my test scripts through TestRunner class. When using. parallel() method with thread count 5 or more, the script execution is getting suspended or getting aborted in the middle. I'm using karate-core version 1.2.0 currently and I'm facing this issue. When running with. parallel(1) no issues are encountered.
Below code executing properly
#Test
public void testParallel() {
Results results = Runner.path("classpath:feature").tags().outputCucumberJson(true).outputJunitXml(true).parallel(1);
}
Below code with thread count 5 is getting suspended or getting aborted in the middle.
#Test
public void testParallel() {
Results results = Runner.path("classpath:feature").tags().outputCucumberJson(true).outputJunitXml(true).parallel(5);
}
Any help is appreciated.
Related
I started working with TESTNG for the first time few days back.
I implemented a retry analyzer using the IRetryAnalyzer interface which retries a failed test method for a maximum of 3 times.
I have a test method called retryInvoker() which fails twice and succeeds the third time.
The way TestNG reports this is 3 separate test runs, 2 of which failed and 1 succeeded, which is correct actually.
Although I just wish to capture it as a single test run, which ultimately succeeded (if it did before the maximum allowed retries ended, which in this case were 3). Or even if it didn't succeed after 3 retries, I wish to report it as a single test run that failed instead of 4 separate test runs, all of which failed.
Any leads on this?
You could try the approach, suggested in this SO answer, removing the duplicates from the test context in case the same method is found either in failed or passed tests:
#Overrride
public void onFinish(ITestContext context) {
Iterator<ITestResult> failedTestCases = context.getFailedTests().getAllResults().iterator();
while (failedTestCases.hasNext()) {
System.out.println("failedTestCases");
ITestResult failedTestCase = failedTestCases.next();
ITestNGMethod method = failedTestCase.getMethod();
if (context.getFailedTests().getResults(method).size() > 1) {
System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString());
failedTestCases.remove();
} else {
if (context.getPassedTests().getResults(method).size() > 0) {
System.out.println(
"failed test case remove as pass retry:" + failedTestCase.getTestClass().toString());
failedTestCases.remove();
}
}
}
}
My code for handling my pipeline result looks like this (some snipped for brevity's sake):
PipelineResult result = pipeline.run();
switch (result.getState()) {
case DONE: {
handleDone();
break;
}
case FAILED: {
handleFailed();
break;
}
case CANCELLED: {
handleCancelled();
break;
}
case UNKNOWN:
case RUNNING:
case STOPPED: {
handleUnknownRunningStopped();
break;
}
default: {
assert false;
throw new IllegalStateException();
}
}
However, I've noticed that instead of returning a value of the enum PipelineResult.State for e.g. FAILED or CANCELLED, an exception is thrown:
For a failed job, a DataflowJobExecutionException is thrown
For a cancelled job, a DataflowJobCancelledException is thrown
What is the correct way (programmatically) to handle the result of a pipeline?
Both DataflowPipelineRunners return the PipelineResult that allows you to query the current status of the pipeline. The DataflowPipelineRunner returns a PipelineResult immediately upon job submission, but the BlockingDataflowPipelineRunner doesn't return it until the job has completed.
In addition, the BlockingDataflowPipeline runner throws an exception if the job does not complete successfully -- since you've specified a blocking run(), we assume you want to know if something goes wrong. So if you've hard coded the Blocking Runner, then relying on an exception is an easy way to handle failure.
Note that the code snippet you wrote uses the more general PipelineResult option, but won't work with the non-blocking runner, since that will return a result while the job is still running.
Does JUnit have an OOB tool to plot the test results of a suite? Specifically, I am using the Selenium 2 webdriver, and I want to plot passed vs failed tests. Secondly, I want to have my tests suite continue even with a failed test, how would I go about doing this? I tried researching the topic, but none of the answers fully addresses my question.
Thanks in advance!
Should probably put my code in here as well:
#Test
public void test_Suite() throws Exception {
driver.get("www.my-target-URL.com");
test_1();
test_2();
}
#Test
public void test_1() throws Exception {
//perform test
assertTrue(myquery);
}
#Test
public void test_2() throws Exception {
//perform test
assertTrue(myquery);
}
If you using Jenkins as your CI server, you got Junit Plugin that will allow you to publish the results in the end of the test. And you got Junit Graph to display them.
How to execute only the specific failed tests. By using 'IRetryAnalyzer' we can re-run the failed tests for x number of time. As mentioned here Restart failed test case automatically. I have also implemented iTestListener to make the tests count more meaning full by following Retry Only failed Tests and update test run count by implementing 'ITestListener'
Is there any way to Re-run ONLY specific failed tests.
Example: We need to execute only tests which are failed because of NoSuchElementException and TimeoutException.
Please find the below screen shot where total 8 tests are failed and there are 6 tests which are failed because of NoSuchElementException-1 and TimeoutException-5.
Please help.
You can try out by checking the result of your tests like:
#Override
public boolean retry(ITestResult result) {
try {
if (result.getThrowable().toString()
.contains("NoSuchElementException")) // Checking for specific reasons of failure
if (retryCount < maxRetryCount) {
retryCount++;
return true;
}
return false;
} catch (Exception e) {
return false;
}
}
Since every result has an attribute m_throwable in case of exception has occurred, you can use it to get your task done in Retry class.
I am invoking a java process using Apache exec library. I need to do some operation if the process is forcefully stopped ( using task manager or some other way). Is there any option available in exec library ? I found a waitfor() operation in ResultHandler, which is doing a busy wait. Is there any notification mechanism available ?
I would expect the exit status to reflect if the process had been forceably killed.
See the tutorial and in particular reference to your comment, this section:
Your worker thread will block until the print process has finished or
was killed by the watchdog. Therefore executing the print job
asynchronously will do the trick. In this example we create an
instance of 'ExecuteResultHandler' and pass it to the 'Executor'
instance in order to execute the process asynchronously. The
'resultHandler' picks up any offending exception or the process exit
code
This has been asked a long time ago but I stumbled upon it today...
Here is the solution: you just have to setup a ExecuteResultHandler and pass it to the execute method. Then no active wait involved. Like this:
DefaultExecuteResultHandler resultHandler = new DefaultExecuteResultHandler() {
#Override
public void onProcessComplete(int exitValue) {
super.onProcessComplete(exitValue);
log.info("Process finished");
}
#Override
public void onProcessFailed(ExecuteException e) {
super.onProcessFailed(e);
log.info("Process failed miserably");
}
};
String process = "C:\\Program Files (x86)\\Adobe\\Acrobat Reader DC\\Reader\\AcroRd32.exe";
CommandLine cmdLine = new CommandLine(process);
cmdLine.addArgument("C:\\Users\\francois.marot\\Downloads\\UtiliserLesDernieres DLL.pdf");
DefaultExecutor executor = new DefaultExecutor();
// the watchdog purpose is to kill the process after a specified timeout
SphereExecutionWatchdog watchdog = new SphereExecutionWatchdog(5000);
executor.setWatchdog(watchdog);
executor.execute(cmdLine, resultHandler);