In my Karate runner I am using .outputCucumberJson(true) as shown below to generate a Cucumber Report (in order to upload this back to our XRAY tests):
class KarateRunnerTest {
#Test
void testParallel() {
Results results = Runner.path("classpath:apiTesting/karateFeatureFiles/")
.outputCucumberJson(true)
.parallel(5);
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
}
However, it is producing one report for each Feature File.
Is there a way for it to just generate one report for all Feature Files?
So this currently can't be done in Karate.
As a workaround I used a little npm tool called 'cucumber-json-merge'.
This merged the reports into 1 and seems to work fine.
Related
I am trying to integrate karate in my project for integration testing. I was trying to use karate.callSingle() to fetch authorization headers.
I have two tests
#Test
void test1() {
Results results = Runner.path("classpath:integrationTests/test1.feature").parallel(2);
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
#Test
void test2 {
Results results = Runner.path("classpath:integrationTests/test2.feature").parallel(1);
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
Now the test1.feature has two scenarios and test2.feature has one scenario. The tests are running fine just that in the logs I see karate.callSingle is executed twice...for both tests. Is this the expected behaviour?
Yes thats the expected behavior. karate.callSingle() is scoped to a single instance of a Runner.
In my regression suite I have 600+ test cases. All those tests have #RegressionTest tag. See below, how I am running.
_start = LocalDateTime.now();
//see karate-config.js files for env options
_logger.info("karate.env = " + System.getProperty("karate.env"));
System.setProperty("karate.env", "test");
Results results = Runner.path("classpath:functional/Commercial/").tags("#RegressionTest").reportDir(reportDir).parallel(5);
generateReport(results.getReportDir());
assertEquals(0, results.getFailCount(), results.getErrorMessages());
I am thinking that, I can create 1 test and give it a tag #smokeTest. I want to be able to run that test 1st and only if that test passes then run the entire Regression suite. How can I achieve this functionality? I am using Junit5 and Karate.runner.
I think the easiest thing to do is run one test in JUnit itself, and if that fails, throw an Exception or skip running the actual tests.
So use the Runner two times.
Otherwise consider this not supported directly in Karate but code-contributions are welcome.
Also refer to the answers to this question: How to rerun failed features in karate?
I have POM based selenium framework,i am using extent reports and the reports are generated fine if all of the scripts run. If one of the script fails abruptly due to browser disappear then my script is failing and then report is not generated.
Ex: I have 3 scripts to run as part of my driver script, when 3rd script is running if something goes wrong(like browser disappear) then report is not generated. I want extent report generated whenever it fails/stops. My driver script has extent.flush which is the last execution as part of scripts. How can we generate report on failure. If the failure is due to object not found then i am able to get the report.
How to generate report whenever i stop the execution?
Any help is greatly appreciated.
thanks
Raju
I'm assuming that you are using TestNG if that is true add the parameter alwaysRun = true in the TestNG methods e.g.
#BeforeMethod(alwaysRun = true)
public void beforeMethod()
//your_code
}
#AfterMethod(alwaysRun = true)
public void afterMethod() {
//your_code
}
I am new in using Selenium IDE. I'm writing this test case where User A clicks this link and then it should direct the user to the correct page. Unfortunately, the page returns:
An error occured. Message: script 'pp/agensi-list.phtml' not found in path (C:/htdffocs/star/application/views\scripts/**)
But on my selenium, it shows the test case has passed (it should fail).
Can someone tell me why?
Hi it is very easy to write test case results pass/fail using selenium with testng framework,please read from below link i have given clear examples.
How can I write Test Result (Pass/Fail) in Excel file using TestNG Framework with Selenium WebDriver?
For more with real time examples ,you can read write test case pass/fail in excel using testng
Simply you have to create Excel utility for fileinput stream and fileoutputstream classes as below and call that class into another classes using extends keyword
Excel Utility:
public class Excelutility {
#BeforeTest
public void exceloperation(){
FileinputStream file = new FileiinputStream("file path");
//do file input stream details here
Fileoutputstream newfile=new Fileoutputstream("filepath");
//do fileoutputstream details here like create workbook,create sheets etc
}
}
Another class:
public class operations{
#Test
public void openbrowser(){
//write driver operations here
//write excel operations using
if(Title.equalsIgnoreCase("HP Loadrunner Tutorial")){
Testcase="PASS";
}else{
Testcase = "FAIL";
}
Label l1=new Label(1,2,"status")
writablesh.addCell(l3);
}
The test is likely passing because you aren't verifying anything after the click.
If the test stops after the link is clicked then it will pass. You have to confirm that something happened. Look for a change in the CSS or perhaps even look for the message you listed above?
Is there a way to (easily) generate a HTML report that contains the tests results ? I am currently using JUnit in addition to Selenium for testing web apps UI.
PS: Given the project structure I am not supposed to use Ant :(
I found the above answers quite useful but not really general purpose, they all need some other major build system like Ant or Maven.
I wanted to generate a report in a simple one-shot command that I could call from anything (from a build, test or just myself) so I have created junit2html which can be found here: https://github.com/inorton/junit2html
You can install it by doing:
pip install junit2html
Alternatively for those using Maven build tool, there is a plugin called Surefire Report.
The report looks like this : Sample
If you could use Ant then you would just use the JUnitReport task as detailed here: http://ant.apache.org/manual/Tasks/junitreport.html, but you mentioned in your question that you're not supposed to use Ant.
I believe that task merely transforms the XML report into HTML so it would be feasible to use any XSLT processor to generate a similar report.
Alternatively, you could switch to using TestNG ( http://testng.org/doc/index.html ) which is very similar to JUnit but has a default HTML report as well as several other cool features.
You can easily do this via ant. Here is a build.xml file for doing this
<project name="genTestReport" default="gen" basedir=".">
<description>
Generate the HTML report from JUnit XML files
</description>
<target name="gen">
<property name="genReportDir" location="${basedir}/unitTestReports"/>
<delete dir="${genReportDir}"/>
<mkdir dir="${genReportDir}"/>
<junitreport todir="${basedir}/unitTestReports">
<fileset dir="${basedir}">
<include name="**/TEST-*.xml"/>
</fileset>
<report format="frames" todir="${genReportDir}/html"/>
</junitreport>
</target>
</project>
This will find files with the format TEST-*.xml and generate reports into a folder named unitTestReports.
To run this (assuming the above file is called buildTestReports.xml) run the following command in the terminal:
ant -buildfile buildTestReports.xml
Junit xml format is used outside of Java/Maven/Ant word.
Jenkins with http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin is a solution.
For the one shot solution I have found this tool that does the job:
https://www.npmjs.com/package/junit-viewer
junit-viewer --results=surefire-reports --save=file_location.html
--results= is directory with xml files (test reports)
I found xunit-viewer, which has deprecated junit-viewer mentioned by #daniel-kristof-kiss.
It is very simple, automatically recursively collects all relevant files in ANT Junit XML format and creates a single html-file with filtering and other sweet features.
I use it to upload test results from Travis builds as Travis has no other support for collecting standard formatted test results output.
There are multiple options available for generating HTML reports for Selenium WebDriver scripts.
1. Use the JUNIT TestWatcher class for creating your own Selenium HTML reports
The TestWatcher JUNIT class allows overriding the failed() and succeeded() JUNIT methods that are called automatically when JUNIT tests fail or pass.
The TestWatcher JUNIT class allows overriding the following methods:
protected void failed(Throwable e, Description description)
failed() method is invoked when a test fails
protected void finished(Description description)
finished() method is invoked when a test method finishes (whether passing or failing)
protected void skipped(AssumptionViolatedException e, Description
description)
skipped() method is invoked when a test is skipped due to a failed assumption.
protected void starting(Description description)
starting() method is invoked when a test is about to start
protected void succeeded(Description description)
succeeded() method is invoked when a test succeeds
See below sample code for this case:
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class TestClass2 extends WatchManClassConsole {
#Test public void testScript1() {
assertTrue(1 < 2); >
}
#Test public void testScript2() {
assertTrue(1 > 2);
}
#Test public void testScript3() {
assertTrue(1 < 2);
}
#Test public void testScript4() {
assertTrue(1 > 2);
}
}
import org.junit.Rule;
import org.junit.rules.TestRule;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class WatchManClassConsole {
#Rule public TestRule watchman = new TestWatcher() {
#Override public Statement apply(Statement base, Description description) {
return super.apply(base, description);
}
#Override protected void succeeded(Description description) {
System.out.println(description.getDisplayName() + " " + "success!");
}
#Override protected void failed(Throwable e, Description description) {
System.out.println(description.getDisplayName() + " " + e.getClass().getSimpleName());
}
};
}
2. Use the Allure Reporting framework
Allure framework can help with generating HTML reports for your Selenium WebDriver projects.
The reporting framework is very flexible and it works with many programming languages and unit testing frameworks.
You can read everything about it at http://allure.qatools.ru/.
You will need the following dependencies and plugins to be added to your pom.xml file
maven surefire
aspectjweaver
allure adapter
See more details including code samples on this article:
http://test-able.blogspot.com/2015/10/create-selenium-html-reports-with-allure-framework.html
I have created a JUnit parser/viewer that runs directly in the browser. It supports conversion to JSON and the HTML report can be easily reused.
https://lotterfriends.github.io/online-junit-parser/
If you are still missing a feature feel free to create an issue on Github. :)