Is there a way to (easily) generate a HTML report that contains the tests results ? I am currently using JUnit in addition to Selenium for testing web apps UI.
PS: Given the project structure I am not supposed to use Ant :(
I found the above answers quite useful but not really general purpose, they all need some other major build system like Ant or Maven.
I wanted to generate a report in a simple one-shot command that I could call from anything (from a build, test or just myself) so I have created junit2html which can be found here: https://github.com/inorton/junit2html
You can install it by doing:
pip install junit2html
Alternatively for those using Maven build tool, there is a plugin called Surefire Report.
The report looks like this : Sample
If you could use Ant then you would just use the JUnitReport task as detailed here: http://ant.apache.org/manual/Tasks/junitreport.html, but you mentioned in your question that you're not supposed to use Ant.
I believe that task merely transforms the XML report into HTML so it would be feasible to use any XSLT processor to generate a similar report.
Alternatively, you could switch to using TestNG ( http://testng.org/doc/index.html ) which is very similar to JUnit but has a default HTML report as well as several other cool features.
You can easily do this via ant. Here is a build.xml file for doing this
<project name="genTestReport" default="gen" basedir=".">
<description>
Generate the HTML report from JUnit XML files
</description>
<target name="gen">
<property name="genReportDir" location="${basedir}/unitTestReports"/>
<delete dir="${genReportDir}"/>
<mkdir dir="${genReportDir}"/>
<junitreport todir="${basedir}/unitTestReports">
<fileset dir="${basedir}">
<include name="**/TEST-*.xml"/>
</fileset>
<report format="frames" todir="${genReportDir}/html"/>
</junitreport>
</target>
</project>
This will find files with the format TEST-*.xml and generate reports into a folder named unitTestReports.
To run this (assuming the above file is called buildTestReports.xml) run the following command in the terminal:
ant -buildfile buildTestReports.xml
Junit xml format is used outside of Java/Maven/Ant word.
Jenkins with http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin is a solution.
For the one shot solution I have found this tool that does the job:
https://www.npmjs.com/package/junit-viewer
junit-viewer --results=surefire-reports --save=file_location.html
--results= is directory with xml files (test reports)
I found xunit-viewer, which has deprecated junit-viewer mentioned by #daniel-kristof-kiss.
It is very simple, automatically recursively collects all relevant files in ANT Junit XML format and creates a single html-file with filtering and other sweet features.
I use it to upload test results from Travis builds as Travis has no other support for collecting standard formatted test results output.
There are multiple options available for generating HTML reports for Selenium WebDriver scripts.
1. Use the JUNIT TestWatcher class for creating your own Selenium HTML reports
The TestWatcher JUNIT class allows overriding the failed() and succeeded() JUNIT methods that are called automatically when JUNIT tests fail or pass.
The TestWatcher JUNIT class allows overriding the following methods:
protected void failed(Throwable e, Description description)
failed() method is invoked when a test fails
protected void finished(Description description)
finished() method is invoked when a test method finishes (whether passing or failing)
protected void skipped(AssumptionViolatedException e, Description
description)
skipped() method is invoked when a test is skipped due to a failed assumption.
protected void starting(Description description)
starting() method is invoked when a test is about to start
protected void succeeded(Description description)
succeeded() method is invoked when a test succeeds
See below sample code for this case:
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class TestClass2 extends WatchManClassConsole {
#Test public void testScript1() {
assertTrue(1 < 2); >
}
#Test public void testScript2() {
assertTrue(1 > 2);
}
#Test public void testScript3() {
assertTrue(1 < 2);
}
#Test public void testScript4() {
assertTrue(1 > 2);
}
}
import org.junit.Rule;
import org.junit.rules.TestRule;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class WatchManClassConsole {
#Rule public TestRule watchman = new TestWatcher() {
#Override public Statement apply(Statement base, Description description) {
return super.apply(base, description);
}
#Override protected void succeeded(Description description) {
System.out.println(description.getDisplayName() + " " + "success!");
}
#Override protected void failed(Throwable e, Description description) {
System.out.println(description.getDisplayName() + " " + e.getClass().getSimpleName());
}
};
}
2. Use the Allure Reporting framework
Allure framework can help with generating HTML reports for your Selenium WebDriver projects.
The reporting framework is very flexible and it works with many programming languages and unit testing frameworks.
You can read everything about it at http://allure.qatools.ru/.
You will need the following dependencies and plugins to be added to your pom.xml file
maven surefire
aspectjweaver
allure adapter
See more details including code samples on this article:
http://test-able.blogspot.com/2015/10/create-selenium-html-reports-with-allure-framework.html
I have created a JUnit parser/viewer that runs directly in the browser. It supports conversion to JSON and the HTML report can be easily reused.
https://lotterfriends.github.io/online-junit-parser/
If you are still missing a feature feel free to create an issue on Github. :)
Related
I have test class MyTest:
package com.my.package;
...
#IntegrationTest
public class MyTest extends ServicelayerTest {
#Test
public myTest1() {...}
#Test
public myTest2() {...}
}
I only need to run the myTest1() test via ant.
To run all integration tests from the class, I can use
ant integrationtests -Dtestclasses.packages='com.my.package.MyTest'
How to run only one myTest1() ? Maybe I can use something like
ant integrationtests -Dtestclasses.packages='com.my.package.MyTest#myTest1()' ?
There is no filter that lets you run JUnit by method name.
Check the Filters documentation on what is supported: https://help.sap.com/docs/SAP_COMMERCE/d0224eca81e249cb821f2cdf45a82ace/f7f454a4f5254944a366db9bdf129be6.html
In my Karate runner I am using .outputCucumberJson(true) as shown below to generate a Cucumber Report (in order to upload this back to our XRAY tests):
class KarateRunnerTest {
#Test
void testParallel() {
Results results = Runner.path("classpath:apiTesting/karateFeatureFiles/")
.outputCucumberJson(true)
.parallel(5);
assertEquals(0, results.getFailCount(), results.getErrorMessages());
}
}
However, it is producing one report for each Feature File.
Is there a way for it to just generate one report for all Feature Files?
So this currently can't be done in Karate.
As a workaround I used a little npm tool called 'cucumber-json-merge'.
This merged the reports into 1 and seems to work fine.
I know that Codeception is designed for command line usage. But as it is completely based on PHP, I am pretty sure there must be a way to dynamically/temporarily create a test by PHP.
In my case I am getting acceptance test steps from a database and need to run the tests dynamically with Codeception. I would prefer a way to test it without always having to generate and delete temporary test folders and running the codeception commands on the commandline.
The problem is that Codeception dynamically generates a bunch of config files and scripts when creating a cest. I couldn't make it work by using the Codeception classes.
Does anyone have an idea what's the best way to achieve this?
I think that the best approach would be to implement custom test loader as documented at https://codeception.com/docs/07-AdvancedUsage#Formats
You still have to use placeholder file in each suite to kickoff the loader, but the tests can be loaded from database.
Copy of documentation:
In addition to the standard test formats (Cept, Cest, Unit, Gherkin)
you can implement your own format classes to customise your test
execution. Specify these in your suite configuration:
formats:
- \My\Namespace\MyFormat
Then define a class which implements the LoaderInterface
namespace My\Namespace;
class MyFormat implements \Codeception\Test\Loader\LoaderInterface
{
protected $tests;
protected $settings;
public function __construct($settings = [])
{
//These are the suite settings
$this->settings = $settings;
}
public function loadTests($filename)
{
//Load file and create tests
}
public function getTests()
{
return $this->tests;
}
public function getPattern()
{
return '~Myformat\.php$~';
}
}
Look at existing Loader classes for inspiration: https://github.com/Codeception/Codeception/tree/4.0/src/Codeception/Test/Loader
I am new in using Selenium IDE. I'm writing this test case where User A clicks this link and then it should direct the user to the correct page. Unfortunately, the page returns:
An error occured. Message: script 'pp/agensi-list.phtml' not found in path (C:/htdffocs/star/application/views\scripts/**)
But on my selenium, it shows the test case has passed (it should fail).
Can someone tell me why?
Hi it is very easy to write test case results pass/fail using selenium with testng framework,please read from below link i have given clear examples.
How can I write Test Result (Pass/Fail) in Excel file using TestNG Framework with Selenium WebDriver?
For more with real time examples ,you can read write test case pass/fail in excel using testng
Simply you have to create Excel utility for fileinput stream and fileoutputstream classes as below and call that class into another classes using extends keyword
Excel Utility:
public class Excelutility {
#BeforeTest
public void exceloperation(){
FileinputStream file = new FileiinputStream("file path");
//do file input stream details here
Fileoutputstream newfile=new Fileoutputstream("filepath");
//do fileoutputstream details here like create workbook,create sheets etc
}
}
Another class:
public class operations{
#Test
public void openbrowser(){
//write driver operations here
//write excel operations using
if(Title.equalsIgnoreCase("HP Loadrunner Tutorial")){
Testcase="PASS";
}else{
Testcase = "FAIL";
}
Label l1=new Label(1,2,"status")
writablesh.addCell(l3);
}
The test is likely passing because you aren't verifying anything after the click.
If the test stops after the link is clicked then it will pass. You have to confirm that something happened. Look for a change in the CSS or perhaps even look for the message you listed above?
because I think I recommend CodenameOne to be used for development I try to investigate deeper into it. I just tried out the Test Recorder which generated a test class.
Now my question: How-to use this test class? Do I have to call the test method from the existing UI code using e.g. a button to start it?
Generated code:
public class RegisterUserATest extends AbstractTest {
public boolean runTest() throws Exception {
clickButtonByName("Register");
keyPress(16);
keyPress(65);
waitFor(112);
keyPress(65);
setText("Name", "A");
keyPress(16);
keyPress(65);
waitFor(113);
keyPress(16);
waitFor(1);
keyPress(97);
setText("Email", "");
setText("Password", "A");
clickButtonByName("Register");
return true;
}
}
I think the solution is very easy but I cannot see it.
If this is on NetBeans right click the project and select "Test". On IntelliJ/IDEA it's under Codename One -> Run Tests.
Notice that the latter has a bug in it that will be fixed in the release coming tomorrow (October 7th 2016).