I have a complex problem, please read -
I wanted to run the extent report with my framework which is created using cucumber, testng and java.
I am running the testng file to run the test class file which will eventually run the #CucumberOptions and features added in it.
My test class is extended with AbstractTestNGCucumberTests to support testng with cucumber.
Now everything is working fine. basic testng report is getting generated.
note : I am running some tests parallel and they are also working fine.
Now i wanted to implement the extent report in my framework. Can you tell me how to do it using plugin which which add the logs same as the cucumber statement added in the feature file.
ps : I have gone through some link which shows me how to run the extent report with junit using #AfterClass, but how to use this with testng.
What I have tried -
libraries used - com.aventstack --> extentreports and com.vimalselvam --> cucumber-extentsreport
Error - cucumber.runtime.CucumberException: Couldn't load plugin class: com.vimalselvam.cucumber.ExtentCucumberFormatter
used plugin - plugin = {"com.vimalselvam.cucumber.ExtentCucumberFormatter:output/report.html"}
and code added under #AfterTest is : Reporter.loadXMLConfig(new File("xml path")
PLEASE HELP !!!
I wanted to generate extent report which will also support parallel execution.
If you want to run test cases in parallel and want to have stable extent report, please go for cucumber 4.x.x version and there are 2 ways of implementing extent report in Cucumber
1. Using Cucumber-JVM 4 adapter for Extent Framework(extentreports-cucumber4-adapter) & below are the steps to implement - (Beauty - You do not need to write any code any where to generate report this way except from setting adapter in runner below)
Add adapter dependency under POM.XML
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports-cucumber4-adapter</artifactId>
<version>1.0.6</version>
</dependency>
Add the com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter plugin to the runner.
#RunWith(Cucumber.class)
#CucumberOptions(plugin = {"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"})
public class RunCukesTest {
// ..
}
Report Output Directory - ../Project Directory/test-output/HtmlReport
2. Adding aventstack dependency under POM.XML
<dependency>
<groupId>com.aventstack</groupId>
<artifactId>extentreports</artifactId>
<version>3.1.5</version>
</dependency>
In this workflow, Do not Add the com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter plugin to the runner.
I assume you are using cucumber-extentsreport.*.jar as dependency.
You must specify the extent properties #beforesuite method.
ExtentProperties extentProperties = ExtentProperties.INSTANCE;
extentProperties.setReportPath();
you will also need to add plugin - com.cucumber.listener.ExtentCucumberFormatter:
Related
I am using Gradle 7.6, Karate 1.3.1, Java 17.0.5 and Junit 5.8.1.
I want to configure a Jenkin job for each feature to create a health check monitor. I need gradle commands to run feature files using tags #smoke, #regression, #featureName etc.,
I have tried with the following command, it worked earlier and stopped working recently.
./gradlew test -Dkarate.options="--tags #smoke" -Dtest.single=TestRunner#testTagsWithoutFeatureName
Where TestRunner is the following Java class
import com.intuit.karate.junit5.Karate;
public class TestRunner {
#Karate.Test
Karate testTagsWithoutFeatureName() {
return Karate.run().tags("#smoke").relativeTo(getClass());
}
}
My advice is use the Runner class, that is better designed for running tests in CI. The JUnit helpers are just for local-dev convenience: https://stackoverflow.com/a/65578167/143475
It should be possible to even pass a feature to karate.options as the last argument. Which might be more convenient than writing a Java class for every combinations. You should experiment.
Otherwise no suggestions, but if you feel there's a bug, follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue
I have recently upgraded to version 1.0.0 from 0.9.6 and noticed that the generated karate-summary.html file, it doesn't display all the tested feature files in the JUnit 5 Runner unlike in 0.9.6.
What it displays instead was the last tested feature file only.
The below screenshots are from the provided SampleTest.java sample code (excluding other Tests for simplicity).
package karate;
import com.intuit.karate.junit5.Karate;
class SampleTest {
#Karate.Test
Karate testSample() {
return Karate.run("sample").relativeTo(getClass());
}
#Karate.Test
Karate testTags() {
return Karate.run("tags").relativeTo(getClass());
}
}
This is from Version 0.9.6.
And this one is from Version 1.0.0
However, when running the test below in 1.0.0, all the features are displayed in the summary correctly.
#Karate.Test
Karate testAll() {
return Karate.run().relativeTo(getClass());
}
Would anyone be kind to confirm if they are getting the similar result? It would be very much appreciated.
What it displays instead was the last tested feature file only.
This is because for each time you run a JUnit method, the reports directory is backed up by default. Look for other directories called target/karate-reports-<timestamp> and you may find your reports there. So maybe what is happening is that you have multiple JUnit tests that are all running, so you see this behavior. You may be able to over-ride this behavior by calling the method: .backupReportDir(false) on the builder. But I think it may not still work - because the JUnit runner has changed a little bit. It is designed to run one method at a time, when you are in local / dev-mode.
So the JUnit runner is just a convenience. You should use the Runner class / builder for CI execution, and when you want to run multiple tests and see them in one report: https://stackoverflow.com/a/65578167/143475
Here is an example: ExamplesTest.java
But in case there is a bug in the JUnit runner (which is quite possible) please follow the process and help the project developers replicate and then fix the issue to release as soon as possible.
I am using karate 0.9.2 with gradle. My project requires to have all karate tests inside src/main/java. So I configured the gradle dependency as ‘compile’ instead of ‘testCompile’ and also modified the sourceSets to point to main instead of test. When I ran my runner class with above configuration I got empty test suite message.
build.gradle snippet:
compile 'com.intuit.karate:karate-junit4:0.9.3'
compile 'com.intuit.karate:karate-apache:0.9.3'
sourceSets {
test {
resources {
srcDir file('src/main/java')
exclude '**/*.java'
}
}
}
Additionally, I have is to run the karate tests from the deployable project jar. Please point me the resources I can refer to achieve the same.
Not something we directly support but teams have done this in Spring Boot etc. It should be possible, see if this thread helps: https://github.com/intuit/karate/issues/520
Also you may not need JUnit also: https://github.com/intuit/karate/issues/427
And see the sample project in this ticket as an example: https://github.com/intuit/karate/issues/529
EDIT - in 1.0 onwards we hope that class-loading from spring-boot JAR files is more reliable: https://github.com/intuit/karate/issues/751
The title may be a bit confusing at this point; hopefully I can clear it up.
What I Have
I'm running Cucumber JVM with Selenium WebDriver to automate our system test cases. These test cases are currently stored in JIRA using the XRay Test Management plugin. XRay also provides APIs to fetch the feature files as well as upload the results back to JIRA.
I have created a custom JIRA utility class to download the tests as feature files and upload the test results from and to JIRA - as well as demonstrated that it does work. These are run in the #BeforeClass and #AfterClass in the Cucumber Runner class respectively.
I have also demonstrated that the developed test framework does work by manually running with feature files created on my computer.
What I Want
I want to be able to (eventually) run the automation test framework automatically with our CI tools. Along with this, it would pull the defined automation tests from JIRA and push the test results back to JIRA.
I do not want the feature files stored with the code. In my opinion, this defeats the purpose of it being dynamic as the tests we execute will change over time (in number executed and the steps themselves).
What Is Happening (Or More Specifically, Not Happening)
When I try to execute the Cucumber Runner class without any feature files in the framework, Cucumber says "No features found at [src/test/resources/features/]". This is understandable since there are no feature files (yet).
However, it does not run the #BeforeClass; thus it does not download the feature files to be run. I have tried this both with and without tags in the runner class.
Code
#RunWith(Cucumber.class)
#CucumberOptions(
tags={"#smoketests"},
features= {"src/test/resources/features/"},
plugin={"json:target/reports/cucumber.json"},
monochrome=true)
public class RunCucumberTest {
#BeforeClass
public static void executeBeforeTests() {
JiraUtil.getFeatureFiles();
//String browser = "firefox";
String browser = "chrome";
//String browser = "safari";
//String browser = "edge";
//String browser = "ie";
DriverUtil.getInstance().setDriver(browser);
}
#AfterClass
public static void executeAfterTests() {
DriverUtil.getInstance().resetDriver();
JiraUtil.uploadTestResults();
}
}
Back To My Question
How can I execute the JIRA Util code so I can download the feature files?
Is it possible to achieve what I want? Or do I have to admit defeat and just have all the feature files stored with the code?
This is the expected behavior when using JUnit. A test suite will not invoke the #BeforeClass, #AfterClass or #ClassRule when there are no tests in the suite or if all tests are ignored[1]. This avoids the execution of a potentially expensive setup for naught.
This does mean you can't use a class rule to bootstrap your tests. Nor should you attempt to do so. In a build process it is a good practice to fetch all sources and resources prior to compilation.
If you are using maven could write a maven instead and attach it to the generate-test-sources phase[2]. Creating a maven plugin is a bit more involved then a JUnit Rule but not prohibitively so. Check the Guide to Developing Java Plugins.
I assume there are similar options for Gradle.
We are using arquillian-junit-container 1.0.0 final version for Junit Test.
We have so many test classes and every test class as #Deployment method so when i run all test together then its creating issue of memory and performance.
Can anyone help me to sort out this issue by telling how we can avoid multiple deployment for each single class. How we can achive Single deployment for all test cases in Arquillian?
You can't, officially, yet.
The JIRA issue ARQ-197 was created to support running multiple tests classes against a single deployment. In 2010! If you want this feature, please vote for it.
This is Arquillian's most voted for issue. It's currently slated for version 2.0.0.CR1, which might be the next version, but I can't find a roadmap / release plan anywhere which confirms this.
In the meantime, there is the Arquillian Suite Extension (latest incarnation is here). It's not official, and so there are limitations, but the original codebase was written by one of the Arquillian core developers to prove they could eventually support JUnit suites. The issue to make this support official is here and is Arquillian's second-most voted issue.
I think you're asking why does Arquillian need to deploy a new war for each test class when you run the test. I have the deployment method as Petr Mensik describes, but each test class will still deploy in it's own war when you run the tests. Putting it in the super class only simplifies the code from a less lines perspective. It will mean that every test class that is sub class will have the same deployment. That means your deployment will be the super set of dependencies, and thus much larger than doing it individually. I think it's easier to manage, and worth the price especially with larger projects.
To answer your question, it looks like you will not be able to group your tests and deploy one war to test until version 2.0 (due out at the end of this year?).
Why should you have deployment method in every class?I use Arquillian for functional testing with Drone and Graphene and I have one base class with deployment method, initialization of Selenium Web Driver, few utils methods and my every other test class is just extending this class and reusing my Web Driver instance.
I don't see why shouldn't this work in your case (or even without extending the base class).
Ok, this is how it looks
public class WebDriverTest extends Arquillian { //I am using TestNG
#Drone
protected WebDriver driver;
#ArquillianResource
private URL contextRoot;
#Deployment(testable = false) //functional tests cannot run in container
public static WebArchive createDeployment() {
File archive = new File("target/myApp.war");
ShrinkWrap.createFromZipFile(WebArchive.class, archive);
}
}
public class TestClass extends WebDriverTest {
#Test
public void test1() {}
#Test
public void test2() {}
}
Everything is working fine here. Also make sure that you have right Maven dependencies, these has to be present in order to run functional tests (then make dependency for anything you need from these BOMs)
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.jboss.arquillian.selenium</groupId>
<artifactId>selenium-bom</artifactId>
<version>${selenium.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian</groupId>
<artifactId>arquillian-bom</artifactId>
<version>${arquillian-core.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.extension</groupId>
<artifactId>arquillian-drone-bom</artifactId>
<version>${arquillian-drone.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
EDIT
Ok, so according to this JIRA you won't see this feature in Arquillian until version 2.0.0.CR1. So the code I mentioned above is the best you can get right now:-)