I have test class MyTest:
package com.my.package;
...
#IntegrationTest
public class MyTest extends ServicelayerTest {
#Test
public myTest1() {...}
#Test
public myTest2() {...}
}
I only need to run the myTest1() test via ant.
To run all integration tests from the class, I can use
ant integrationtests -Dtestclasses.packages='com.my.package.MyTest'
How to run only one myTest1() ? Maybe I can use something like
ant integrationtests -Dtestclasses.packages='com.my.package.MyTest#myTest1()' ?
There is no filter that lets you run JUnit by method name.
Check the Filters documentation on what is supported: https://help.sap.com/docs/SAP_COMMERCE/d0224eca81e249cb821f2cdf45a82ace/f7f454a4f5254944a366db9bdf129be6.html
Related
I know using this reflection we can surely get list of all groups of the #test.
#BeforeMethod
public void befrMethod(Method met){
Test t = met.getAnnotation(Test.class);
System.out.println(t.groups());
}
But Is there a way to get groups list while inside the #test and not beforemethod?
Because I am running tests in parallel and this method is not working well for me.
Test method supports ITestContext dependency injection
And this class has a method called
getIncludedGroups java.lang.String[] getIncludedGroups() Returns: All
the groups that are included for this test run.
"ITestContext (testng 7.3.0 API)" https://javadoc.io/doc/org.testng/testng/latest/org/testng/ITestContext.html
So try
#Test
public void befrMethod(ITestContext met){
I know that Codeception is designed for command line usage. But as it is completely based on PHP, I am pretty sure there must be a way to dynamically/temporarily create a test by PHP.
In my case I am getting acceptance test steps from a database and need to run the tests dynamically with Codeception. I would prefer a way to test it without always having to generate and delete temporary test folders and running the codeception commands on the commandline.
The problem is that Codeception dynamically generates a bunch of config files and scripts when creating a cest. I couldn't make it work by using the Codeception classes.
Does anyone have an idea what's the best way to achieve this?
I think that the best approach would be to implement custom test loader as documented at https://codeception.com/docs/07-AdvancedUsage#Formats
You still have to use placeholder file in each suite to kickoff the loader, but the tests can be loaded from database.
Copy of documentation:
In addition to the standard test formats (Cept, Cest, Unit, Gherkin)
you can implement your own format classes to customise your test
execution. Specify these in your suite configuration:
formats:
- \My\Namespace\MyFormat
Then define a class which implements the LoaderInterface
namespace My\Namespace;
class MyFormat implements \Codeception\Test\Loader\LoaderInterface
{
protected $tests;
protected $settings;
public function __construct($settings = [])
{
//These are the suite settings
$this->settings = $settings;
}
public function loadTests($filename)
{
//Load file and create tests
}
public function getTests()
{
return $this->tests;
}
public function getPattern()
{
return '~Myformat\.php$~';
}
}
Look at existing Loader classes for inspiration: https://github.com/Codeception/Codeception/tree/4.0/src/Codeception/Test/Loader
I've tried many ways with no success and I'm starting to believe this is not achievable in TestNG and I just like to confirm with you.
I have web service I'm testing and I need to run few basic scenarios.
My current test methods, each with #Test annotation (each need to be testable as a single test):
dbResetTest
clearCacheTest
openURLTest
loginTest
actionXTest
actionYTest
I also need to run these scenarios consisting from above tests run IN ORDER:
Test login feature (openURLTest -> dbResetTest -> clearCacheTest -> loginTest)
Test X after login (openURLTest -> dbResetTest -> clearCacheTest -> loginTest -> actionXTest)
Test Y after clearing cache (clearCacheTest -> actionYTest)
The issue is, if I made tests from point 1 & 2 dependant on others I won't be able to run scenario 3 because clearCacheTest does not depend on any other in this particular scenario. I've tried running those test in order through xml and by using dependencies but with no success.
Of course I could make actionYTest to call clearCacheTest directly but then if clearCacheTest fails the report will show that actionYTest was the failing one which is what I try to avoid.
I'm pretty sure now what I need is not achievable in TestNG but maybe I'm wrong...
I think you should change your tactics slightly. Instead of perceiving these ~(dbResetTest, etc.) as test Classes you should make them test methods instead and use dependsOnMethods programatically (not from XML) instead of dependsOnGroups. Then you will be able to implement your required logic rather easily (every test is unique --> #Test annotation, every test is executed in certain priority --> use priority parameter). Then the 1,2,3 tests should be your test classes. So here it is how you do it:
public class LoginFeature {
#Test (priority=1)
public openURLTest(){
}
#Test (priority=2, dependsOnMethods="openURLTest")
public dbResetTest (){
}
#Test (priority=3, dependsOnMethods="dbResetTest")
public clearCacheTest (){
}
#Test (priority=4, dependsOnMethods="clearCacheTest" )
public loginTest(){
}
}
This way if something fails in between your tests the rest of scenarios will automatically be skipped and you won't need to call directly clearCacheTest.
Hope this helps!
Update
After OP's comment
Well again I think you kinda have of a design issue. For your methods to be called multiple times they need to sit somewhere that they are accessible. You are almost there with your approach but not quite. So here is how you can call the methods; multiple times and run them every time from scratch (I'll show you the code first and then explain in detail):
parent class
public class TestBase{
//include here all your important methods *without* #Test Annotation
public void dbReset(){
//perform db reset
}
public void clearCache(){
//clear browser cache
}
public boolean openURL(){
//try to open test URL
return didIreachTestURLSuccessfully;
}
}
child class
public class loginFeature extends TestBase{
#Test (priority=1)
public void attemptToResetDataBase(){
dbReset();
}
#Test (priority=2, dependsOnMeth0ds="attemptToResetDataBase")
public void clearCacheTest(){
clearCache();
}
#Test (priority=3, dependsOnMeth0ds="clearCacheTest")
public void verifySuccessfulLogin(){
login();
}
}
So, you include all of your test methods in a parent class, called TestBase. Then you create your test (loginTest for example) with a class that extends TestBase. Now you can call your methods multiple times, treat them each time as an individual test and connect them with dependencies according to your needs (i.e I have each one of them depending on the prior method; but you could rearrange them and put them all to depend on one, or to no-one).
Because your test class inherits from TestBase you don't even need to create an object to access the internal methods; you can call them directly instead.
Hope this solves it for you, do not hesitate to write a comment if you need more info.
Is there a way to (easily) generate a HTML report that contains the tests results ? I am currently using JUnit in addition to Selenium for testing web apps UI.
PS: Given the project structure I am not supposed to use Ant :(
I found the above answers quite useful but not really general purpose, they all need some other major build system like Ant or Maven.
I wanted to generate a report in a simple one-shot command that I could call from anything (from a build, test or just myself) so I have created junit2html which can be found here: https://github.com/inorton/junit2html
You can install it by doing:
pip install junit2html
Alternatively for those using Maven build tool, there is a plugin called Surefire Report.
The report looks like this : Sample
If you could use Ant then you would just use the JUnitReport task as detailed here: http://ant.apache.org/manual/Tasks/junitreport.html, but you mentioned in your question that you're not supposed to use Ant.
I believe that task merely transforms the XML report into HTML so it would be feasible to use any XSLT processor to generate a similar report.
Alternatively, you could switch to using TestNG ( http://testng.org/doc/index.html ) which is very similar to JUnit but has a default HTML report as well as several other cool features.
You can easily do this via ant. Here is a build.xml file for doing this
<project name="genTestReport" default="gen" basedir=".">
<description>
Generate the HTML report from JUnit XML files
</description>
<target name="gen">
<property name="genReportDir" location="${basedir}/unitTestReports"/>
<delete dir="${genReportDir}"/>
<mkdir dir="${genReportDir}"/>
<junitreport todir="${basedir}/unitTestReports">
<fileset dir="${basedir}">
<include name="**/TEST-*.xml"/>
</fileset>
<report format="frames" todir="${genReportDir}/html"/>
</junitreport>
</target>
</project>
This will find files with the format TEST-*.xml and generate reports into a folder named unitTestReports.
To run this (assuming the above file is called buildTestReports.xml) run the following command in the terminal:
ant -buildfile buildTestReports.xml
Junit xml format is used outside of Java/Maven/Ant word.
Jenkins with http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin is a solution.
For the one shot solution I have found this tool that does the job:
https://www.npmjs.com/package/junit-viewer
junit-viewer --results=surefire-reports --save=file_location.html
--results= is directory with xml files (test reports)
I found xunit-viewer, which has deprecated junit-viewer mentioned by #daniel-kristof-kiss.
It is very simple, automatically recursively collects all relevant files in ANT Junit XML format and creates a single html-file with filtering and other sweet features.
I use it to upload test results from Travis builds as Travis has no other support for collecting standard formatted test results output.
There are multiple options available for generating HTML reports for Selenium WebDriver scripts.
1. Use the JUNIT TestWatcher class for creating your own Selenium HTML reports
The TestWatcher JUNIT class allows overriding the failed() and succeeded() JUNIT methods that are called automatically when JUNIT tests fail or pass.
The TestWatcher JUNIT class allows overriding the following methods:
protected void failed(Throwable e, Description description)
failed() method is invoked when a test fails
protected void finished(Description description)
finished() method is invoked when a test method finishes (whether passing or failing)
protected void skipped(AssumptionViolatedException e, Description
description)
skipped() method is invoked when a test is skipped due to a failed assumption.
protected void starting(Description description)
starting() method is invoked when a test is about to start
protected void succeeded(Description description)
succeeded() method is invoked when a test succeeds
See below sample code for this case:
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class TestClass2 extends WatchManClassConsole {
#Test public void testScript1() {
assertTrue(1 < 2); >
}
#Test public void testScript2() {
assertTrue(1 > 2);
}
#Test public void testScript3() {
assertTrue(1 < 2);
}
#Test public void testScript4() {
assertTrue(1 > 2);
}
}
import org.junit.Rule;
import org.junit.rules.TestRule;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
public class WatchManClassConsole {
#Rule public TestRule watchman = new TestWatcher() {
#Override public Statement apply(Statement base, Description description) {
return super.apply(base, description);
}
#Override protected void succeeded(Description description) {
System.out.println(description.getDisplayName() + " " + "success!");
}
#Override protected void failed(Throwable e, Description description) {
System.out.println(description.getDisplayName() + " " + e.getClass().getSimpleName());
}
};
}
2. Use the Allure Reporting framework
Allure framework can help with generating HTML reports for your Selenium WebDriver projects.
The reporting framework is very flexible and it works with many programming languages and unit testing frameworks.
You can read everything about it at http://allure.qatools.ru/.
You will need the following dependencies and plugins to be added to your pom.xml file
maven surefire
aspectjweaver
allure adapter
See more details including code samples on this article:
http://test-able.blogspot.com/2015/10/create-selenium-html-reports-with-allure-framework.html
I have created a JUnit parser/viewer that runs directly in the browser. It supports conversion to JSON and the HTML report can be easily reused.
https://lotterfriends.github.io/online-junit-parser/
If you are still missing a feature feel free to create an issue on Github. :)
I have:
#OnApplicationStart
public class SomeClass {
.. doJob() ...
}
How I can test it in my Unit Test that doJob() actually launched when application started?
I would argue that this is not a unit test, but an integration test.
You can test your Job, by simply calling it using the syntax new MyJob().now();, but as you are looking to test the #OnApplicationStart function, the you would be better off doing this as a Selenium test, and checking the data that you expect to be made available from the bootstrap job is present.