I'm trying to port Test Management For Jira JUnit Integration to JUnit5. This module generates a JSON report of the test run and associates the results with Jira tickets by using annotations on the test methods, example.
From the TestExecutionListener I'm not sure what the best approach to retrieve the TestCase annotation is.
I looked at Reflection using the TestIdentifier.getSource and doing manipulations to rebuild the method signature and extracting the annotation from there but the approach felt clumsy.
I came across this post Allow extensions to register TestExecutionListeners which proposed the following:
Proposal: Have your extension publish the WebDriver bean's session id, e.g.
String sessionId = ...;
extensionContext.publishReportEntry("webDriverSessionId", sessionId)
In your TestExecutionListener, implement reportingEntryPublished and store it in a Map with the TestIdentifier as a key. In executionFinished report the test outcome along with the value from this Map.
This approach looks promising but I want to make sure there isn't another way that doesn't require both an extension and a test execution listener. Is there a way to retrieve test method annotation information directly in the TestExecutionListener?
#Alex, the following might be used inside the listener...
((MethodSource) testIdentifier.source).javaMethod.getAnnotation(TestCase.class)
Seems like you can't get test annotation from TestExecutionListener but instead you can implement TestWatcher or e.g AfterEachCallback and get custom annotation value like that:
import org.junit.jupiter.api.extension.AfterEachCallback;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestWatcher;
public class MyExtention implements TestWatcher, AfterEachCallback {
#Override public void testSuccessful(ExtensionContext context) {
if (context.getElement().isPresent() && context.getElement().get().isAnnotationPresent(MyCustomAnnotation.class)) {
int val = context.getElement().get().getAnnotation(MyCustomAnnotation.class).value();
// Report on success
}
}
#Override public void afterEach(ExtensionContext context) throws Exception {
if (context.getElement().isPresent() && context.getElement().get().isAnnotationPresent(MyCustomAnnotation.class)) {
int val = context.getElement().get().getAnnotation(MyCustomAnnotation.class).value();
// Report each
}
}
}
Related
I am using serenity BDD for my automation testing and Page Object Model for my framework. I have created a BasePage class which will be inherited by all the other Pages. I want to minimize the logging messages from the Pages by adding all the log.info messages to a central Base page. Example, when calling the click() method, I will log before click and after click methods as shown below in the basePage class:
public class BasePage extends PageObject{
private static final Logger log = LogManager.getLogger(BasePage.class.getClass());
private final WebElementFacade element;
public static void clickBtn(WebElementFacade btnName) {
log.info("About to click " + btnName + " button");
btnName.click();
log.info("Successfully clicked on " + btnName + " button.");
}
Later I figured that instead of individually trying to figure out in advance what actions the user will perform on the webElements, and write new methods for each action (like the one shown above), just implement WebDriverFacade interface, so that I get all the unimplemented method list in BasePage from WebDriverFacade and then write the log messages inside each of them, like so:
public class BasePage extends PageObject implements WebElementFacade{
private static final Logger log = LogManager.getLogger(BasePage.class.getClass());
private final WebElementFacade element;
#Override
public void submit() {
// TODO Auto-generated method stub
}
#Override
public void sendKeys(CharSequence... keysToSend) {
// TODO Auto-generated method stub
}
#Override
public String getTagName() {
// TODO Auto-generated method stub
return null;
}
#Override
public boolean isSelected() {
// TODO Auto-generated method stub
return false;
}
.
.
.
.
.
}
This will serve two purposes:
I will not have to create new methods for every action in BasePage class, example the 'clickBtn()' function in the first code
As I mentioned before, I will not have to figure out what method any other person who adds methods to my code might use and having to change the BasePage class to create the new actions. So basically less maintenance in the long run.
The problem I am facing is an error that I receive in the second use case:
The return types are incompatible for the inherited methods WebElementFacade.withTimeoutOf(int, TimeUnit), PageObject.withTimeoutOf(int, TimeUnit)
Now my question is:
How can solve this problem?
Is this the right way to do things or should I be going with the first method and have maintenance overhead.
Just figured that another scenario where this might be useful. To make sure that subclass methods do not use methods from pageObject and are forced to use the methods from BaseClass only. This can be done by wrapping the WebElementFacade and adding the log messages as an added functionality. Any thought on this would be appreciated.
Thank you!
Honestly it is a neat trick and if you get it working you should be proud.
I think I figured something similar out in a dynamic language.
But you are better off just adding the logging entries and learning the following.
How to name functions so you don't feel like they need renaming.
How to log clearly for debugging use.
This is because loggings power is in its flexibility.
When you learn how to dump something complex like a matrix so you can eye it and go uh-oh you are increasing your overall skills.
I apologize for not giving you code but I felt some "chase the other rabbit" advice was better.
I have a test class that has several tests. At the moment I have this to start up the server, wipe the database etc:
#ClassRule
public static final DropwizardAppRule<ServiceConfig> RULE =
new DropwizardAppRule<ServiceConfig>(ServiceApp.class, ResourceHelpers.resourceFilePath("config.yml"));
All my tests work with this individually. But when I run them all together some fail since other tests modify data. I tried doing the following but I'm getting null pointers when calling RULE.getPort():
#ClassRule
public static DropwizardAppRule<ServiceConfig> RULE;
#Before
public void beforeClass() {
RULE = new DropwizardAppRule<ServiceConfig>(ServiceApp.class, ResourceHelpers.resourceFilePath("config.yml"));
}
I would have expected this to work but it doesn't seem to set the values of RULE properly. Any ideas?
Hi,
I don't know how to handle db "from within" DropwizardAppRule, so I may not really answer your question... I'm actually having another issue myself trying with DropwizardAppRule not properly being setup and torn down between tests. (So if you made progress going this way I'd like you insights).
Anyway, I think you need to handle your DB outside DropwizardAppRule and give it in the Rule. We resolved DB clearing by relying on custom and external TestsRules:
public class CockpitApplicationRule implements TestRule {
public static class App extends CockpitApplication<CockpitConfiguration> {
// only needed because of generics
}
public final DropwizardAppRule<CockpitConfiguration> dw;
public final EmbeddedDatabaseRule db;
public CockpitApplicationRule(String config, ConfigOverride... configOverrides) {
this.db = EmbeddedDatabaseRule.builder()
.initializedByPlugin(LiquibaseInitializer.builder().withChangelogResource("migrations.xml").build())
.build();
this.dw = new DropwizardAppRule<>(App.class, ResourceHelpers.resourceFilePath(config),
ConfigOverride.config("database.url", () -> this.db.getConnectionJdbcUrl()));
}
#Override
#Nullable
public Statement apply(#Nullable Statement base, #Nullable Description description) {
assert base != null;
assert description != null;
return RulesHelper.chain(base, description, dw, RulesHelper.dropDbAfter(db), db);
}
public DSLContext db() {
return DSL.using(db.getConnectionJdbcUrl());
}
}
Basically we override TestRule apply(...) to chain custom Statements. There's our RulesHelper if you want to take a look. That way the DB is cleanly handled by the Rules, we can fill our test DB in test classes using #Before setup methods.
org.zapodot.junit.db.EmbeddedDatabaseRule Is an external dependency that allows us to rather easily instantiate a DB for our tests.
The RulesHelper.dropDbAfter does the actual cleaning:
public static TestRule dropDbAfter(EmbeddedDatabaseRule db) {
return after(() -> DSL.using(db.getConnectionJdbcUrl()).execute("DROP ALL OBJECTS"));
}
You should be able to setup and clean the DB from #Before and #After methods without fully using TestRules though, but I'm not sure it's really easier in the end.
Hope this helped !
I am new to both libraries and before committing to their usage on a large project I need clarification on my options for low-code effort automocking in my unit tests.
After spending some time on Google I have concluded that, unlike some other IOC/Mocking product pairings, a ready-made plugin library is not available for LightInject+Nsubstitute to simplify the declaration of do-nothing default mocks in the arrange stage of a unit test.
I have read the LightInject docs on how to override a LightInject container with a temporary enhanced mock object just for the scope of a unit test but what about all the do-nothing default isolation mocks that a unit test might touch. Is there a way to automate their creation within the LightInject container?
The internal IOC container behaviour I am looking for is:
public class LightInject.ServiceContainer
{
..
public T GetInstance<T)
{
if (( this.RegisteredInterfaces.Any( i => i.Itype == T ) == false )
&& ( this.TemporaryUnitTestOverrides.Any( i => i.Itype == T ) == false ))
&& ( /* this container is configured with an automocking delegate */ ))
return autoMockCreatorDelegate<T>.Invoke();
}
It seems like LightInject's IProxy and Interceptors provide some internal mock object building blocks but the Nsubstitute library is full featured in comparison.
Clarification on what I mean by default do nothing mock and an enhanced mock.
// default do nothing mock
var calculator = Substitute.For<ICalculator>();
// Enhanced mock that will return 3 for .Add(1,2)
var calculator = Substitute.For<ICalculator>();
calculator.Add(1, 2).Returns(3);
Obviously the second enhanced type of mock will need to be crafted locally per unit test.
I am the author of LightInject and would really like to help you out.
Let me look into this and get back to you. In the meanwhile you might want to check out this library at
LightInject.AutopMoq which is a third party contribution to the LightInject container. It uses Moq instead of NSubstitute, but the general concept should be similar to what you are asking for.
That being said, I did some work a while ago that simplifies automocking even further and will take a look at it it and see how that can be integrated with NSubstitute.
Edit
This is a super simple automocking implementation that works with any "substitute" framework.
using System.Diagnostics;
using LightInject;
using NSubstitute;
public interface IFoo { }
class Program
{
static void Main(string[] args)
{
var serviceContainer = new ServiceContainer();
serviceContainer.RegisterFallback((type, s) => true, request => CreateMock(request.ServiceType));
var foo = serviceContainer.GetInstance<IFoo>();
Debug.Assert(foo is IFoo);
}
private static object CreateMock(Type serviceType)
{
return Substitute.For(new Type[] { serviceType }, null);
}
}
Best regards
Bernhard Richter
Some feedback as promised in my comment to the accepted answer. I applied the suggestion from the author of LightInject with success in some simple unit tests.
After getting the basics working I decided to hide the Ioc service mocking setup code in a base class plus something I have called a MockingContext, the end result is cleaner lighter unit test code. The mocking context class also ensures that foreach Nsubstitute configured mock type passed to the Ioc service as a short term automock override, there is a matching LightInjet.Service.EndMocking( T ) call. This removes the danger that configured mocks might pollute the auto mocking assumptions of a following unit test.
In the example ClassC depends on IFooA and IFooB (no constructor injection). For the unit test below, IFooA is auto mocked by LightInject without explicit code whereas IFooB is configured via an Nsubstitute call and also passed to LightInject in the MockingContext.Add<>() method.
[TestClass]
public class UnitTest1 : AutoMocking
{
[TestMethod]
public void Test_1()
{
using (var mc = MockingContext)
{
// No need to mention IFooA here, LightInject will auto mock
// any interface not previously declared to it.
// Given
var mockB = mc.Add<IFooB>();
mockB.MethodY().Returns("Mock Value OOO");
var sut = new ClassC();
// When
var testResult = sut.MethodZ();
// Then
Assert.AreEqual(testResult, "MethodZ() received=Mock Value OOO");
}
}
I am trying to integrate TestLink with TestNG
Approach is below
1>Write ITestListner with onTestFailure and onTestSuccess
2> get Annotation of the method(like testName which will be equivalent to test name in testlink) which is being failed/success in a variable
3>Make connection with TestLink using API available and update the test case.
However I am struggling to find method Annotation value in ITestListner and requirement is to get annotation values in ITestListner only so that correct test cases can be updated in Test_link
Can someone please help me how to get Test Method annotation value in ITestListner or any other approach in which i can integrate testlink update with TestNG
Hi Thanks niharika for help
,First of all you are correct in explaining use of TestNG but we are using TestNG for Selenium and already there are around 1000 test cases writen in test Methods and we have to live with that
Some how i have figured the solution ,we can still get the testName of the test method using two listners
This is just work around I am not sure if this is the best approach but as of now solving my purpose
package com.automation.testng.listner;
import org.testng.*;
public class MyIInvokeMethodListner_TestName_TestLink implements IInvokedMethodListener {
public static String testName;
public void afterInvocation(IInvokedMethod arg0, ITestResult arg1) {
// TODO Auto-generated method stub
}
public void beforeInvocation(IInvokedMethod m, ITestResult tr) {
// TODO Auto-generated method stub
//This give the Annotation Test object
org.testng.annotations.Test t=m.getTestMethod().getMethod().getAnnotation(org.testng.annotations.Test.class);
MyIInvokeMethodListner_TestName_TestLink.testName = t.testName().toString();
}
}
MyITestListner goes like below
package com.automation.testng.listner;
import org.testng.*;
public class MyITestListner_TestLink extends TestListenerAdapter {
/*IAnnotationTransformer at;
public Listner_1()
{
this.at = new Annotation_listner();
}*/
#Override
public void onTestFailure(ITestResult tr)
{
System.out.println("Hurray !I am being inboked from Test listner");
MyIInvokeMethodListner_TestName_TestLink a = new MyIInvokeMethodListner_TestName_TestLink();
System.out.println(MyIInvokeMethodListner_TestName_TestLink.testName);
}
public void onTestSuccess(ITestResult tr)
{
MyIInvokeMethodListner_TestName_TestLink a = new MyIInvokeMethodListner_TestName_TestLink();
System.out.println(MyIInvokeMethodListner_TestName_TestLink.testName);
}
}
Basically we are getting the method and then using Test Annotation class setting the static variable which can be used in MyITestListner
The ITestListener is the one which is used after <test> tag. For getting the method name and annotation specifics, you need to implement IInvokedMethodListener and in the after/before methods of this interface, and use something like method.getTestMethod().getMethodName() to get the executing method name.
If you are adding testName at the method level, I think you are doing it wrong since the help of testng mentions this "The name of the test this test class should be placed in. This attribute is ignore if #Test is not at the class level."
If you are indeed specifying the #Test at your class level then you can get it as below :
method.getTestMethod().getTestClass().getTestName()
A bit ugly and you probably want to wrap those parts in null checks in your code but this is how you get the testName specified in the annotation from the ITestResult:
iTestResult.getMethod().getConstructorOrMethod().getMethod().getAnnotation(Test.class).testName()
I have a TestNG listener that implements IInvokedMethodListener. On #BeforeMethod I need to setup some test context, here is the example:
public class ThucydidesInvokedMethodListener implements IInvokedMethodListener2 {
public void beforeInvocation(final IInvokedMethod method, final ITestResult testResult) {
boolean areBeforeMethods = method.getTestMethod().getTestClass().getBeforeTestMethods().length > 0;
if ((areBeforeMethods && method.getTestMethod().getTestClass().getBeforeTestMethods()[0] == method.getTestMethod()) ||
!areBeforeMethods && method.isTestMethod()) {
final ThucydidesTestContext context = new ThucydidesTestContext(testResult);
testResult.setAttribute(contextKey(), context);
context.before();
}
}
but also I need a test name that will be executed after the BeforeMethod to use this test name in the reports. Is this possible using TestNG?
Also I've tried IInvokedMethodListener2 that additionally has ITestContext, but it doesn't provide the test name as well.
Using a listener for configuring your tests sounds wrong to me - that is what the #Before* annotations are for.
I do not know how to get your desired information with a listener, but with #BeforeMethod it is simple: Just add a parameter of type java.reflect.Method to your method signature and TestNG will inject the current method which you can then ask for its name and everything else you want to know.
All "magic' for the TestNG annotations is documented here:TestNG dependency injection
HTH
/Jens
import java.lang.reflect.Method;
public class TestToGetMethodName
{
#BeforeMethod
public void handleTestMethodName(Method method)
{
String testName = method.getName();
}
}
Well, with IInvokedMethodListener, the beforeInvocation method gives you IInvokedMethod method.
method.getTestMethod.getMethodName() give you method name.