JUnit4 has #FixMethodOrder annotation which allows to use alphabetical order of test methods execution. Is there analogous JUnit5 mechanism?
Edit: JUnit 5.4 is officially released now, so no need to use snapshots anymore.
This is now possible with JUnit 5.4.
https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order
To control the order in which test methods are executed, annotate your
test class or test interface with #TestMethodOrder and specify the
desired MethodOrderer implementation. You can implement your own
custom MethodOrderer or use one of the following built-in
MethodOrderer implementations.
Alphanumeric: sorts test methods alphanumerically based on their names
and formal parameter lists.
OrderAnnotation: sorts test methods numerically based on values
specified via the #Order annotation.
No, not yet. For unit tests, execution order should be irrelevant. For more complex tests, JUnit is aiming to provide explicit support - test ordering would be part of that.
With version 5.8.0 onwards, test classes can be ordered too.
src/test/resources/junit-platform.properties:
# ClassOrderer$OrderAnnotation sorts classes based on their #Order annotation
junit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer$OrderAnnotation
Other Junit built-in class orderer implementations:
org.junit.jupiter.api.ClassOrderer$ClassName
org.junit.jupiter.api.ClassOrderer$DisplayName
org.junit.jupiter.api.ClassOrderer$Random
For other ways (beside junit-platform.properties file) to set configuration parameters refer here.
You can also provide your own orderer. It must implement ClassOrderer interface:
package foo;
public class MyOrderer implements ClassOrderer {
#Override
public void orderClasses(ClassOrdererContext context) {
Collections.shuffle(context.getClassDescriptors());
}
}
junit.jupiter.testclass.order.default=foo.MyOrderer
Note that #Nested test classes cannot be ordered by a ClassOrderer.
Refer to JUnit 5 documentations and ClassOrderer api docs to learn more about ordering test classes.
Related
I use Spock framework to run my tests. Each test class has Spring's annotation #ContextConfiguration over it. As I see context is brought up for every test class and it takes a lot of time to run a couple of dozens of tests in different classes. Is there a way to configure Spock test classes to run under common spring context?
It's not about spock but about spring.
Spring can cache in general application context across many test cases but you have to know how to do it right.
As a first resort, make sure, that 'locations' attribute of #ContextConfiguration is the same.
In addition, it's possible to take advantage of #ContextHierarchy annotation.
This caching facility is quite fragile, but works.
Maybe it makes sense to create a common specification with all the annotation and inherit from it:
#ContextConfiguration(locations = ...)
// or maybe
#ContextHierarchy (...)
public abstract class MyCommonSpec extends Specification {
}
// and not in tests:
public class MyTest1 extends MyCommonSpec {
...
}
public class MyTest2 extends MyCommonSpec {
...
}
Here is a link on a very relevant discussion
Possible approach that will work: create multiple classes with tests. For example, A, B, C, D. Make inheritance like: D -> C -> B -> A. Run tests in D, this will execute all the tests from A,B,C,D under common spring context.
I assume that this approach is not so neat and perfect, but it solves initial problem.
I am supposed to migrate on Cucumber. I do have project framework with Selenium, TestNG with Data Driven Framework, Maven. I am exploring Cucumber feasibility with TestNG annotation.
My question is, How we can create connection between #Test method and Step definition of cucumber. Let's example our code is written in #BeforeClass, #Test, #AfterClass method. So how we can migrate with Step definition.
Feature File :
Feature: Is it Friday yet?
Everybody wants to know when it's Friday
Scenario: Sunday isn't Friday
Given today is Sunday
When I ask whether it's Friday yet
Step Definition:
#Given("^today is Sunday$")
public void today_is_Sunday() {
// Write code here that turns the phrase above into concrete actions
System.out.println("this is demo1");
}
#When("^I ask whether it's Friday yet$")
public void i_ask_whether_is_s_Friday_yet() {
// Write code here that turns the phrase above into concrete actions
System.out.println("this is demo2");
}
Class Exection:
#CucumberOptions(features = "cfeature/firstDemo.feature", glue = { "mytest/Stepd" })
public class demo01 extends AbstractTestNGCucumberTests {
private TestNGCucumberRunner tcr;
#BeforeClass(alwaysRun = true)
public void beforeClass() throws Exception {
tcr = new TestNGCucumberRunner(this.getClass());
}
#Test(groups="cucumber", description="Runs CucumberFeature")
public void testdemo() {
System.out.println("Hello");
}
#AfterClass(alwaysRun = true)
public void afterClass() {
tcr.finish();
}
}
Console:
Hello
[33mUndefined scenarios:[0m
[33mcfeature/firstDemo.feature:4 [0m# Sunday isn't Friday
1 Scenarios ([33m1 undefined[0m)
5 Steps ([33m5 undefined[0m)
0m0.073s
You can implement missing steps with the snippets below:
As of now, #Test annotation is calling. But, How to replace it with Step Definition. Please assist.
Not sure what the confusion here. Here's how you can relate TestNG and cucumber terminologies.
<test> tag in TestNG can be visualized as a feature file in cucumber.
#Test method in TestNG can be visualized as a scenario in cucumber.
A Step definition in cucumber has nothing directly equivalent to in TestNG because, its part of a scenario. But for the sake of understanding you can visualize it as one line of code doing a logical operation in TestNG.
The default implementation of AbstractTestNGCucumberTests is as below:
It contains a data provider internally which provides one feature file at a time.
It contains a #Test method which is bound to the above mentioned data provider, which retrieves all the scenarios in the feature file and then runs them one after the other.
You can build your own variant of AbstractTestNGCucumberTests to do various different things (such as support concurrent scenario execution which is currently not available in Cucumber JVM bindings).
As an example you can take a look at Cucumber-roadrunner library that I built which uses the above concept to support parallel scenario execution and also provides thread safe reports.
With respect to the error you are facing viz., You can implement missing steps with the snippets below: is basically because cucumber jvm bindings perhaps isn't able to bind your feature file with a glue code (which is what you are providing via the #CucumberOptions annotation). You should perhaps take a closer look at the cucumber jvm bindings documentation to understand how to provide the correct values.
You can also take a look to gherkin with QAF which is pure TestNG implementation for gherkin. It is using TestNG (NOT cucumber runner) and provides you all the features of testNG including parallel execution, listeners, grouping, priority etc...
Each scenario converted as TestNG test and you can run scenarios as parallel. Furthermore you can also use inbuilt or custom data-providers while authoring BDD. No need additional runner just configure as usual using appropriate factory class for the BDD syntax you are using.
I am not sure if what i am doing is actually the "correct" way of doing unit tests with DI. Right now i ask my ViewModelLocator to actually create all the instances i need, and just get the instance i need to test, which makes it very simple to test a single instance because lets asume that Receipt needs a Reseller object to be created, reseller needs a User object to be created, user need some other object to be created, which creates a chain of objects to create just to test one single instance.
With di usally interfaces will get mocked and parsed to the object which you would like to create, but how about simple Entities/ViewModels?
Whats the best practice to do unit testing with DI involved?
public class JournalTest
{
private ReceiptViewModel receipt;
private ViewModelLocator locator;
[SetUp]
public void SetUp()
{
locator = new ViewModelLocator();
receipt = SimpleIoc.Default.GetInstance<ReceiptViewModel>();
}
[TearDown]
[Test]
public void CheckAndCreateNewJournal_Should_Always_Create_New_Journal()
{
receipt.Sale.Journal = null;
receipt.Sale.CheckAndCreateNewJournal();
Assert.NotNull(receipt.Sale.Journal);
}
}
First, you aren't using Dependency Injection in your code. What you have there is called Service Locator (Service Locators create a tight coupling to the IoC/Service Locator and makes it hard to test).
And yes, it's bad (both Service Locator and Dependency Injection), because it means: You are not doing a UnitTest, you are doing an integration Test.
In your case the ReceiptViewModel will not be tested alone, but your test also tests the dependencies of ReceiptViewModel (i.e. Repository, Services injected etc.). This is called an integration test.
A UnitTest has to test only the class in question and no dependencies. You can achieve this either by stubs (dummy implementation of your dependencies, assuming you have used interfaces as dependencies) or using mocks (with a Mock framework like Moq).
Which is easier/better as you don't have to implement the whole class, but just have to setup mocks for the methods you know that will be required for your test case.
As an additional note, entities you'll got to create yourself. Depending on your UnitTest framework, there may be data driven tests (via Attributes on the test method) or you just create them in code, or if you have models/entities used in many classes, create a helper method for it.
View Models shouldn't be injected into constructor (at least avoided), as it couples them tightly
Units tests should run quickly and should be deterministic. That means you have to mock/stub everything that brokes these two rules.
The best way to mock/stub dependancies is to inject them. In the production, classes are assembled by DI framework, but in unit tests you should assemble them manually and inject mocks where needed.
There is also a classic unit test approach where you stub/mock every dependency of your class, but it's useless since you don't gain anything by that.
Martin Fowler wrote great article about that: link
You should also read Growing Object-oriented software: Guided by tests. Ton of useful knowledge.
I am working with testNG where I run an external test framework, receive the result data and assert it. To run the external test framework I need to set up a specification for which tests that should be run. To generate this specification I need to know which tests that are selected in the testNG .xml file.
The only way I could think of doing this is to parse the file manually. But I am hoping for a better solution than this.
Thanks for any answers!
//Flipbed
Edit:
My colleague found solutions to the problem.
In #Factory and #DataProvider annotated methods it is possible to add a parameter of the type ITestContext. Using the variable of that type, one can use the method .getAllTestMethods().
Create a new class that implements IMethodInterceptor. In this class one can override the method 'intercept'. The method takes a parameter of the type List which is a list of all methods that will be run by testNG.
If someone has any other suggestions feel free to add.
//Flipbed
The solution that we used was to number 2 in my edit. We implemented the IMethodInterceptor and used the methods list as well as the ITestContext to both view what tests will run and modify that list.
I'm currently extending Degraph to check for violation of dependency rules.
So far I have created matchers for ScalaTest, since this is my favorite test library for the JVM. But many others prefer or have to use other libraries. So I'd like to provide matchers (or whatever is the appropriate equivalent) for those as well.
On the other hand I'm lazy so I don't want to provide a matcher for every conceivable library.
So the question is:
Is there some kind of matcher library that can be used in all (or the most / many) test libraries? Or maybe a wrapper that converts Hamcrest matchers to matchers for all the other libs?
The libraries I would like to support (in that order of importance):
JUnit
ScalaTest
most popular testing library for Groovy
most popular testing library for Clojure
Specs2
TestNG
Every unit testing framework I've seen will fail a test that throws an exception.
Hamcrest's assertThat(T value, Matcher<T> matcher) will throw an AssertionError if matcher.matches(value) returns false.
Therefore, Hamcrest should work out-of-the-box with most testing frameworks. For other libraries, such as for building mock objects, you'll need to either use Hamcrest's Matcher objects natively or write an integration layer. One way would be to create a method to decorate any matcher, for example*,
ArgumentMatcher<T> decorate(final Matcher<T> matcher) {
return new ArgumentMatcher() {
public boolean <T> accepts(T value) {
return matcher.matches(value);
}
}
}
* This is a contrived example using a theoretical Mocking API.