I am working with testNG where I run an external test framework, receive the result data and assert it. To run the external test framework I need to set up a specification for which tests that should be run. To generate this specification I need to know which tests that are selected in the testNG .xml file.
The only way I could think of doing this is to parse the file manually. But I am hoping for a better solution than this.
Thanks for any answers!
//Flipbed
Edit:
My colleague found solutions to the problem.
In #Factory and #DataProvider annotated methods it is possible to add a parameter of the type ITestContext. Using the variable of that type, one can use the method .getAllTestMethods().
Create a new class that implements IMethodInterceptor. In this class one can override the method 'intercept'. The method takes a parameter of the type List which is a list of all methods that will be run by testNG.
If someone has any other suggestions feel free to add.
//Flipbed
The solution that we used was to number 2 in my edit. We implemented the IMethodInterceptor and used the methods list as well as the ITestContext to both view what tests will run and modify that list.
Related
Im porting some Python code using hypothesis, and trying to keep the sources as close as possible. The python test file has both parameterized and non-parameterized methods. If I mark them all as #Property, the non-parameterized (and so identical) methods get called 1000 times.
Just learning jqwik, so may be missing something easy. If not, ill just break them into two files. Thanks.
For example based tests jqwik has the annotation #Example, which will run your test method just a single time. Use it for your non-parameterized tests.
I would like to write a JUnit5 Extension where I have to take some action when a test method annotated with #Disabled is found. Unfortunately, beforeTestExecution() is not called for such methods. Does anybody have an idea how to intercept such #Disabled test methods ?
Thanks !
As described in the User Guide, you can disable the built-in ExecutionCondition that handles #Disabled by default by setting junit.jupiter.conditions.deactivate to org.junit.*DisabledCondition (see Configuration Parameters on how to set it). This will cause your tests to be executed.
Next, you need to implement your own ExecutionCondition extension, check for #Disabled, take your action and return ConditionEvaluationResult.disabled("...").
In order to avoid having to register your extension on each test class, you can activate Automatic Extension Registration and register your extension globally.
Depending on what you want to achieve, it may be easier to register your own TestExecutionListener (see Plugging in Your Own Test Execution Listeners) and implement executionSkipped().
The extension point used here is https://github.com/junit-team/junit5/blob/master/junit-jupiter-api/src/main/java/org/junit/jupiter/api/extension/ExecutionCondition.java
It might be interesting to find out how several implementors compose, i.e. if the 2nd one will be called at all and under what circumstances. You’ll probably have to dive into Jupiter code or do some experiments.
Is there any way to get more debugging information? It doesn't always happen but it seems to almost always happen if I try to immediately rerun tests. I know it's not a classpath issue b/c the tests will run often.
If you created a class with annotations, then don't use/find any page elements before the first annotation.
Testng will looks for the code to instantiate then only to the methods.
Hope this helps.
When I execute the following experimental code subset at http://groovyconsole.appspot.com/
class FileHandler {
def rootDir
FileHandler(String batchName) {
rootDir = '.\\Results\\'+batchName+'\\'
}
}
//def fileHandler = new FileHandler('Result-2012-12-15-10-48-55')
An exception results:
java.lang.NoSuchMethodException: FileHandler.<init>()
When I uncomment the last line that instantiates the class, the error goes away.
Can someone explain why this is? I'm basically attempting to segregate the definition and instantiation of the class into 2 files to be evaluated separately. Thanks
I'm not sure of the exact implementation details behind http://groovyconsole.appspot.com/ (source linked to from there points to Gaelyk, which I've not looked over). I'd bet it's looking for a no-arg constructor for the class you've presented, in an effort to find something runnable. (note that if you provide that, it still won't work, as it wants a main() :/)
Running locally in groovyConsole will die a bit sooner, with the following error message:
groovy.lang.GroovyRuntimeException: This script or class could not be run. It should either:
- have a main method,
- be a JUnit test or extend GroovyTestCase,
- implement the Runnable interface,
- or be compatible with a registered script runner.
This is perhaps more descriptive and to the point. If you want to run some Groovy as a simple script, you'll need to supply a jumping-in point. The easiest way is an executable statement in your groovy file, outside of any class definition (e.g, uncommenting your instantiation statement). Alternatively, a class with a main method should do it. (see here).
If 2 files is how you want to break it up, you can save the class file def in one groovy file (e.g., First.groovy) and create a second (e.g., Second.groovy) with just your executable statements. (I believe the first one will be in the classpath automatically when you run groovy Second, if both are in same directory)
I have an existing assembly that contains hundreds of classes. I'd like to run Pex against one class at a time but I can't figure out a way to do so. Any help in this regard would be greatly appreciated...
I tend to use the Unit Test Wizard that is built into the visual studio test tools to create a test for me for a particular class and methods when i want to do this. (Tests->New Test -> choose unit test wizard)
All a test class is is a class that has the [TestClass] attribute ... if you want a pex class just add the [PexClass] attribute.
Then you add your parameterized unit tests using the [PexMethod] attribute, it will generate tests wit the [TestMethod] attribute for you.