I'm traying to execute few Jmeter tests using Jmeter maven plugin in my case the tests order can be important, I tried to figure how can I control the different tests order using the Jmeter maven plugin with no success , any idea if it's possible ?
<!-- The flows we want to execute -->
<testFilesIncluded>
<jMeterTestFile>flows/flow1</jMeterTestFile>
<jMeterTestFile>flows/flow2</jMeterTestFile>
<jMeterTestFile>flows/flow3</jMeterTestFile>
</testFilesIncluded>
in this case 3 runs first then 1 and last is 2
The order of execution is not guaranteed, if you need to run tests in particular order either execute your Maven build 3 times providing a single .jmx script or combine these scripts into one "umbrella" script which will be invoking the others stored as Test Fragments via Include Controller or Module Controller.
Related
I have created a custom TestEngine using the JUnit 5 (junit-platform-engine) framework.
The custom TestEngine registers using the ServiceLoader mechanism with an entry in META-INF/services/org.junit.platform.engine.TestEngine.
When I run my tests, this works well, but the tests get run a second time by the built-in JUnitTestEngine.
Is it possible to replace the default TestEngine in this circumstance instead of supplement it?
After checking the JUnit 5 user guide and documentation for maven surefire plugin it seems there's currently no way to filter out certain test engine with Maven :-(.
Using the console launcher, however, does allow to choose test engines: https://junit.org/junit5/docs/current/user-guide/#running-tests-console-launcher-options. And so does Gradle: https://junit.org/junit5/docs/current/user-guide/#running-tests-build-gradle.
I'm trying to get tests results while the job is building.
When we run tests suite by eclipse we get tests results from TestNG viewer while running the suite, I want to get the same viewer or similar in Jenkins to know the current status of the build before finish.
I mean this in TestNG Viewer:
Results of running suite TestNG viewer
Thanks All :)
AFAIK it is not inbuilt as part of any plugin. But there are couple of options that you can try.
Write results to a database in the IInvokedMethodListener after implementation. Build a ui over the database.
Maintain a datastructure of results , do console out of summary(if you are the only one who needs to know the results) on jenkins in test listener or method listener of the results based on the frequency at which you need to know results. Or you can start of a parallel script which parses the consoleText either as a shell or a separate utility doing curl on the consoletext.
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
I have Groovy Script in my Test Case in SOAP UI.
There are one big script and a lot of test steps which are run by this script. I know about functionality of TestRunner but it run all steps in test case, but i need to run only my groovy script. How?
UPDATE
I disable all Test steps and left active only groovy script. Then I Launch TestRunner on Test Case, it return exception :
java.lang.Exception: TestCase [Case1] failed without assertions
at com.eviware.soapui.tools.SoapUITestCaseRunner.throwFailureException(SoapUITestCaseRunner.java:535)
at com.eviware.soapui.tools.SoapUITestCaseRunner.runRunner(SoapUITestCaseRunner.java:437)
at com.eviware.soapui.tools.AbstractSoapUIRunner.run(AbstractSoapUIRunner.java:162)
at com.eviware.soapui.tools.AbstractSoapUIRunner.runFromCommandLine(AbstractSoapUIRunner.java:93)
at com.eviware.soapui.tools.SoapUITestCaseRunner.main(SoapUITestCaseRunner.java:119)
Just disable the rest of the steps and run the test case. The disabled test steps are not run automatically by soapui, you can call those conditionally :)
I have the following set-up: an integration tests project which has a suite of tests written in Groovy/Geb + Spock, which are running perfectly both using Selenium WebDriver and Selenium Grid (RemoteWebDriver).
The problem is that no matter how much I try to tweak the "system", I can't get the tests to run in parallel (i.e. although I have 3 slaves [nodes] registered to the hub, only one of the slaves actually receives the requests). I've enforced maxSession=1 to the Selenium nodes and tried different combinations of parallel=classes|methods, threadCount and fork settings in failsafe plugin configuration (pom.xml file).
I have the feeling that the problem lies somewhere between the maven configuration and selenium grid, probably in relation to Geb/Spock config.
Does any of you have any insight on this issue?
PS: someone suggested that running tests in parallel using Geb / Spock is not possible - because for some reason ?Geb? locks the JUnitRunner (not sure what this means).
Add following configuration to your build.gradle file:
tasks.withType(Test) {
maxParallelForks = 3 // here three forks shall open in parallel
forkEvery = 1
include '**/*TestName*.class' // name of your test class
}
There are test frameworks, TestNG for example, that support parallel testing on the method level out of the box.
Spock, as an example to the contrary, does not support it.
But you do not have to have multithreading implemented by your test framework to make this work.
You can use your build tool to run test classes in parallel, both Maven and Gradle support this.
If you are using Maven, this documentation page and examples might help you:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html
Specifically have a look at "Forked Test Execution".
You can run it for sure, The point is you have to put them (your tests) in threads. Here is the link.