IncrediBuild states about new tests distribution feature. How to use it?
How to distribute gtest tests execution using Incredibuild?
cd <test_dir>
xgconsole /test=gtest /command="<the_test.exe>"
This causes to launch the exe tests on multiple nodes.
I am not sire if this is still relevant but IncrediBuild has an official unit testing solution so please refer to the following link:
https://docs.incredibuild.com/win/latest/windows/ibtestconsole_interface.html?Highlight=gtest
As you can see, there is a new dedicated command - ibtestconsole in order to invoke the tests.
Related
I'm trying to build some tests with googleTest which should test some MPI parallel code. Ideally I want our CI server to execute them through ctest.
My naive approach was to simply call ctest with MPI:
mpirun -n 3 ctest
However, this results in even trivial tests failing as long as at least two are executed.
Example of a trivial tests:
TEST(TestSuite, DummyTest1) {
EXPECT_TRUE(true);
}
TEST(TestSuite, DummyTest2) {
EXPECT_TRUE(true);
}
Am I supposed to launch ctest in some other way? Or do I have to approach this completely differently?
Additional info:
The tests are added via gtest_discover_tests().
Launching the test executable directly with MPI (mpirun -n 3 testExe) yields successful tests but I would prefer using ctest.
Version info:
googletest: 1.11.0
MPI: OpenMPI 4.0.3 AND MPICH 3.3.2
According to this CMake Forum post, which links to a Merge Requests and an issue in the CMake GitLab repository, gtest_discover_tests() currently simply doesn't support MPI.
A possible workaround is to abuse the CROSSCOMPILING_EMULATOR property to inject the wrapper into the test command. However, this changes the whole target and not only the tests so your mileage may vary.
set_property(TARGET TheExe PROPERTY CROSSCOMPILING_EMULATOR '${MPIEXEC_EXECUTABLE} ${MPIEXEC_NUMPROC_FLAG} 3')
gtest_discover_tests(TheExe)
See the forum post for a full minimal example.
I am having 3 feature file and I am trying to execute specific feature in karate using
#CucumberOptions(features = "classpath:Karate/Karate/APIM_LAYER.feature") over the test runner class.
But on execution, we are able to find reports for all the 3 feature files present in the "target/sure-fire reports path".
Please let us know is there a way to resolve this issue.
You should upgrade to V0.6.2, and when you run with #RunWith(Karate.class) with Cucumber options it will run those files sequentially and generate pretty html reports for each feature file.
As for the location of the reports, its usually mentioned in the console/terminal.
So make a testfolderunner.java file. add your cucumber options and then from terminal do mvn test -D test=testfolderunner
Al the best
I'm sure you still have the #RunWith(Karate.class) annotation even though it is clearly mentioned in the documentation that you should NOT use it for the parallel runner. Please confirm.
Currently I'm using maven-failsafe-plugin to run multiple feature files in parallel with Selenium Grid + nodes ( all running in docker containers )
I'm basic questions as below
when to use cucumber-jvm-parallel-plugin ?
what benefits / disadvantages of cucumber-jvm-parallel-plugin over maven-failsafe-plugin parallel execution ?
Thanks in advance.
Below is the answer
"A common approach for running Cucumber features in parallel is to create a suite of Cucumber runners, one for each suite of tests you wish to run in parallel. For maximum parallelism, there should be a runner per feature file.
This is a pain to maintain and not very DRY."
https://github.com/temyers/cucumber-jvm-parallel-plugin/issues/139
additional details -
..Despite its name cucumber-jvm-parallel-plugin does not run any tests, it just automatically generates the Cucumber runners for you. This saves you time from making empty runner files and also just by changing the config of the cucumber-jvm-parallel-plugin you can have brand new set of runner files..
copied from http://automationrhapsody.com/running-cucumber-tests-in-parallel/#comment-3455579028
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
I have the following set-up: an integration tests project which has a suite of tests written in Groovy/Geb + Spock, which are running perfectly both using Selenium WebDriver and Selenium Grid (RemoteWebDriver).
The problem is that no matter how much I try to tweak the "system", I can't get the tests to run in parallel (i.e. although I have 3 slaves [nodes] registered to the hub, only one of the slaves actually receives the requests). I've enforced maxSession=1 to the Selenium nodes and tried different combinations of parallel=classes|methods, threadCount and fork settings in failsafe plugin configuration (pom.xml file).
I have the feeling that the problem lies somewhere between the maven configuration and selenium grid, probably in relation to Geb/Spock config.
Does any of you have any insight on this issue?
PS: someone suggested that running tests in parallel using Geb / Spock is not possible - because for some reason ?Geb? locks the JUnitRunner (not sure what this means).
Add following configuration to your build.gradle file:
tasks.withType(Test) {
maxParallelForks = 3 // here three forks shall open in parallel
forkEvery = 1
include '**/*TestName*.class' // name of your test class
}
There are test frameworks, TestNG for example, that support parallel testing on the method level out of the box.
Spock, as an example to the contrary, does not support it.
But you do not have to have multithreading implemented by your test framework to make this work.
You can use your build tool to run test classes in parallel, both Maven and Gradle support this.
If you are using Maven, this documentation page and examples might help you:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html
Specifically have a look at "Forked Test Execution".
You can run it for sure, The point is you have to put them (your tests) in threads. Here is the link.