This question already has answers here:
GatlingSystem-akka.actor.default-dispatcher-2 error - crashed with 'j.l.NullPointerException' when running Karate Gatling test
(2 answers)
Closed 1 year ago.
With a recent change to our codebase we're starting to get the following error.
Caused by: java.lang.UnsupportedOperationException: There were no requests sent during the simulation, reports won't be generated
We changed nothing with our gatling setup compared to our master branch in which the tests do continue working fine.
The only meaningful change we made in the feature files was the usage of custom java code to fetch an api-key stored in our secret manager.
Said code is called so:
* def AwsUtil = Java.type('be.telenet.yelo.qrcode.util.AwsUtil');
* def apiKey = AwsUtil.getParameterFromSSMByName(ssmApiKeyName, proxyFlag)
Behind this function is a simple AWS SDK getParameter call which works fine in the regular karate cucumber runs and the tests seem indeed to run as well during the karate run because we are getting results:
Karate version: 0.9.6
======================================================
elapsed: 9.11 | threads: 5 | thread time: 36.70
features: 3 | ignored: 0 | efficiency: 0.81
scenarios: 15 | passed: 15 | failed: 0
======================================================
Yet at the end of the run gatling says no calls have been made.
Some more config for our gatling setup:
<profile>
<id>load</id>
<activation>
<property>
<name>load</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
<configuration>
<simulationsFolder>src/test/java</simulationsFolder>
<includes>
<include>KarateSimulation</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
Is there anyone who has experienced similar issues and who might be able to help?
Kind regards
My guess is that some library in the AWS utils conflicts with some of the Gatling ones. It may require some digging into JAR dependency trees.
One hint - if the API key retrieval is an HTTP call behind the scenes, guess what Karate is really good at.
Related
I have an one feature file which have multiple scenarios , which is used to create users account so that it can be used in followed features ,
I have setup to run the featured in paralled using surefire plugins
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<parallel>methods</parallel>
<useUnlimitedThreads>false</useUnlimitedThreads>
<threadCount>1</threadCount>
<perCoreThreadCount>false</perCoreThreadCount>
</configuration>
</plugin>
Now the problem is that since its running 2 feature file in paralled , 2nd feature file failed since user is not created at that time ,
How can I manage this flow of execution ?
Using Junit - Cuccumber.
Thanks in advance.
I had create an UI BDD framework and using testNG as a runner.
Problem I am facing is , I have 3 feature file with in total 12 scenario so far , so when I run in parallel , 12 browser open up , because of which I am getting timeout exception , I tried to search entire www , and done changes in my pom file but still same.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven.surefire.version}</version>
<configuration>
<includes>
<include>**/ParallelRun.java</include>
</includes>
<parallel>classes</parallel>
<threadCount>4</threadCount>
<perCoreThreadCount>false</perCoreThreadCount>
<forkCount>3</forkCount>
</configuration>
</plugin>
Runner Class
public class ParallelRun extends AbstractTestNGCucumberTests {
#Override
#DataProvider(parallel = true)
public Object[][] scenarios() {
return super.scenarios();
}
}
I believe that the core of your problem is not in surefire config but in your tests.
Anyway I would rework your configuration to be (which has to limit your threads to 3)
<configuration>
<includes>
<include>**/ParallelRun.java</include>
</includes>
<properties>
<property>
<name>dataproviderthreadcount</name>
<value>3</value>
</property>
</properties>
</configuration>
According to cucumber documentation you need to have parallel=classes only if you have several test runners.
Forks are also not applicable here. According to surefire plugin doc forks with parallel=classes do not make sense since forks themselves process classes in parallel.
Scenarios will be executed in parallel, so please decrease your thread count to 1.
<threadCount>1</threadCount>
if you are using TDD, use thread count.
if you are using BDD via
cucumber file, do not use thread count - set as 1 thread count.
I am new to cucumber-jvm parallel plugin and want to understand 2 things basically -
First - Which is correct combination of version i shall choose of cucumber-jvm parallel plugin, maven surefire and compiler plugin
Second - what is difference between cucumber-jvm implementation supporting parallel execution from V 4.0 and cucumber-jvm-parallel plugin
There are 2 phases people configure in cucumber-jvm-parallel plugin. Goal as validate or generateRunners. Can someone guide me whts the difference between these 2.
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
Is there any mandatory configuring missing in below jvm implementation
<plugin>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>2.2.0</version>
<executions>
<execution>
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
<configuration>
<glue>com.jacksparrow.automation.steps_definitions.functional</glue>
<outputDirectory>${project.build.directory}/generated-test-sources</outputDirectory>
<featuresDirectory>src/test/resources/features/functional/</featuresDirectory>
<cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir>
<format>json,html</format>
<tags>"~#ignored"</tags>
</configuration>
</execution>
</executions>
</plugin>
Please guide me as this is making me confused.
Maven Surefire has two levels of parallel execution for JUnit tests. It can run individual classes in parallel and it can provide a scheduler to test runners that correctly implement the ParentRunner. Normally this is used to run methods in parallel by the BlockJUnit4ClassRunner.
Until version v4 Cucumber did not correctly implement ParentRunner so feature files could not be executed in parallel using the scheduler.
As explained in the cucumber-jvm-parallel-plugin/README the plugin will create an individual runner class for each feature. This means that parallel execution depends on Surefires ability to run classes in parallel rather then its ability to provide a scheduler.
If you are using Cucumber v4 you should not use the parallel plugin. For all but a few limited use cases there are no benefits to doing so and a whole slew of downsides.
The cucumber-jvm/junit/README.md describes how to configure surefire parallel execution (the latest stable version is surefire:2.22.1 and should work).
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<parallel>both</parallel>
<threadCount>4</threadCount>
</configuration>
</plugin>
</plugins>
</build>
So overall picture is, if we are using cucumber v 4.0.0 or above then we shall not use cucumber-jvm parallel plugin. Because one of the main principle of Cucumber 4.0.0 implementation is to provide support to parallel execution and coming to Maven Surefire version, we can use latest one as well i.e. 3.0.0-M3.
I've got a vary simple profile in my maven pom file to run some integration tests during the normal test phase. Note I do not want to run these tests during the normal integration-test phase because I do not want to build the war and deploy etc. The tests run fine as normal JUnit tests.
So here is my profile:
<profile>
<id>AdminSeltests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<includes>
<include>**/*/TestSellerSignupWizard.java</include>
</includes>
</configuration>
<executions>
<execution>
<id>execution2</id>
<phase>test</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
My test is called:
com.xxxxx.xxx.client.selenium.seller_signup.TestCustomerSignupWizard
However when I run the above profile with:
mvn test -P AdminSeltests
No tests are run. I have tried the following as the value(s):
<include>**/TestSellerSignupWizard.*</include>
<include>**/TestSeller*.*</include>
<include>**/TestSeller*.java</include>
<include>**/*/TestSeller*.java</include>
<include>
com.xxxxx.xxx.client.selenium.seller_signup.TestCustomerSignupWizard.java
</include>
None of these work.
Any ideas anyone?
Thanks
Adam
SOLVED:
I'm using the maven-surefire-plugin and this has a automatic includes section which includes your normal test stuff. So I made an exclude configuration to exclude the normal unit tests, and then an include section to include a patter of integration test that I want to run
Not sure why it works this way, but it does:
<configuration>
<excludes>
<exclude>**/Test*.java</exclude>
<exclude>**/*Test.java</exclude>
<exclude>**/*TestCase.java</exclude>
</excludes>
<includes>
<include>**/ITTestSellerSignupWizard.java</include>
</includes>
</configuration>
Thanks for your help everyone.
The actual piece of code you did paste obviously doesn't work because you're mentioning class TestSellerSignupWizard while having tests in TestCustomerSignupWizard. I assume however it's a typo which doesn't matter actually because one of the Surefire's default mask for inclusion is **/Test*.java that fits well with you in this case.
So it all looks like working solution, so I'm afraid a problem is you don't have this class in your test classpath. You mentioned this is somehow related to integration testing so probably this class is located in src/it/java and not src/test/java that is Maven's default for Surefire. If I'm right, you should move this class into src/test/java or use (as you tried) alternative Surefire execution but with testSourceDirectory parameter overrided (link).
using Maven surefire, I'm unable to fork parallel test execution. That is, each of my test cases hs to run in a serapate JVM, hence the forking. In addition, I want my test cases to run in parallel. the first part is working without problem: I'm able to run each test case in its own JVM. the second part, however is still a challene for me. I haven't managed to get the paralle execution of test cases working. Here is how my plugin declaration look like:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.5</version>
<configuration>
<parallel>methods</parallel>
<forkMode>always</forkMode>
<argLine>-Xms512m -Xmx512m</argLine>
</configuration>
</plugin>
I've tried both methods and classes but haven't see any parallelization.
My JUnit version is 4.7 as shown by the depency declaration:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>compile</scope>
</dependency>
Any help would be much appricated.
Gregoire.
I think that you are supposed to use the threadCount parameter when using the parallel mode:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.6</version>
<configuration>
<forkMode>always</forkMode>
<argLine>-Xms512m -Xmx512m</argLine>
<parallel>methods</parallel>
<threadCount>4</threadCount>
</configuration>
</plugin>
I had the same problem, because i was using surefire version 2.7, after upgrade to 2.12 it worked with the following configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12</version>
<configuration>
<parallel>classes</parallel>
<forkMode>perthread</forkMode>
<threadCount>4</threadCount>
</configuration>
</plugin>
It spawned 4 threads, each running it's own jvm.
Make sure you get a log message something like this
[INFO] Concurrency config is {perCoreThreadCount=false, threadCount=1, parallel=classes, configurableParallelComputerPresent=true}
Just before this heading:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
This message indicates that the parallel surefire junit provider is active.
If this is not present surefire may be picking up a different version of junit than you think. Anything below 4.7 will not work. Run mvn dependency:tree to check which version(s) are present.
You should also upgrade to surefire 2.6 since a number of minor bugs related to parallel running have been fixed. For the same reason you should use the latest junit.
Surefire's parallel mode is extremely buggy. For example, see http://jira.codehaus.org/browse/SUREFIRE-747 and http://jira.codehaus.org/browse/SUREFIRE-730
I haven't managed to get a single test running in parallel to date (not to mention forking).
Try changing your forkMode from always to "never". It does not state this in their documentation, but you can not have fork plus parallel at this time (we found this after digging through the surefire code.)
Just so you know, you will probably run into tests that are not thread safe due to many test/supporting libraries (easymock, powermock, etc) invalidating the ability to parallel your tests.
Are you sure it's not working? You may not gain much speedup if your tests do not contain many test methods. With forkMode=always the best you can do is run all test methods within a class, in parallel.
Isn't parallel setting a TestNG only attribute? according to this:
http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#parallel
The surefire 2.16 fixed the parallel execution regarding the JUnit tests.