How to use assertions for multiple scenario in gatling? - karate

Right Now I am trying to do performance testing of all my api's.I already created one feature file having different scenarios(every scenario having different tag).Now I want to do use assertions on mean ResponseTime with different scenarios different assertions.
val Performance1 = scenario("Performance1").exec(karateFeature("classpath:mock/Testing1.feature#Performance"))
val Performance2 = scenario("Performance2").exec(karateFeature("classpath:mock/Testing2.feature#v3ContentMeta"))
val v4SearchTest = scenario("SearchTest").
group("SearchTesting") { exec(karateFeature("classpath:mock/Testing1.feature#Performance"))
}
setUp(
(Performance1.inject(rampUsers(10) over (5 seconds)).protocols(protocol)),
Performance2.inject(rampUsers(10) over (5 seconds)).protocols(protocol)
).assertions(details("SearchTesting").responseTime.mean.lte(680))```

You can add Gatling assertions as Global asserts. This will perfectly work with Karate Gatling. This is a sample scenario which we tried
setUp(
firstScenario.inject(
nothingFor(5 seconds), // Pause for a given duration
atOnceUsers(10), //Inject 10 Users at once
constantUsersPerSec(10) during (20 seconds), // Induce 10 requests on every second and continues this process for 30 seconds
rampUsers(10) over (10 seconds) // Linear Ramp up of the user
).protocols(protocol),
secondScenario.inject(
nothingFor(10 seconds), // Pause for a given duration
atOnceUsers(20), // Inject 10 Users at once
constantUsersPerSec(10) during (10 seconds), // Induce 10 requests on every second and continues this process for 40 seconds
).protocols(protocol),
thirdScenario.inject(
nothingFor(15 seconds), // Pause for a given duration
rampUsers(20) over (1 minute) // Linear Ramp up of the user
).protocols(protocol),
fourthScenario.inject(
nothingFor(20 seconds), // Pause for a given duration
constantUsersPerSec(10) during (20 seconds), // Induce 10 requests on every second and continues this process for 20 seconds
).protocols(protocol)
).assertions(
global.responseTime.max.between(100, 5000),
global.failedRequests.percent.is(0),
global.successfulRequests.percent.gt(90)
).maxDuration(10 minutes) // Configuring the maximum duration of your simulation. It is useful when we need to bound the duration the simulation when we can’t predict it.
The global asserts will be displayed as a separate section in the Gatling reports. This is a useful feature of Karate Gatling. Test specific failures will also get displayed in the report of Karate Gatling. For example, if this is your scenario
Scenario: My First Sample Scenario
Given url endpointUrl
And header karate-name = 'Feature 1_Scenario3'
When method get
Then status 200
And if the status code is not responded as 200, this also gets recorded in the Karate Gatling reports.
Asserts in Gatling: https://gatling.io/docs/current/general/assertions/#scope

Related

Increase delay and load over time in jMeter

I'm trying to get an increase in delay/pause and load over time in jmeter load testing while keeping the sequence constant. For example -
Initially - 10 samples (of 2 get requests - a,b,a,b,a,b...)
Then after 10 samples, a delay/pause of 10 secs and then 20 samples (a,b,a,b,a,b...)
After the 20 samples, another delay/pause of 20 secs; then 30 samples (a,b,a,b,a,b...)
And so on.
Constraints here being -
Getting exact number of samples
Getting the desired delay
The order of requests should be maintained
The critical section controller helps with maintaining the order of threads but only in a normal thread group. So if I try the ultimate thread group to get the desired variable delay and load, the order and number of samples go haywire.
I've tried the following-
Run test group consecutively
Flow control action
Throughput controller
Module controller
Interleave controller
Synchronizing timer (with and without flow control)
Add think times to children
Is there any way to get this output in jMeter? Or should I just opt for scripting?
Add User Defined Variables and set the following variables there:
samples=10
delay=10
Add Thread Group and specify the required number of threads and iterations
Add Loop Controller under the Thread Group and set "Loop Count" to ${samples}. Put your requests under the Loop Controller
Add JSR223 Sampler and put the following code into Script area:
def delay = vars.get('delay') as long
sleep(delay * 1000)
def new_delay = delay + 10
vars.put('delay', new_delay as String)
def samples = vars.get('samples') as int
def new_samples = samples + 10
vars.put('samples', new_samples as String)

advanced user simulation using Gatling [duplicate]

This question already has an answer here:
#karate How to pass parameter to a feature file in gatling simulation class?
(1 answer)
Closed 1 year ago.
I am trying to simulate a user scenario over an api using Gatling. I am a newbie to scala and gatling and would like to simulate a load scenario such that:
it starts with 0 users,
increase a load of 5 users in 10 seconds,
holds the user load for the next 10 seconds,
increase 5 more users over the next 10 seconds,
holds the user load for the next 10 seconds,
ramp down users from 10 to 0 over the next 10 seconds.
I am using karate feature files to define the api specifications.
Thanks for the help.
It should be possible by referring to the Karate docs and then the Gatling docs:
setUp(
create.inject(
rampUsers(5) during (10 seconds),
nothingFor(10 seconds),
rampUsers(5) during (10 seconds),
nothingFor(10 seconds),
rampUsers(0) during(10 seconds)
).protocols(protocol)
)

How to speed up selenium tests on AWS device farm?

I'm using Python for testing on AWS device farm. It seems that starting a selenium takes very very long. This is the code I use:
from time import time
from boto3 import client
from selenium import webdriver
def main():
start = time()
device_farm_client = client("devicefarm", region_name='us-west-2')
test_grid_url_response = device_farm_client.create_test_grid_url(
expiresInSeconds=666,
projectArn="arn:aws:devicefarm:us-west-2:..."
)
driver = webdriver.Remote(
command_executor=test_grid_url_response['url'],
desired_capabilities=webdriver.DesiredCapabilities.CHROME,
)
driver.get('https://api.ipify.org')
print(f"Your IP is: {driver.find_element_by_tag_name('pre').text}")
driver.quit()
print(f"took: {time() - start:.2f}")
if __name__ == '__main__':
main()
Output:
Your IP is: 100.10.10.111
took: 99.89s
Using existing selenium-hub infrastructure the IP is obtained in less than 2 seconds!
Is there any way how to reduce the time radically?
To reduce the overall execution time for complete test suite execution take advantage of the 50 concurrent sessions given you by default at no cost. Check this link. For eg:
Lets assume following details
one test Suite has 200 Selenium test cases
each test case takes around 10 seconds to execute
One AWS Device Farm Selenium Session takes around 60 seconds to start
then I will divide my 200 test cases into 50 concurrent sessions by running concurrent batches of 4 test cases per session.
Total Execution Time = (60 seconds to start each session + 10 seconds to start all 50 concurrent sessions with rate of 5 sessions per second + 4*10 seconds to execute the test cases in each session) = 60+10+40 = 110 seconds to finish complete test suite execution
WHEREAS
If you are existing selenium-hub infrastructure and lets say following details are assumed
200 Selenium test cases to execute
2 seconds to start a session
assume at max you can run 10 concurrent sessions
Total Execution Time = 2 seconds to start each session + 20*10 seconds to execute the test cases in each session = 200+2 = 202 seconds to finish complete test suite execution

Gatling user injection for 50 total users in 1 hour adding 10 users per 5 minutes

I need to setup a Gatling Test with a total of 50 concurrent users, but I have a problem because there is no choice to get it.
I use rampUsers(10) over (60 minutes) but it gets only 10 concurrent users.
Using constantUsersPerSec(users) during (60 minutes) is too stressful.
Is there any suggestion?
Thanks.
This could be done as follow:
val scn = scenario("Test").during(1 hours) {
exec(http("test").get("/"))
}
setUp(scn.inject(splitUsers(50) into atOnceUsers(10) separatedBy(5 minutes))
.protocols(httpConf))
see http://gatling.io/docs/2.0.3/general/simulation_setup.html:
splitUsers(nbUsers) into(injectionStep) separatedBy(duration): Repeatedly execute the defined injection step separated by a pause of the given duration until reaching nbUsers, the total number of users to inject.

JMeter HTTP Request Post Body from File

I am trying to send an HTTP request via JMeter. I have created a thread group with a loop count of 25. I have a ramp up period of 120 and number of threads set to 30. Within the thread group, I have 20 HTTP Requests. I am a little confused as to how JMeter runs these requests. Do each of the 20 requests within a thread group run in a single thread, and each loop over a thread group runs concurrently on a different thread? Or do each of the 20 requests run in different threads as and when they are available.
My other question is, Over each loop, I want to vary the body of the post data that is being sent via the HTTP request. Is it possible to pass the post data body via a file instead of inserting the data into the JMeter Body Data Tab as show below:
However, instead of doing that, I want to define some kind of variable that picks a file based on iteration of the threadgroup that is running, for example, if it is looping over the thread group the second time, i want to call test2.txt, if the third time test3.txt etc and these text files will contain different post data. Could anyone tell me if this is possible with JMeter please and if so, how would I go about doing this.
Point 1 - JMeter concurrency
JMeter starts with 1 thread and spawns more threads as per ramp-up set. In your case (30 threads and 120 seconds ramp-up) another thread is being added each 4 seconds. Each thread executes 20 requests and if there is another loop - starts over, if there is no loop - the threads shuts down. To control load and concurrency JMeter provides 2 options:
Synchronizing Timer - pause all threads till specified threshold is reached and then release all of them at the same time
Constant Throughput Timer - to specify the load in requests per minute.
Point 2 - Send file instead of text
You can replace your request body with __fileToString function. If you want to parametrize it you can use nested function to provide current iteration - see below.
Point 3 - adding iteration as a parameter
JMeter provides 2 options on how you can increment a counter each loop
Counter config element - starts from specified value and gets incremented by specified value each time it's called.
__counter function - start from 1 and gets incremented by 1 each time it's being called. Can be "per-user" or "global"
See How to Use JMeter Functions post series for comprehensive information on above and more JMeter functions.