Is it necessary to have load generator in stress testing. I am stress testing a e-commerce website and in my case I think I don't need run more that 300 virtual users. This amount of users I can run from JMeter non-gui mode. Is this will efficient? Thanks.
It is. You need to "generate the load" from somewhere so you need to have at least one load generator.
If you need only 300 virtual users which can be simulated from a single machine without any issues - you don't need to run JMeter in distributed mode, most probably you can generate this load from a single machine, just make sure to:
follow JMeter Best Practices
monitor resources consumption like CPU, RAM, Network, Disk IO, pagefile usage, etc. as JMeter needs to have enough headroom to operate in order to be able to send the requests as fast as possible or according to your test scenario. If you don't have better monitoring options in place - you can consider using JMeter PerfMon Plugin. The reason for monitoring resources usage by JMeter is that if JMeter cannot send requests fast enough your test results won't be accurate (false-negative) as even if the application has the capacity to handle more load JMeter won't be able to deliver this load if it's configuration isn't suitable or it doesn't have sufficient amount of resources from the operating system
Related
I would like to know if there is any open source load generator for stress testing. I am using JMeter script to run the test. I know there is one called flood.io but think it is not free. Can you please recommend me some option and documentation on how to use it properly? Thanks!!
Well, JMeter is open source, you can observe source repositories so it matches your needs.
The point is where to run JMeter so you need to have your own bare metal machines (or normal desktop/laptop/virtual machine). If you're looking for an open-source operating system there is a wide range of free and open source Linux distributions, FreeBSD, illumos, etc.
If you don't have any machines and any money to invest into these machines there are some free cloud options like AWS Free Tier or Oracle Cloud Free Tier, also some SaaS service providers offer free pricing plans, for example BlazeMeter gives you 50 virtual users, LoadFocus gives you 20 concurrent users, etc.
What is the difference between
Using the performance testing tool directly(Jmeter ,..)
Integrate the performance testing tool with selenium using plugin(Jmeter ,..).
Whether I can achieve all the functionalities in both the ways.
If used as a plugin will there be any limitations?
Thanks.
Performance testing tool acts on HTTP protocol level, basically pretty much the same as browser does, however in particular JMeter:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
therefore you can only test backend performance using JMeter however you will not get client-side performance metrics
Protocol-based tests have much less footprint in terms of resources (CPU, RAM, etc.) so you can simulate thousands of virtual users from a mid-range modern laptop.
Selenium is a browser automation framework, it operates real browsers so:
you have client-side performance metrics (including ability to query Window.Performance metrics)
and you don't have HTTP-protocol related metrics (connect time, latency, concurrency, throughput, etc.)
Browser-based tests have huge footprint in terms of resources as browsers are very resource intensive, for example Firefox 74 requires 1 CPU core and 2 GB of RAM per browser instance so you can kick off only several browsers on a mid-range modern laptop
Depending on your requirements you might want to either test the backend using JMeter or the frontend using Selenium or create the main load using JMeter and use 1-2 real browsers to test client-side performance.
If you're looking for a way of integrating JMeter with Selenium take a look at WebDriver Sampler (it's a JMeter Plugin which can be installed using JMeter Plugins Manager)
I try to run 10 user accessing a website concurrency by using Selenium Webdriver in JUnit and it caused my PC to lag because it open 10 Browser at the same time. I even tried to run Jmeter in Command Prompt, it is just laggy. Is there any methods to actually run 1000 user concurrently without stressing my PC?
Each browser has its system requirements, for example for Firefox 71 they are:
512MB of RAM / 2GB of RAM for the 64-bit version
Pentium 4 or newer processor that supports SSE2
If you want to kick off several browsers - you need to have:
2 GB of RAM per browser instance
1 CPU core per browser instance
For 10 browsers you will need to have 11+ CPU cores and 22+ GB of RAM, for 1000 browsers - proportionally more.
If you have to conduct performance testing using real browsers you will need to go for Distributed Testing and allocate sufficient amount of machines to act as load generators. Remember that machines must not be overloaded as if they will not be able to operate fast enough - you won't get accurate results.
Another option is migrating your Selenium tests to JMeter, you can basically run your Selenium tests through JMeter proxy so JMeter will be able to capture the relevant HTTP requests and convert them into HTTP Request samplers or replay them via Proxy2JMX Converter module of Taurus tool, check out How to Convert Selenium Scripts into the JMX article for more details.
JMeter's HTTP Request samplers have very small footprint comparing to real browsers so you will be able to mimic several thousands of virtual users from a modern mid-range laptop given you follow JMeter Best Practices
Currently I use jmeter aggregate report or summary report for submitting reports. But they expect something extra.. How can I give. Is there any plugins for getting server resources usage when testing load.
Reporting: since JMeter 3.0 there is a HTML Reporting Dashboard which can be generated during the test run. It contains exhaustive overview information. If you need to find out the reason of the bottleneck or memory leak or whatever you can consider extra Graphs available via JMeter Plugins project.
The same JMeter Plugins project provides PerfMon - client-server application which is able to collect over 70 different metrics and plot them via JMeter Listener. See How to Monitor Your Server Health & Performance During a JMeter Load Test guide for detailed setup and usage instructions.
There are quite a few plug-ins available that can help you analyze the results better. You can refer to https://jmeter-plugins.org/ for the same.
Most popularly used ones are:
Response Times Over Time
Response Times Percentiles
Transactions per Second
Response Latencies Over Time
In case of server usage you can use following that comes with JMeter plug-ins
PerfMon Metrics Collector and Server Agent or
In case of Unix based system use sar command that comes with sysstat package or VMstat. In case of windows based system use Perfmon to capture the system utilization data while the test is running and then use Ksar to plot graphs with the data collected using sar. https://sourceforge.net/projects/ksar/
If you have collected data using Perfmon then plot the graphs using PAL. https://pal.codeplex.com/
In this case, I would suggest using Grafana. It shows realtime results. And the best thing is, it can be configured according to the need.
Now, the thing is how to use it? Using it is not that tough.
If you're using a Mac or Linux (Any Flavour) things become easy. If you're using Windows, I would suggest using a virtual machine. The reason behind that is windows block traffic after some requests. And that causes a lot of pain in the head.
In my case, I used a virtual machine to setup ubuntu inside it and then configured Grafana.
For working with Grafana, you need to have these two things installed.
Grafana Itself
Influx Db for the backend
Links for both here below:
https://grafana.com/grafana/download?platform=linux
https://portal.influxdata.com/downloads/
Once installed and setup,
You need to use Backen Listener to push results o Graphite Client (Installed along with Influx DB Automatically).
I know it is a bit confusing but once you understand the thing, you and your client will love the detailed reports.
Remeber, Grafana is all about configuration.
Let me know if you have any confusion regarf=ding this.
Happy to help. :)
I want check my AWS autoscling is working fine when the CPU utilization is greater then 80%.So i want a script for send many http resuest to my AWS server for testing.Please help me.
There are several tools to accomplish this task, two of them are
ab, a command line tool build into Apache:
ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving.
jMeter, a graphical testing tool written in Java:
The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.
Handle them with care, as they really do what you are requesting. I've managed to brute force my test server until it collapsed with ab quite easy...