Currently I'm trying to run a load test which walks through a uniquely created URL. I know JMeter is often used for load testing, but I was specifically asked to do it through something like Selenium that uses real browsers to create the URL then open that URL and complete the steps within the URL. I have created a Selenium script that can easily do this, but I need to do this 100 times concurrently and can't find a good way to do.
Is there a way to do this? I've looked into Selenium Grid but I'm not sure if I even have enough nodes to run 100 browsers concurrently. Please if you have recommendations for software or a different method of doing this I would love to hear it. Thank you!
JMeter can be integrated with Selenium using WebDriver Sampler so you can re-use your code and rely on JMeter's multithreading capabilities.
If one machine won't be powerful enough to kick off 100 browsers - you can consider going for Distributed Testing
In general be aware that browsers don't do any magic, they just send HTTP requests, wait for responses and render them. JMeter is not capable only of rendering the page, but if you need to load test the backend - it can mimic browser's network footprint with 100% accuracy, just make sure to configure JMeter accordingly in order to behave like a real browser
JavaScript execution time and page rendering speed can be checked either using single WebDriver Sampler or a separate solution like Lighthouse
I'm looking to perform a browser based load test on my application. I was looking this option to use selenium plugin with Jmeter. So that I can write the required UI flow with Selenium within Jmeter and parameterize using Jmeter options.
I've used Jmeter for http/https request with 400/500 concurrent users.
Is it possible to use Jmeter + Selenium plugin for a browser level load test with around 400/500 concurrent users?
Thank you
Theoretically yes but you will need to think about the hardware.
As per Firefox 102 System Requirements one browser instance needs:
1 CPU core
2 GB of RAM
so for 500 browsers you will need at least 501 core and at least 1001 GB or RAM so most probably you will have to go for Distributed Testing
Also as per WebDriver Sampler documentation:
Note: It is NOT the intention of this project to replace the HTTP Samplers included in JMeter. Rather it is meant to compliment them by measuring the end user load time.
Also as per Selenium documentation
Performance testing using Selenium and WebDriver is generally not advised. Not because it is incapable, but because it is not optimised for the job and you are unlikely to get good results.
So I think you need to conduct the main load on HTTP protocol level and use 1-2 instances of real browser for frontend (client-side) performance testing.
I'm currently trying to load test a homepage I develop. Till now Loader.io was good enough for my purposes, but I realized it does not download/use the embedded assets.
Is there a load test service, which get's as close as possible to real users?
I haven't found anything until now. Hopefully somebody of you guys knows a suitable service.
Thanks in advance!
Apache JMeter does it for sure:
See Web Testing with JMeter: How To Properly Handle Embedded Resources in HTML Responses
Moreover it simulates browser cache via HTTP Cache Manager
If you rather looking for a "service" there are several options of running a JMeter test in the cloud starting from shell scripts like JMeter ec2 Script and ending up with end-to-end solutions like Flood.io or BlazeMeter
I am trying to load test a website where lot of images are called via Ajax and the very first thing i.e. logging into the application opens up a pop up when we click on log in button. I tried JMeter proxy settings to record it but failed. Also tried with Badboy, and it didn't work out too.
The limitations I have with me is I need to load test it with JMeter only. Are there any plugins available which can be integrated to JMeter so that AJAX and pop ups in an application can be handled in a much better way.
In general, JMeter does not have a problem with Ajax or Popups - so there is no plugin to address such. Both recording and playback happens at the HTTP layer so things like popups are somewhat irrelevant and Ajax is more a matter of timing; but in both cases it is only the HTTP call that matters. So, if you are having a problem, it could be something else that is holding you up. Try playing with the Proxy settings or using a different browser, beyond that you would need to expand 'didn't work out' and 'it failed' into a more detailed problem statement!
One solution - regardless of your exact problem - is to build the testplan manually, this is often actually the easiest way to work with JMeter. You can use a tool like fiddler or charles to examine the traffic and create the requests directly based on what you see. You can also use browser dev tools to do this. You might instinctively think this is going to be difficult but it's not and the added bonus is that the process gives you a solid understanding of how whatever it is you are testing actually works - which is always nice to have.
JMeter is a tool for testing server-side activity; as long as you record every request to server-side it doesn't care whatever happens on the client. Can you give details abt why JMeter "failed" to record Ajax requests?
Are you sure that you have included recording controller to your Jmeter thread group?
I think you are missing with you configuration for "recording controller" or "HTTP Request Sampler"
Some more information about your "didn't work" situation will be much more helpful.
I'll suggest that you should also have a look at the Jmeter Log file(jmeter.log) created in "Bin" folder to understand the root cause of of the issue.
Thanks,
I have a system to test, which is a video ads distribution technology. I need to load every video like 1-2 mins to serve the ads. The videos are played in a Flash client and streamed as FLV streams like in YouTube.
The reason why I need to test it only via browsers -- and every other method won't work -- is to stress test both the video streaming servers and the ads servers simultaneously and displaying ads in real-time.
I have used Selenium, WatiN, Automation Anywhere and many other automation tools. However, when I am trying to start like 10000 browsers on my machine (32GB RAM, 16-core CPU), none of them are able to do the job.
With Selenium, I am able to start the maximum FireFox instances so far, but that's still too low: half of the instances don't run the test.
Any suggestions to do with Selenium?
You aren't going to run 10,000 browsers on your machine. That would give 3.2MB of physical memory per browser instance and I'm pretty sure FireFox just won't like that.
You could create a JMeter script that hits your server with many threads. It won't interact with the UI but would simulate the load of many clients hitting whatever URLs you tell it. I believe it also includes the ability to record a session and play it back for easy setup of your sessions.
Selenium isn't really optimized for load/stress testing, especially if you're running your browsers locally. Running 1000+ browsers is going to choke even the beefiest server. Though RAM is an obvious bottleneck, you also have limited CPU resources and bandwidth. The latter being a primary concern if you are loading videos.
Not to mention you'd be testing from a single IP with 10k browsers, so load balancing may not kick in properly, as well as the actual distribution of video ads to specific virtual users.
If you want to stick with existing Selenium tests, I've had good experiences with BrowserMob. They basically have a huge grid to do real browser load-testing, distributed across AWS.
Another recommendation would be an actual performance testing tool. I'd recommend Soasta CloudTest. They have a free version that runs 100 users so you can see if it will be a good fit for you. I have found that scripting for CloudTest is relatively simple.
Disclaimer: My experiences with both companies have been as a paying customer and I have never worked for either.
If you are using Windows machine then as per my experience there is a limit on number of browser window instances to be opened. As per my test last time, it does restrict between 100-150 browser windows.
I would recommend you using headless robot, which doesn't require opening browser window. I think latest version of Selenium has that capability. But it seems to be more like a load test as you are trying to simulate 10,000+ user instances, I would recommend you using load testing tool like JMeter or LoadRunner.
It looks to me that you are trying to verify what the client will see based on high traffic, no?
In that case, Joel is quite correct. If you absolutely have to see what the client sees, you could use threaded hits and just dump the results in a database. That'll show you anything the client would see anyway, and it's a lot easier to sort through than thousands of browser instances.
Either way, your client will not see errors if there are no errors present on the server side. If you're testing functionality in bandwidth restricted environments, CPU-intensive environments, or memory-intensive environments, those are much easier achieved than running thousands of browser instances.
Your post smells of some form of ad-based fraud to me, but either way: have you considered using different web browsers besides Firefox? PhantomJS is a headless webkit-based browser that is compatible with Selenium. It supports all the core browser features like DOM handling, CSS selectors, Javascript and Canvas. I do not know if it supports Flash.
This post has a decent list of other headless and automatable web-browsers that you might consider.
Also, if each browser instance is instantiating a Flash plugin, don't neglect the possibility that the issue could be with Flash and not Firefox. Alternatively, why instantiate several different Firefox processes? Can you accomplish what you want through the use of tabs instead?
The in-house way to this wiht selenium is using browsermob proxy and multiple broswser agents to recreate the experience of different users, changing the ip is more difficult because it requires changing your home network.
Here is a good example