So we are doing distributed testing of our web-app using JMeter. For that you need to have the jmeter-server.bat file running in background as it acts as sort of a listener. The problem arises when one of the slave machine out of 4 restarts due to the load and the test is effectively stuck right there as the master machine expects some output from the 4th machine. Currently the automation is done via ansible-playbooks which are called in Jenkins. There are more or less 15 tests that are downstream to one another. So even if one test is stuck, the time is wasted until someone check on the machines.
Things I've tried so far:
I've tried using the Windows Task Scheduler and kept the jmeter-server.bat to run without any user loggin in, but it starts the bat file in background which in-turn spawns all the child processes in the background as well i.e. starts Selenium Chrome in headless mode.
I've tried adding the jmeter-server.bat in startup and configuring the system to AutoLogon without any password to trigger a session which will call the startup file. But unfortunately the idea was scrapped by IT for being insecure.
Tried using the ansible playbook by using the win_command but it again gets stuck as the batch file never returns anything.
Created a service as well for the bat file, but again the child processes started in background.
The problem arises when one of the slave machine out of 4 restarts due to the load
Instead of trying to work around the issue I would rather recommend finding the root cause and fixing it.
Make sure to follow JMeter Best Practices
Configure Java to take heap dump on failure
Inspect Windows PerfMon and operating system/application logs
Check presence of .hprof files in the "bin" folder of your JMeter installation and see what do they say
In general using Selenium for conducting the load is not recommended, I would rather suggest using JMeter's HTTP Request samplers for that, given you properly configure JMeter to behave like a real browser from the system under test perspective there won't be any difference whether the load comes from HTTP Request samplers or from the real browser.
The same states documentation on the WebDriver Sampler
Note: It is NOT the intention of this project to replace the HTTP Samplers included in JMeter. Rather it is meant to compliment them by measuring the end user load time.
Related
We would love to adopt Nightwatch.js for testing on browsers, but we're stuck on one major caveat: at the time of this writing, Nightwatchjs does not support running different tests using the same browser session. In short, it means that:
Creating the browser session is handled by the Nightwatch module from lib/index.js, in the startSession function;
Killing the browser would correspond to the delete command place in the Selenium action queue in the terminate function of that module;
A new Nightwatch client is created at every test run, which happens every time we load a different test file;
According to this source, it is possible to reuse the current browser session in Selenium, instead of opening a new window.
Has anyone managed to fix this problem in Nightwatch?
Here's the feature request on Github, which was requested on Mar 31, 2014 and is still open.
Another approach would be to circumvent the problem altogether by getting Nightwatch to merge all different files into one Test Suite, but that seems to be harder to solve than the problem with sessions...
Nightwatch now has support for resuing browser sessions. You can simply use the flag --reuse-browser to reuse sessions while running your nightwatch tests.
Selenium GRID2 is setup with a hub and three nodes (VMs Windows). All the config files are set and I start the services with a couple batch files executed on each machine. Once everything is started up it works great. When the remotely connected user who executed the batch files actually logs out of the machine the tests fail to connect to the browsers anymore.
I could leave the user account logged on after starting everything and just stay logged on. However, I would much rather find a way to get the services to startup and stay active without any user logged on the systems. Is there a way to force all the services to stay active after all users logout of the machine. Also need the full browser to function as there are screenshots as well. Could someone point me in the right direction or tell me how to get that to work or if it's possible?
Setup a scheduled task via Task Scheduler to execute the batch files to start the node on each machine. You can set a trigger to execute the batch file on system startup, or at a specific time.
Also, when creating the task, select the option "Run whether user is logged on or not"
I also add a task to restart the node machines (every night) to keep it clean of stale browser windows.Since the node batch files runs at startup, you do not need to login to the hub/node machine to start it manually.
I've been working on a webdriver framework for a while now, I guess it is
keyword driven now. We would like for there to be a central place for users to
store tests, preferably on a wiki, but then when they are run they would open up
the browser on users local machine.
I originally started working using Fitnesse, which works great for storing the
tests however when we hosted it on a server when a user tries to run a test it
opens the browser on the server which the user can't view. Does anyone know a
way that I could force Fitnesse to open the users local browser or display the
browser to the user? Or do you know another framework/way to store tests in a
central place but run them in local.
I've been looking at sending through the local users ip through a fixture to start up the initial framework, I was hoping that fitnesse would already know the ip.
Thanks,
James
You can either find a framework that does what you want, or the bare minimum would be to create a thin wrapper that copies the test dll's and executeable to a machine and executes psexec to execute the tests on the remote machine. You could probably write the entire thing in maybe 20 lines of code.
On my linux box somewhere in the United States, it's running Selenium Grid 2.
Currently 3 people will be regularly running parallel tests, but there's potential for more people to join and running more parallel tests....
now they are in other parts of the world, they need to submit an xml file which contains the test data, and the server needs to parse this and figure out the rules.
How can the client invoke a test to run on the remote server? via HTTP POSTing the xml file to the url in which the grid is running on?
Is it cheaper to just rent out a fast linux server and then buy more as number of parallel tests increase?
Or should I right off the bat, hook it up to Amazon ec2? If there are parallel tests on an ajax heavy web applications running 24/7, would it be cheaper to go with the single dedicated box or amazon? google app engine (no plugins for grid?)?
I am not sure I understood the xml submitting part in your question. However, I can tell you an example which will help to you understand how to submit the tests to grid.
Three people, PersonA, PersonB and PersonC are creating selenium tests in their local machine. They currently run the test using an ANT build (or something similar) against the selenium server jar which is in their local machine. In their code they would be having a line of code which tells which selenium instance should be used to run this test. This will be mostly like
new DefaultSelenium("localhost",port,browsername,URL)
Now these people want to move to selenium grid (1 or 2). Here is what they will have to do to use the grid
All they have to do is change the command
new DefaultSelenium("localhost",port,browsername,URL)
to
new DefaultSelenium("hubIPAddress",portInWhichHubWasStarted,browsername,URL)
Note:- browserName - Make sure there is a remote node registered in Hub for the same browserName.
Now all the commands will be sent to Selenium Hub and Hub would execute the commands using the remote nodes.
Hope this helps. Please post if you have any questions.
My iOS application, among its features, download files from a specific server. This downloading occurs entirely in the background, while the user is working on the app. When a download is complete, the resource associated with the file appears on the app screen.
My users report some misbehavior about missing resources that I could not reproduce. Some side information leads me to suspect that the problem is caused by the download of the resource's file to be aborted mid-way. Then the app has a partially downloaded file that never gets completed.
To confirm the hypothesis, to make sure any fix works, and to test for such random network vanishing under my feet, I would like to simulate the loss of the network on my test environment: the test server is web sharing on my development Mac, the test device is the iOS simulator running on the same Mac.
Is there a more convenient way to do that, than manually turning web sharing off on a breakpoint?
Depending on how you're downloading your file, one possible option would be to set the callback delegate to null halfway through the download. It would still download the data, but your application would simply stop receiving callbacks. Although, I don't know if that's how the application would function if it truly dropped the connection.
Another option would be to temporarily point the download request at some random file on an external web server, then halfway though just disconnect your computer from the internet. I've done that to test network connectivity issues and it usually works. The interesting problem in your case is that you're downloading from your own computer, so disconnecting won't help. This would just be so you can determine the order of callbacks within the application when this happens, (does it make any callbacks at all? In what order?) so that you can simulate that behavior when actually pointed to your test server.
Combine both options together, I guess, to get the best solution.