I have noticed a throttling behavior with Grid 4.1.3 that was not present with Selenium Grid 3.141.59. For example: I try to run a suite with 120 tests. The grid concurrency is 48 and I'm running the tests on 24 threads. In the beginning, the grid executes 24 tests concurrently. After 60 tests where executed, it starts running the rest of the tests on 12 threads, and so on. Is this documented behavior? Can I configure the Grid somehow else?
Related
Safari does not support execution of automation scripts on multiple threads, Please let me know if there are any alternatives to run selenium tests in parallel. I am running tests in parallel on other browsers like chrome and firefox. The Framework is developed in python using pytest and I am using pytest-xdist to run the scripts in parallel. I also tried pytest-parallel but even this did not help.
Have you searched other related threads?
Example thread from about 8 months ago: here
Paraphrased quote from there:
My expectation is that you're hitting default maxSession limit of
5 browser instances.
You can double check it by opening your Selenium Grid console and
looking into node configuration:
The value can be ramped up by providing the relevant maxSession
parameter to your Selenium Grid Node startup command line like:
java -jar selenium-server-standalone-3.141.59.jar -role node -maxSession 10 -hub http://localhost:4444/grid/register
^^^^^^^^^^^^^^
References
I've spent about a day looking for solutions around the web to my issue but none work for me.
Here is my scenario:
I am running Selenium scripts with ChromeDriver using pyATS framework on my Ubuntu 18.04 VM. The VM has 4 GB of memory. I also have setup Jenkins on the machine and am trying to run the pyATS script with the pyATS plugin.
When running headless mode from the terminal, the script runs in the same or faster time than non-headless mode. However, when I run in Jenkins on the same machine, I am getting extreme slowdowns. It looks almost as if Jenkins is running my script in sections, with >2 minutes of delay in between steps at random.
I've tried out Xvfb, headless with various chrome options (noproxy, proxy options, gpu disable, etc), increasing heap memory for jenkins, but I always get the same random 2 min of delay in between script steps.
The script doesn't fail - it will complete eventually. But for a step that I expect to take around 2 min, jenkins will take 10 minutes.
I currently don't have a way to increase the memory my VM has, but are there any other solutions that I can try in the meantime?
Found the issue, I had to set the "--proxy-server" for Chrome to the proxy my VM was running behind. For some reason Firefox was working fine without that option so I didn't think to set this option for Chrome.
When I run Selenium tests on Chrome in parallel with two threads(both threads running on same machine), sometimes I've seen one of the chrome instances stuck(i.e UI interactions do not happen) for some time or indefinitely.
This causes test execution to take more time or fail test unnecessarily.
Could you please help to understand if this is known behaviour when tests run on same machine? If not, what are the causes and solutions? Also guide on how we can pro-grammatically(using Java) set memory for Chrome browser which is running through Selenium?
I'm trying to use these all things together to run parallel tests in a headless chrome:
Docker, Selenium, Pytest
However, I'm wondering where it makes sense to run the parallel part of the system?
Docker can do this (using selenium grid). Both these can be used to run parallel (and distributed) selenium tests. e.g.
https://github.com/elgalu/docker-selenium
https://github.com/zalando/zalenium
Also Pytest has its own way of running parallel tests (using pytest-xdist) e.g.
http://pytest.org/dev/xdist.html
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
I would be grateful to find out the advantages/disadvantages are for each.
Also, any idea how to use these things together? Information on this seems really sparse.
You create as much as you need/want containers then you will let know xdist IPs of containers and if you need UI tests then pytest has pytest-splinter and if you need bdd scenarios you can use pytest-bdd.
However, I'm wondering where it makes sense to run the parallel part of the system?
Each part will contribute for the parallelism to happen.
You need Selenium HUB to orchestrate available browsers to run a test. You can have n browser running in headless mode, each one isolated in its own container.
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
Pytest will parallelize the test execution for you, bur not the instantiation and orchestration to the available browsers.
Summarizing:
Problem: You need to run UI (Selenium) tests in parallel. You will need N amount of browsers available to run this test.
Solution: You can start N nodes of headless chrome from docker.
problem: You have 10 different connection options to give to your drivers in your tests.
Solution: Start selenium hub and let it manage these for you. So you have to concern with only one connection point and then it will give you the browser that is free to run that test.
Problem: You run you tests now and only one browser is being used.
Solution: Use xdist to instruct pytest to run X amount of tests per time. X in this case can match with N number of browser available.
I have 20 test cases, i want to run the first 5 on one remote machine, the other five test cases on the other remote machine....that means run the 20 test cases on 4 different remote machines.
i am using selenium 2.0 webdriver. please tell me how to do it.
What test framework are you using? Although Selenium Grid gives you the ability to run remotely, it is the test framework which provides the mechanism to run in parallel.
Some of the frameworks which support this includes;
Parallel-Test (Ruby) - supports Rspec, Cucumber and TestUnit
TestNG (Java)
JUnit (Java) - I haven't tried it myself as I have always used TestNG in Java
MbUnit (C#)
There may be others