I'm trying to use these all things together to run parallel tests in a headless chrome:
Docker, Selenium, Pytest
However, I'm wondering where it makes sense to run the parallel part of the system?
Docker can do this (using selenium grid). Both these can be used to run parallel (and distributed) selenium tests. e.g.
https://github.com/elgalu/docker-selenium
https://github.com/zalando/zalenium
Also Pytest has its own way of running parallel tests (using pytest-xdist) e.g.
http://pytest.org/dev/xdist.html
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
I would be grateful to find out the advantages/disadvantages are for each.
Also, any idea how to use these things together? Information on this seems really sparse.
You create as much as you need/want containers then you will let know xdist IPs of containers and if you need UI tests then pytest has pytest-splinter and if you need bdd scenarios you can use pytest-bdd.
However, I'm wondering where it makes sense to run the parallel part of the system?
Each part will contribute for the parallelism to happen.
You need Selenium HUB to orchestrate available browsers to run a test. You can have n browser running in headless mode, each one isolated in its own container.
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
Pytest will parallelize the test execution for you, bur not the instantiation and orchestration to the available browsers.
Summarizing:
Problem: You need to run UI (Selenium) tests in parallel. You will need N amount of browsers available to run this test.
Solution: You can start N nodes of headless chrome from docker.
problem: You have 10 different connection options to give to your drivers in your tests.
Solution: Start selenium hub and let it manage these for you. So you have to concern with only one connection point and then it will give you the browser that is free to run that test.
Problem: You run you tests now and only one browser is being used.
Solution: Use xdist to instruct pytest to run X amount of tests per time. X in this case can match with N number of browser available.
Related
When I run Selenium tests on Chrome in parallel with two threads(both threads running on same machine), sometimes I've seen one of the chrome instances stuck(i.e UI interactions do not happen) for some time or indefinitely.
This causes test execution to take more time or fail test unnecessarily.
Could you please help to understand if this is known behaviour when tests run on same machine? If not, what are the causes and solutions? Also guide on how we can pro-grammatically(using Java) set memory for Chrome browser which is running through Selenium?
Trying to enable Xvfb in Jenkins declarative pipeline to be able to run Selenium headless tests from the pipeline definition.
Have been able to run Selenium tests in a standard Jenkins (Linux) job. That's fine, i.e. Xvfb can be enabled under build (after plugin install) in the Jenkins job and then can Python virtual env be setup and Selenium tests executed from shell.
But I want to have a pipeline scope/setup. But in pipeline type jobs the Xvfb doesn't show up. And I haven't been able to find an answer if and how it can be enabled from the declarative pipeline code itself. Is it possible?
Is there any workaround?
Yes you can, every job which is pipeline have a link on left side of job page "Pipeline Syntax" when you go there it will help you a lot for some non obvious cases. So, for your case:
I have been running Selenium test cases successfully on my local Jenkins. When I run test cases on my local jenkins it opens up chrome/safari or any specified browser on my machine to run.
Now I would like to run the same job on my company Jenkins. But Jenkins is not set up on real machine. So it does not have browser.
How do I run my suite on company Jenkins. I do not want to run headless.
Thanks
You need to run selenium grid on your server machine, where jenkins is installed. After that you need to run a register script to your grid in the machine where you will run browsers.
Before run the browser, get some files ready in a folder. Selenium-server and the browser driver (chromedriver, iedriver whichever you want to use. You don't need firefox)
Script should be like this
cd /path/path/selenium
java -jar selenium-server-standalone-2.46.0.jar -role node -hub http://your.ip.address:4444/grid/register -browser "browserName=firefox,maxInstances=10,platform=MAC" -browser "browserName=chrome,maxInstances=10,platform=MAC" -maxSession 20 -Dwebdriver.chrome.driver=/path/path/selenium/chromedriver
You can edit the script according to your system (MAC/Windows, file paths. file names)
Install the required browsers on Company Jenkins:
It may be the good and effective option, but in case it can't be done refer other options.
Use Headless Browsers Like PhantomJS, etc:
These browsers are good for testing simple features but not meant for advanced feature testing. Headless browsers are not equivalent to real browsers.
Setup a Selenium Grid on Company Jenkins:
You can distribute your tests over different nodes and run tests parallel as well using the Grid. Make your Jenkins machine the Hub and add different Nodes to run your tests, Refer this.
Create Virtual Machines to run your tests:
Virtual machine can help you save the Hardware cost by creating virtual servers for running your tests. This can also be done in conjunction with Selenium Grid. Refer this.
Run your tests on cloud services platform like Saucelabs, BrowserStack, etc:
Best is to use cloud for running your tests. You need not worry about the testing environment like browsers and platforms as they have almost every possible browser + platform combination to run your test.
Does anyone know how to run a selenium test suite from a selenium grid.
I know you can do it programatically with a single test, but I was wondering if there was an easy way to do it with a full test suite.
Something like java -jar selenium-server.jar -host -htmlSuite .
Thanks for the help.
This page will help you a lot. It demonstrates how to set up a hub and nodes which connect to it registering their 'configurations' (what browsers/'capabilities' they have). And the last piece of the jigsaw which is having code programmatically connect to the hub and run tests on it (the hub will forward stuff to a relevant node so the tests get performed there).
No. Selenium Grid is a facility for Selenium RC users to run their tests across a variety of machines with fine-grained configurations (e.g., a particular version of a browser). As far as I know, there's no ability to specify that in the htmlSuite. If there were, you might be able to make it work. But running just a suite like this is not supported and likely never will be because that's not what it was designed for.
Right now I am using Hudson CI to launch browser in xvnc and run my tests through Webdriver (pre 2.0)
Should I use Selenium remote webdriver instead? is that more efficient than hudson ci?
I want to run my tests on Amazon....are there any selenium remote webdriver server or Selenium Grid plugin that can already integrate with Amazon? Should I pursue Grid or remotedriver server?
My ultimate goal is to be able to launch multiple tests parallel on Selenium on Amazon and be able to quickly scale up and down.
Right now I pay $100/month for server that has hudson CI on it....but I never end up using the whole capacity....
Grid or the Remote Server does not replace a CI tool like Hudson. They will not schedule jobs, parallelize your tests, report results (with history) or send email/IM notifications when builds fail.
So you would still have Hudson (or similar) for all those things, possibly using Grid or the Remote Server to decouple the machine(s) running the browser(s) from the one running the tests.
Selenium Grid will allow you launch multiple tests in parallel. And with the right configuration you can even run the tests of different browsers in parallel.
Selenium grid is still using selenium 1.x remote control drivers. So if you already have your tests written in webdriver code, then you might not be able to use it with current Selenium grid. There is a new version of selenium grid - Grid 2.0 that will be out soon. That will be support webdriver as well.