I am running selenium testng tests parallely by running:
mvn test -Dtest="test-x" on different machines for each of the tests.
I am also using #AfterSuite and #BeforeSuite. Problem is that these methods are being run on each machine. I don't want that.
What I want is:
#BeforeSuite runs on a separate machine only once.
when this completes
Run all the tests parallely on different machines.
when all of these tests completes
#AfterSuite is run on a separate machine only once.
Machines on which #AfterSuite and #BeforeSuite are run should not run any tests. And, machines on which tests run, should not run these hooks.
Thanks in advance!
Related
I'm trying to use these all things together to run parallel tests in a headless chrome:
Docker, Selenium, Pytest
However, I'm wondering where it makes sense to run the parallel part of the system?
Docker can do this (using selenium grid). Both these can be used to run parallel (and distributed) selenium tests. e.g.
https://github.com/elgalu/docker-selenium
https://github.com/zalando/zalenium
Also Pytest has its own way of running parallel tests (using pytest-xdist) e.g.
http://pytest.org/dev/xdist.html
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
I would be grateful to find out the advantages/disadvantages are for each.
Also, any idea how to use these things together? Information on this seems really sparse.
You create as much as you need/want containers then you will let know xdist IPs of containers and if you need UI tests then pytest has pytest-splinter and if you need bdd scenarios you can use pytest-bdd.
However, I'm wondering where it makes sense to run the parallel part of the system?
Each part will contribute for the parallelism to happen.
You need Selenium HUB to orchestrate available browsers to run a test. You can have n browser running in headless mode, each one isolated in its own container.
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
Pytest will parallelize the test execution for you, bur not the instantiation and orchestration to the available browsers.
Summarizing:
Problem: You need to run UI (Selenium) tests in parallel. You will need N amount of browsers available to run this test.
Solution: You can start N nodes of headless chrome from docker.
problem: You have 10 different connection options to give to your drivers in your tests.
Solution: Start selenium hub and let it manage these for you. So you have to concern with only one connection point and then it will give you the browser that is free to run that test.
Problem: You run you tests now and only one browser is being used.
Solution: Use xdist to instruct pytest to run X amount of tests per time. X in this case can match with N number of browser available.
I put together a machine (Windows Server 2012R2) for POC reasons where a Jenkins installed and it executes Selenium UI tests using msunit
But, when I log in the server where the Jenkins runs and I watch what happens during CI build (compile and test execution) I can't see that the browser (Firefox) starts automatically, however, the test results and the logs show that a browser was executed.
My question is that, what the is happening when my tests are executed by Jenkins? If I execute the command which from visual studio on the same machine then I can see that Firefox starts, does what is programmed in the tests and the results are in the result.trx.Can I somehow set up Jenkins the way the browser really executed (I can believe it when I see it :)
In Jenkins when you run selenium test cases, they are executed in the background by default.
Your Jenkins might be configured to run those test cases in some video buffer(usually it happens on Linux but can also be configured on Windows) or in a headless state.
As your question, if you are using MSTest which basically used to convert the test cases result from trx to JMX format but also can be used to run selenium tests. when you run the same in Jenkins it will run in background on any slave or on master.
I'm using selenium and phantomjs to run automated tests. My tests run fine on my local machine. I need to run my tests on a windows server as part of a scheduled task that then emails the results each morning. When my tests are run on the server, a good majority fail and the screenshot that is taken upon failure is blank. Logs show it would appear its failing to load some pages when navigating from one page to another.
I have been running Selenium test cases successfully on my local Jenkins. When I run test cases on my local jenkins it opens up chrome/safari or any specified browser on my machine to run.
Now I would like to run the same job on my company Jenkins. But Jenkins is not set up on real machine. So it does not have browser.
How do I run my suite on company Jenkins. I do not want to run headless.
Thanks
You need to run selenium grid on your server machine, where jenkins is installed. After that you need to run a register script to your grid in the machine where you will run browsers.
Before run the browser, get some files ready in a folder. Selenium-server and the browser driver (chromedriver, iedriver whichever you want to use. You don't need firefox)
Script should be like this
cd /path/path/selenium
java -jar selenium-server-standalone-2.46.0.jar -role node -hub http://your.ip.address:4444/grid/register -browser "browserName=firefox,maxInstances=10,platform=MAC" -browser "browserName=chrome,maxInstances=10,platform=MAC" -maxSession 20 -Dwebdriver.chrome.driver=/path/path/selenium/chromedriver
You can edit the script according to your system (MAC/Windows, file paths. file names)
Install the required browsers on Company Jenkins:
It may be the good and effective option, but in case it can't be done refer other options.
Use Headless Browsers Like PhantomJS, etc:
These browsers are good for testing simple features but not meant for advanced feature testing. Headless browsers are not equivalent to real browsers.
Setup a Selenium Grid on Company Jenkins:
You can distribute your tests over different nodes and run tests parallel as well using the Grid. Make your Jenkins machine the Hub and add different Nodes to run your tests, Refer this.
Create Virtual Machines to run your tests:
Virtual machine can help you save the Hardware cost by creating virtual servers for running your tests. This can also be done in conjunction with Selenium Grid. Refer this.
Run your tests on cloud services platform like Saucelabs, BrowserStack, etc:
Best is to use cloud for running your tests. You need not worry about the testing environment like browsers and platforms as they have almost every possible browser + platform combination to run your test.
I have two test suites configured in build.gradle and i am using selenium-server-standalone-2.1.0.jar. How can i make use of selenium-grid to run selenium tests parallel. Is it possible to run two different test suites at a time?