I'm new to parallel testing and I was wondering what the difference is between them.
Apparently, pytest-xdist does not need Selenium Grid to run. It can be used with Selenium alone.
Does anybody have any clue or resource where I can learn the difference?
Thanks and kind regards.
Just in case this is useful for anybody I will write down what I learnt:
pytest-xdist: It is used for running parallel tests in the browsers that are installed in the local machine where tests will be run. This means that each test has a browser configured (e.g.: Firefox, Chrome) such as:
driver = webdriver.Firefox()
or
driver = webdriver.Chrome()
And so each test will run with the driver that was specified in the code. Obviously, the local machine needs the browser drivers to be available in any PATH location, so that tests can be run with them.
Selenium Grid: It allows the execution of tests in different browsers, browser versions and operating systems configurations.
Selenium Grid combined with pytest-xdist allows the execution of parallel tests in different browser-OS environments (configured with capabilities, I think).
An execution command example would be:
pytest -n5 -v -s -m "test or ready" --capability browserName firefox
-n5: (pytest parameter) means that 5 instances of the browser will be launched simultaneously.
test or ready: these are the markers that can be combined to execute tests that have these markers.
browserName firefox: It is a capability that indicates that tests must be run in the specified browser, in this case, Firefox. Some possible values are: chrome, firefox, internet explorer, safari.
Some other capabilities are:
version: The browser version to use.
platform: The platform in which the browser will be executed. Some possible values are: WINDOWS, XP, VISTA, MAC, LINUX, UNIX, ANDROID.
To set up the Selenium Grid environment go along the following steps:
Download selenium-server-standalone-[version].jar from
https://www.selenium.dev/downloads/.
Initiate the HUB:
java -jar selenium-server-standalone-[version].jar -role hub
Initiate as many NODES as you want with:
java -jar selenium-server-standalone-[version].jar -role node -hub http://[URL_HUB]/wd/hub
Configure a RemoteDriver in the tests, for example: driver = webdriver.Remote( desired_capabilities = DesiredCapabilities.CHROME, command_executor = 'http://[URL_HUB]:4444/wd/hub')
I still need to learn how to pass the capabilities args as parameters for the RemoteDriver. In conftest.pyfile I can get the capabilities as a list with config.getoption('--capability')). I still need to figure out how to pass the capabilities I want to the setUp method of all my tests.
If someone knows, I will really appreciate a hint on this.
I hope this helps someone who is as lost as I was at the beginning :)
Related
I have been running Selenium test cases successfully on my local Jenkins. When I run test cases on my local jenkins it opens up chrome/safari or any specified browser on my machine to run.
Now I would like to run the same job on my company Jenkins. But Jenkins is not set up on real machine. So it does not have browser.
How do I run my suite on company Jenkins. I do not want to run headless.
Thanks
You need to run selenium grid on your server machine, where jenkins is installed. After that you need to run a register script to your grid in the machine where you will run browsers.
Before run the browser, get some files ready in a folder. Selenium-server and the browser driver (chromedriver, iedriver whichever you want to use. You don't need firefox)
Script should be like this
cd /path/path/selenium
java -jar selenium-server-standalone-2.46.0.jar -role node -hub http://your.ip.address:4444/grid/register -browser "browserName=firefox,maxInstances=10,platform=MAC" -browser "browserName=chrome,maxInstances=10,platform=MAC" -maxSession 20 -Dwebdriver.chrome.driver=/path/path/selenium/chromedriver
You can edit the script according to your system (MAC/Windows, file paths. file names)
Install the required browsers on Company Jenkins:
It may be the good and effective option, but in case it can't be done refer other options.
Use Headless Browsers Like PhantomJS, etc:
These browsers are good for testing simple features but not meant for advanced feature testing. Headless browsers are not equivalent to real browsers.
Setup a Selenium Grid on Company Jenkins:
You can distribute your tests over different nodes and run tests parallel as well using the Grid. Make your Jenkins machine the Hub and add different Nodes to run your tests, Refer this.
Create Virtual Machines to run your tests:
Virtual machine can help you save the Hardware cost by creating virtual servers for running your tests. This can also be done in conjunction with Selenium Grid. Refer this.
Run your tests on cloud services platform like Saucelabs, BrowserStack, etc:
Best is to use cloud for running your tests. You need not worry about the testing environment like browsers and platforms as they have almost every possible browser + platform combination to run your test.
How do I run test cases made with Selenium IDE (Firefox Plugin) in a headless browser?
When I create test cases with the firefox plugin it saves them as .html files.
I am trying to setup a way of running them in a headless browser, (using phantomJS or another tool/lib). I have seen many people mention running firefox in a hidden frame, but this is not the same as headless (ie. PhantomJS)
I have already figured out how to run selenium test cases from command line, but only in firefox or ie, I can't get it to run .html testcases on phantomjs:
java -jar selenium-server-standalone-2.39.0.jar -htmlSuite "*firefox" "http://127.0.0.1" "ts-ProjectList/TestSuite.html" "ProjectList-results.html"
I know the IDE can export test cases in different languages, but I want non-tech team members to be able to create tests using the firefox plugin.
Have you looked at http://code.tutsplus.com/tutorials/headless-functional-testing-with-selenium-and-phantomjs--net-30545?
Changing "*firefox" to phantomjs should get you started.
You'll need the phantomJS driver:
http://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver_phantomjs/selenium.webdriver.phantomjs.webdriver.html
--- TL;DR
At this point I suggest everyone to tied their Continuous Integration server/service to https://ghostinspector.com/
OLD QUESTION
after three days googling and testing I gave up, and I need help.
My objective is allow my co-workers to record one or more tests with Selenium IDE. Export them, upload them into a server, and get this server running these tests using the webdriver with htmlunit. As we build or fix the app, we will upload the tests to make out test library.
Record a test with Selenium IDE is okay. But getting it running is the problem. The machine we intend to let the tests is an linux amazon server. No front-end stuff, no kde, gtk, so no firefox, chrome, etc... This is why I've specified the htmlunit driver.
So far I wasn't able to get this task running even into my machine - Ubuntu 12.04 x86_64.
I downloaded the selenium-server tarball, and tried running:
java -jar selenium-server.jar -htmlSuite "*webdriver" "our.site.org" "/path/to/testsuite1.html" "/path/to/report1.html"
No success. Even changing the "*webdriver" (using other pops-up a browser screen).
I've tried running the server and the standalone server and connecting via browser.
I've tried PHP bindings by facebook.
I've tried PHPUnit and Testing Selenium classes - along with their respectives exported scripts from Selenium Formatters.
I really do not know where I'm slipping. Can anyone give me a safe direction, tutorial, etc, to follow with?
--- EDIT
Okay, my question may be resumed to:
What si the command line that would allow me to run selenese scripts with selenium-server, using the HtmlUnit driver?
Are you using Continuous Integration?
If so, you should consider getting a plugin to have your CI software run the Selenium tests. Works like a charm for me with Jenkins.
Considering your particular setup, you could both have the amazon linux server run the tests with HTMLUnitDriver, and declare other machines (with a GUI and proper browser) as "nodes" to run your test on other browsers.
Link to a tutorial
Have you read this blog post by David Burns (Automated Tester):
http://www.theautomatedtester.co.uk/tutorials/selenium/selenium_rc_setup.htm
He describes the way to run selenese tests using HTMLSuite.
We are going to use the HTMLSuite commands of the Selenium Remote
Control. This allows you run your Selenese Test Suites as is. The
command should look like java -jar selenium-servre.jar -htmlsuite
. Browser
could be : -*firefox
-*chrome
-*iexplore
-*iehta
-*safari
-*custom /path/to/browser
The path to the test suite and the results file should be a full path.
Here is an example command; java -jar selenium-server.jar -htmlsuite
*iexplore http://www.theautomatedtester.co.uk c:\testsuite\testsuite.html c:\testsuite\results.html
I would point out that htmlunit does not seem to be a supported option so I would expect to use -*custom and provide a path to htmlunit.
This is legacy functionality so there is a chance it doesn't work as expected any more. HTMLSuite expects the tests to be in Selenese (HTML table) format, you mention trying with the PHP binding, I would not expect this to work. If you do want to use some PHP bindings I would suggest using Adam Saunter's fork of the facebook bindings, they are the most up to date and best supported.
https://github.com/Element-34/saunter.php
With Selenium WebDriver you can point to start a HtmlUnit in a already started node
In Java you'll do something like this:
IWebDriver driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), DesiredCapabilities.HtmlUnit());
To start the node just make sure to set browserName to 'htmlunit'.
I'm working on Selenium and I currently have a hub connected to two vms running linux with all the browsers.
I was able to launch the browsers . However, event hough I have two vms all my tests run on a single browser on a single vm. Selenium does not launch more than one browser in a vm. I have set maximum instance to 5 and it reflects in my gris console. I have testsuite which needs to run on linux and windows vm and it has to run simultaneously . Im using the WebDriver to call my grid like this .
capabilities = new DesiredCapabilities();
capabilities.setBrowserName("firefox");
driver = new RemoteWebDriver(new URL("http://XXX.XXX.XXX.:4444/wd/hub"), capabilities);
And this is how I create my test Suite:
testNames.add(SupportPortalTestSuite.class.getName());
JUnitCore.main(testNames.toArray(new String[testNames.size()]));
Also when I register my nodes I get:
11:28:47.403 INFO - using the json request : {"class":"org.openqa.grid.common.RegistrationRequest","capabilities":[{"platform":"LINUX","seleniumProtocol":"Selenium","browserName":"*firefox","maxInstances":5},{"platform":"LINUX","seleniumProtocol":"Selenium","browserName":"*googlechrome","maxInstances":5},{"platform":"LINUX","seleniumProtocol":"Selenium","browserName":"*iexplore","maxInstances":1},{"platform":"LINUX","seleniumProtocol":"WebDriver","browserName":"firefox","maxInstances":5},{"platform":"LINUX","seleniumProtocol":"WebDriver","browserName":"chrome","maxInstances":5},{"platform":"WINDOWS","seleniumProtocol":"WebDriver","browserName":"internet explorer","maxInstances":1}],"configuration":{"port":5555,"register":true,"host":"10.235.132.161","proxy":"org.openqa.grid.selenium.proxy.DefaultRemoteProxy","maxSession":5,"hubHost":"10.235.1.50","role":"node","registerCycle":5000,"hub":"http://10.235.1.50:4444/grid/register","hubPort":4444,"url":"http://10.235.132.161:5555","remoteHost":"http://10.235.132.161:5555"}}
11:28:47.407 INFO - starting auto register thread. Will try to register every 5000 ms.
I need to run the tests on all available browsers on the vms simultaneously .
Selenium Grid give you the ability to run tests in parallel but doesn't handle it automatically. This is typically done with a test runner that handles it for you. Some frameworks like TestNG have this built-in but would likely need some customizing depending on your code.
Though it is preferred to use some frameworks such as testNG, you can run them with a plain java Threaded way too.
I should start off by saying that I am regretfully and painfully a noob. But I'm trying to change that!! I do not know any programming languages, but have managed to "make things happen" by doing enough research to get whatever job I've ever needed done done.
Anyway, I have been creating Selenium tests using the Selenium IDE and I am having a bit of trouble getting these test to run via Selenium Grid.
I have been exporting the tests as JUnit 4 (Webdriver) files. I am running the grid on a Ubuntu headless server, and my remote controls on one Windows 7 machine running IE9 and Firefox, and a Windows Vista machine running IE8 and Chrome.
My goal is to take the tests that I've exported from Selenium IDE as JUnit 4 (Webdriver) files and run them from the grid in parallel on my two Windows machines. I have edited my hosts files on my Windows machines to recognize the Ubuntu server by the name of "seleniumgrid". For example:
On the Ubuntu server terminal 1:
ant launch-hub
Win7 terminal1:
ant -Dport=5555 -Denvironment="IE9 on Windows" -Dhost=Win7 -DhubURL=http://seleniumgrid:4444 launch-remote-control
Win7 terminal2:
ant -Dport=5555 -Denvironment="Firefox on Windows" -Dhost=Win7 -DhubURL=http://seleniumgrid:4444 launch-remote-control
Vista termina1:
ant -Dport=5555 -Denvironment="IE8 on Windows" -Dhost=WinVista -DhubURL=http://seleniumgrid:4444 launch-remote-control
Vista terminal2:
ant -Dport=5555 -Denvironment="Chrome on Windows" -Dhost=WinVista -DhubURL=http://seleniumgrid:4444 launch-remote-control
Now, from here, I'm trying to launch the JUnit4 (webdriver) file that I have exported from Selenium IDE to run this configuration. The name of the file is titled : Registration.java.
What do I have to do now to run the Registration.jar file? I can't seem to find any documentation that answers this question, which leads me to believe that I have a fundamental misunderstanding of how this all works...
Pardon if this question has been answered before. I have poor terminology when it comes to this stuff.
HUGE thanks for taking the time to read this, and even more for an answer if there is one.
-brandon
There is no need to launch hub and nodes via ant. You can run them from cmd:
java -jar selenium-server-standalone-2.21.0.jar -role hub -- will run hub
java -jar selenium-server-standalone-2.21.0.jar -role node -hub http://seleniumgrid:4444/grid/register -- will run node
Default port for node is 5555, so for the second terminal you should specify port that differs from default one, e.g. 5556:
java -jar selenium-server-standalone-2.21.0.jar -role node -port 5556 -hub http://seleniumgrid:4444/grid/register
Also you should specify browser parameters for each node, e.g.:
-browser browserName=firefox,maxInstances=5,platform=WINDOWS
In your JUnit tests you should use RemoteWebDriver with DesiredCapabilities:
DesiredCapabilities capability = DesiredCapabilities.firefox();
WebDriver driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);
For parallel execution you should edit your tests additionally (sorry, don't work with jUnit, so can't help here much except of link that you can find below.)
Include JUnit class files, your class files, including your JUnit test classes, libraries your class files depend on in your classpath on Linux machine:
export CLASSPATH=$JUNIT_HOME/junit.jar:/myproject/classes:/myproject/lib/something.jar
Invoke the Junit command on Linux machine:
java org.junit.runner.JUnitCore [test class name]
Or you can use ant instead.
I will recommend to start with hub on Linux and one node with one browser on Windows without any parallelization, so you will be sure that this part works correctly. As a next step run tests for two nodes sequentially and then try to run them in parallel.
For complete tutorials read these materials: How do I run JUnit using Ant, Activating Junit tests from Command Line, Grid2 tutorial, Parallel JUnit 4 and Selenium (three parts)