I would like to run multiple Selenium Tests (on a Jenkins server) at the same time.
It currently runs only a single test at a time cause ChromeDriver seems to communicate over a special port. So somehow I guess I have to pass some kind of port settings via Selenium to the ChromeDriver to start up multiple tests.
The Selenium website unfortunately is empty for that topic:
http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#parallelizing-your-test-runs
From my point of view it makes no difference if the Test runs locally or on Jenkins, the problem is the same. We need to somehow configure ChromeDriver. The question is just how.
Anybody has some ideas or pointers where to look at and what files are involved to get this done?
You can run multiple instances of chromedriver locally quite easily, just instantiate multiple driver objects, chromedriver will keep the profiles separate and find a port to run on all by itself.
Here a link to an example that can run multiple tests using TestNG and Maven:
https://github.com/Ardesco/Selenium-Maven-Template
Just clone the above project and run the following in the command line:
mvn verify -Pselenium-tests -Dbrowser=chrome -Dthreads=2
It takes advantage of TestNG's ability to manage the thread pool and will open up multiple instances if specified. You can do the same thing with jUnit but you'll need to write a custom test runner to fire the tests off into individual threads.
If you decide to use gradle it can deal with managing the thread pools for you with both TestNG and jUnit and a lot of people prefer it to maven.
This is an old question, but for anyone still reading along, it is very possible to run multiple Selenium WebDriver instances in parallel without using Grid. I have successfully tested this using Chrome, FireFox, and PhantomJs (up to 5). Each WebDriver instance uses an isolated context, so session conflict should not be an issue. Be wary of server side conflicts though, depending on the requirements of your website!
For NUnit users, NUnit 3.2.1 now has a 'TestContext.Current.WorkerId' property that will allow you to isolate one WebDriver instance per NUnit worker.
Running multiple browsers on the same machine will often hinder performance, so be careful not to use too many browsers instances, or you may actually increase your testing time!
What you are looking for is Selenium Grid 2.
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to run the tests against a vast combination of browsers / OS.
minimize the maintenance time for the grid by allowing you to implement custom hooks to leverage virtual infrastructure for instance.
I agree using grid in combination with Maven parallelized class, you can run multiple instance in one PC. Jenkins is possible when you are using Ant for your build ,then you can specify which test can be run parallel.
Its quite easy to set it up though ;)
Related
I made a Selenium code by Java to test a certain web-application that our company developed.
Now, developers fix the application quite often and every time they update or fix, testers should test this new version of webapp to assert all functions working fine before application release.
Let's suppose that there are 100 testers who do not know how to run Selenium code or install Java.
I decide to create a testing server so that testers can access this server and run test. They also can see all test histories and details so far as well.
Is it possible and realistic to develop a system that runs like server and client? If yes, Can Jenkins do that? Other solutions are welcome!
Thank you in advance and happy new year!
Jenkins is one of the tools, you able to use for this, since it provides a simple way to delegate some tasks to already configured envs, nodes share them for multiple users and hide technical complexity. Also this would be aligned to your CI process, e.g. first - deploy the new code to test env, next - run test automation.
But the same also might be said for some other CI tools, so I suggest to pick some CI tool which your development already uses.
The architecture could be:
1 CI task for run tests -->
2 CI Node or docker image with java, selenium,
maven(gradle), it may be some headless Linux -->
3 Selenium cluster which able to launch multiple
selenium sessions (to cover your testers needs). It could
be some selenium cloud service, or configured
onpremise env. -->
4 Selenium grid hub (may be headless)-->
5 Selenium grid nodes... The final nodes env
should match your test requirements. It could
be Docker with linux (headless or not) or
Windows/MacOs.
Pick some tools and look for quick start guides/tutorials.
Start from simple implementation and improve it continuously.
I may say that for many cases Docker + Ubuntu + Headless Chrome is fine, lightweight and rapid.
Some references (examples the tools I've used):
Jenkins + Selenium + Maven https://www.lambdatest.com/blog/selenium-maven-jenkins-integration/
Selenoid (selenium grid implementation based on docker containers) https://github.com/aerokube/selenoid
Report Portal (just reporting tool) - something more than the default testng report provides. https://reportportal.io/
This is very shortly. The same might be done with a lot of other tools.
I have big task in work. I need to run my tests in parallel to reduce time of execute. At the moment i have around 150 end to end tests which are testing windows desktop app and web app (~1,5h) but i need to run it with 4 different systems which we support, so execution time went up to around 6h for all.
Runner for my tests is nunit, code is written with C#, and everything is stored in TFS. I need to create new project in Jenkins (Pipeline or other within i can achive my goal). I tried a lot of methods, but i don't know how to run multiple test on few VMs or RDPs (to have isolated desktop where i can use mouse and keyboard for each test). Did someone of you challenging same problem as i mentioned here and have some examples to try? I downloaded Parallel tests executor plugin for jenkins but don't have knowledge of how to use it with TFS repository.
At the moment, it is possible to run the same test on different browser assuming that you have all those browsers installed on your machine and that those browsers have a version compatible with your drivers version.
Is there a way, such as a library or everything else, that allows us to run the same test on different versions of the same browser, downloading those browsers on demand (bit like WebDriverManager does with the drivers)?
We want a test harness runnable on every machine and every operating system and we don't want to rely on any browser already installed in that machine.
A tool that allows us to choose on what browser and what version we want to run the test, it downloads a sandboxed or virtual version of that browser, and it runs the test.
I heard also about Saucelabs, but the systems under test are not available by publicly available URLs because they are in a dev environment.
I thought we could use Docker, but in that case we have to use a Headless browser, and we want to keep to possibility to see what the test is actually doing.
Thanks in advance.
No longer do you need (as we did in the dark old days) to manage our own infrastructure and have machines running instances of browsers and configure a Selenium Grid.
SauceLabs is a Selenium Grid in the cloud and is most definitely the recommended approach here. They offer a wide (ever growing) array of platforms.
With all that infrastructure taken care for you in the cloud the problem changes to one of correctly providing SauceLabs the DesiredCapabilities
I would highly recommend you also use Jenkins and its Sauce OnDemand plugin which allows you to select your target environments and at the same time integrate cross device testing into your build pipeline.
Constructing the DesiredCapabilities objects for SauceLabs in your test code is not a straightforward exercise however and can lead to hard to maintain spaghetti code if not done with a lot of discipline.
There are some NuGet packages that solve the SauceLabs DesiredCapabilities problem and integrates with Jenkins as well.
Saucery2 for NUnit2 (NuGet, Source, Example)
Saucery3 for NUnit3 (NuGet, Source, Examples)
SauceryJ for JUnit4 (Source, Example)
Saucery is DesiredCapability factory for SauceLabs. Full disclosure: I wrote Saucery and maintain it.
Saucery pulls in all the dependencies you would want allowing you to write tests against:
Desktop browsers (simply extend SauceryBase)
with Selenium;
or native app tests against:
IOS (simply extend SauceryIOSBase); or
Android (simply extend SauceryAndroidBase) devices
with Appium.
The home page provides benefits and a How To.
Presently we built a Automation framework which uses Selenium Webdriver+ specflow + Nunit, and we suing bamboo as our CI to run our Job against our every build.
we written a build.xml to handle our targets (like clean, init, install latest build, run Selenium scripts, uninstall build. etc)
ant command will read the tag name from the build.xml and runs the respective feature/scenarios based on Tags (like #smoke, #Regression)with Nunit in CI machine.
Now our requirement is to use Selenium Grid to divide scripts into different machine and execute with above set-up. Grid has to divide the scripts based on feature file or based on Tags.How to achieve this.
Is there any thing need to done under [BeforeFeature] and [BeforeScenario] ?
If you provide in details steps or any link which explains detail steps that would be a great help.
Please any one can help in this regards.
Thanks,
Ashok
You have misunderstood the role Grid plays in distributed parallel testing. It does not "divide the scripts", but simply provides a single hub resource through which multiple tests can open concurrent sessions.
It is the role of the test runner (in your case Specflow) to divide tests and start multiple threads.
I believe that you require SpecFlow+ (http://www.specflow.org/plus/), but this does have a license cost.
It should be possible to create your own multithread test runner for Specflow but will require programming and technical knowledge.
If you want a free open source approach to parallel test execution in DotNet, then there is MbUnit (http://code.google.com/p/mb-unit) but this would require you to rewrite your tests
I would like to run local Selenium test script written in Java, via Jenkins/Hudson. Is it possible to run scripts from my local windows machine? So far I have written some simple Selenium tests in Java, which run perfectly if I execute them via Eclipse IDE. I would be thankful for an in-depth explanation.
Selenium test in Java: assuming them to be laid out as unit tests (junit or testng), second assumption is that project is governed by either ant or maven, so there is some test (or rather integration-test) target or phase being present and be running smoothly when invoked from IDE.
When such tests are launched, they hit to a running selenium server (remote-control) which in turn launch a browser and runs its magic. Here are options: selenium server might be running in background (and be always available), or it might be started right before that testing and shut down afterwards. The latter is a common case for maven: pre-integration-test phase is configured to launch selenium rc, (then integration-test phase runs the tests against it), post-integration-test shuts selenium rc down.
So up to this moment we might want to learn what targets (ant) or phases(goals) your IDE invokes when it launches your tests successfully (also, what variables it sets or what profiles it enables).
If you invoke the same command from cmd (like 'mvn clean integration-test') and it runs successfully (same as IDE), then just instruct jenkins to run the same goals/targets (I assume that jenkins is running on the same machine, at the same user session).
If cmd doesn't do the trick (and you've looked quite well into what IDE does for you when it launches your tests), then I'd asked for more details.
So, involved participants are: 0. ant/maven that run your junit tests 1. selenium rc that should be running in bg or launched by ant/maven first 2. browser (path to browser executable) 3. jenkins (which was assumed to be running in the same environment).
If any of the assumptions are false, please come up with more details of your setup.