I have a selenium automation framework which uses junit to run tests locally on a browser of my choice. I currently use junitparams to parameterize some of my tests. e.g
#RunWith(JUnitParamsRunner.class)
public class loginPage extends BaseTestClass{
#Test
#FileParameters(value = "src/test/resources/Test data/login.csv", mapper = CsvWithHeaderMapper.class)
public void login(String username, String pwd) throws Exception{
}
}
There are tests I have for logging into a website and I use junitparams with a csv file to run through multiple different login scenarios. I am now looking to start using saucelabs to run my tests across multiple different browser/os combinations simultaneously. My question is how do I achieve both the saucelabs parallel tests and parametrized tests at the same time? I have seen examples for saucelabs like the following:
https://github.com/saucelabs-sample-test-frameworks/Java-Junit-Selenium
But the issue I will run into is that I cannot use multiple different runners. I need to use a single runner as the Junit #RunWith annotatation requires. Is there an easy way to combine both the ConcurrentParameterized.class runner used in the saucelabs example and the JUnitParamsRunner.class I am currently utilising for local execution?
EDIT:
I found the following that confirms I cannot use 2 separate runners and appears to suggest merging two runners would be very difficult. Instead I'm guessing I will have to change the way sauce labs integration is handled. https://github.com/Pragmatists/junitparams-spring-integration-example
I would suggest taking a look at SauceryJ. It integrates Jenkins, the Sauce OnDemand plugin, and your testing code with SauceLabs.
Example class here.
Full disclosure: I wrote and maintain SauceryJ
Related
I have the question about the NUnit test setup (.NET Core 3.1 and NUnit 3) for Selenium tests in Visual Studio 2019.
In AssemblyInfo.cs, I add 2 lines.
[assembly: Parallelizable(ParallelScope.Children)]
[assembly: LevelOfParallelism(4)]
The Code is easy. Initialize the driver in the SetUp(). However, when using the test explorer to run 2 tests, 2 chrome windows are open. But they are not running in parallel ( Still not working using Setup, OneTimeSetup attributes)
If I initialize the driver in the TestMethod directly, it is fine , but it is the dupe code.
Does it mean NUnit Selenium Tests inside the same TestFixture cannot be running in parallel?
Thanks,
Ray
[TestFixture]
public class Account : BaseTest
{
[SetUp]
public void Setup()
{
_driver = new ChromeDriver();
_driver.Manage().Window.Maximize();
}
[Test]
[Category("UAT")]
[Order(1)]
public void Test1()
{
_driver.Navigate().GoToUrl("https://www.msn.com");
Assert.AreEqual("https://www.msn.com/", _driver.Url);
}
[Test]
[Category("UAT")]
[Order(0)]
public void Test2()
{
_driver.Navigate().GoToUrl("https://www.google.com");
Assert.AreEqual("https://www.google.com/", _driver.Url);
}
I had the same question, but I've managed to put together an example where VS2019 + Selenium + Parallelism does work, though I believe there is a limitation that seems likely to be what you are encountering (I'll speak to it at the end) based upon your example.
To make it work, I added an AssemblyInfo.cs file with the two attributes you noted:
[assembly: Parallelizable(ParallelScope.Children)]
[assembly: LevelOfParallelism(6)]
I tested having with the "Parallelizable" attributed to the class or in the AssemblyInfo.cs file and both worked. Including the "LevelOfParallelism" was required -- but in my tests, it was not required for parallel execution of non-Selenium unit tests.
My model involves executing my tests against multiple WebDrivers, which I am currently doing by passing an IEnumerable collection to each test using the [TestCaseSource] or [ValueSource] attributes. This allows me to reuse a WebDriver instance with each test to reduce the overall execution time (it is so expensive to spin up/down instances) and ensure correct clean-up, but I observed that although the tests are run in parallel, each WebDriver instance can only execute one test at a time. If you were using a different WebDriver instance with the second test, they would be executed in parallel.
I am trying to add RunListener to Karate ParallelRunner class. I have done this for Karate runner using #Karate.class and add a custom runner. I am writing the data to infuxdb and generating reports is grafana, I am able to successfully achieve it in karate runner. Below is the code snippet for the same. I am running my karate runner using this custom runner where I have added this listener. I want to achieve the same for parallel runner.
#Override
public void run(RunNotifier notifier) {
notifier.addListener(new ExecutionListener());
super.run(notifier);
This is not directly possible, the parallel runner is a very specific implementation and has nothing to do with JUnit by design.
Since you seem to be experienced in adding JUnit listeners and the like, you can refer to this code in case it gives you any ideas.
CliExecutionHook.java.
For more details about the "ExecutionHook" refer this: https://github.com/intuit/karate/issues/970#issuecomment-557443551
But let me say I think you are un-necessary trying to put effort into reports that will not really get you any benefit in the long run except for "looking good" :) And if you feel something needs to change in Karate, please contribute, it is open-source.
Thanks for your suggestion peter. I was able to send scenario and test run details to influx db in order to generate reports in grafana. I just made use of karate results extracted all the values required and called it in Junit #After.
public void writeScenarioResults(List<ScenarioResult> results){
String status;
for (ScenarioResult a:results) {
status=((a.isFailed()==true)?"FAIL":"PASS");
gb.sendTestMethodStatus(a.getScenario().getName(),status,build);
}
}
As per cucumber 4 with TestNG:
When using TestNG in parallel mode, scenarios can be executed in
separate threads irrespective of which feature file it belongs too.
Different rows in a scenario outline can be also executed in separate
threads. The two scenarios in feature1.feature file will be executed
by two threads in two browsers. The single feature2.feature will be
executed by another thread in a separate browser.
Now suppose I have a scenario like below in feature1:
1st scenario : Create an user with some details.
2nd scenario : Edit an user with some details.
Now if in TestNG if both scenario invoke at the same time then my 2nd scenario will fail for sure as the user is not created yet.
Do I just switch to Junit as:
When using JUnit in parallel mode, all scenarios in a feature file
will be executed in the same thread. The two scenarios in
feature1.feature file will be executed in one browser. The single
feature2.feature will be executed by another thread in a separate
browser.
Below function just having the parameter to run it as parallel.
#Override
#DataProvider(parallel = true)
public Object[][] scenarios() {
return super.scenarios();
}
So my main question is how to configure my test in parallel so my test can run systematically. i.e execute parallel per feature file, or any tag which can mark scenario depended on another like we have in TestNG #Test(dependsOnMethods = { "testTwo" }).
Kindly suggest any configuration setting for cucumber or strategy which can be use for same.
I want to run each .feature file as single TestNG test using Karate, how can I do it?
We are using karate to automate REST API's. We have custom listener which will record status of each TestNG test and other information to postgress DB.
One more way running by tags:
#CucumberOptions(tags = { "#getVersion" })
public class GetVersionTest extends KarateRunner {
}
Feature File:
#getVersion
Feature: Testing Rest API with Karate and Java
Scenario:
Given url 'https://......'
When method get
Then status 200
And match response contains '{version= x.xx.x}'
Did you try using the Karate TestNG support ? The documentation has details on how to use: https://github.com/intuit/karate#running-with-testng
As the developer of Karate I actually strongly recommend that teams don't use TestNG. My suggestion is that you use the new 'hooks' feature to call your custom Java code from Karate directly and you won't depend on TestNG or even JUnit.
If you still need a custom way of running, the TestNG support is actually a single class: KarateRunner.java. I really don't know if it runs each feature as a TestNG test or not since I am not a TestNG expert. But you should be able to create your own version of this Java code that does what you want and maybe contribute it back to the project.
I have an application made in php for which am using selenium for unit testing using phpUnit. The problem is that I have to set the environment before I can go for tests. For eg. I have to set session variables, login and fetch data from remote server. All this takes a lot of time and it is not feasible to re-set this in every test function.
I am looking for a method so that I can use the same browser session for running all the tests in it. I tried looking for resources online, but couldn't find any good sources for this. The code I have written is
protected function setUp()
{
parent::setUp();
$this->setBrowserUrl("http://localhost/devel/");
}
public function start()
{
parent::start();
$this->open("");
//Setting up the environment here
}
public function testFunction()
{
//A test function
}
public function testFunction2()
{
//Another test function
}
But this is opening browser instance for both the functions. Is there any work around for this? Or is there any command line parameter while launching selenium server for this?
"[I am] using selenium for unit testing using phpUnit"
No, you're not. You're using PHPUnit with selenium for functional testing. :-)
But since it's probably not in your best interest to re-invent that wheel, you want Mink: http://mink.behat.org/
It wraps around Guzzle and lets you do session-based acceptance testing using a bunch of different drivers. It has Goutte for a headless browser, and can work with Selenium and Sahi and a bunch of others.
Also of note, depending on your needs, is Behat: http://behat.org/
It lets you write client-readable test documents that can be turned into Mink-based acceptance tests.
HTH.
Question already answered.
The unaccepted answer did the job for me.
#see How do I run a PHPUnit Selenium test without having a new browser window run for each function?