NUnit, Run Parametrized Tests in Parallel with Selenium - selenium

I'm having an issue trying to do as the title says. When i run this locally, it does launch 2 chrome instances, however it uses only one of the browsers for both tests, as opposed to using each browser for each test. Any ideas how how to set this up correctly?
public class BaseClass
{
public IWebDriver driver;
[SetUp]
public void BaseSetUp()
{
driver = new ChromeDriver();
driver.Manage().Window.Maximize();
}
[TearDown]
public void BaseTearDown()
{
driver.Quit();
}
}
[Parallelizable(ParallelScope.All)]
[TestFixture]
public class DerivedClass : BaseClass
{
[TestCase("https://www.python.org/", "Welcome to Python.org")]
[TestCase("https://www.w3.org/", "World Wide Web Consortium (W3C)")]
public void Test3(string url, string title)
{
driver.Navigate().GoToUrl(url);
Thread.Sleep(4500);
Assert.AreEqual(driver.Title, title);
}
}

You are indeed creating the driver twice, but you are storing both instances in the same member field, driver. The reason this happens is that NUnit uses the same instance of a test class for all the tests within that class. (A new feature will be available in a future release to use a separate instance, but that's no help to you right now.)
As the tests run in parallel, the first test to run performs its setup and stores the driver in that field. Then the second test starts and stores it's instance in the same field. It's not possible to predict exactly when - during the execution of the tests- that replacement will take place. However, it will most likely happen consistently as you re-run on the same machine.
In the case of the example, the solution is simple. If you want each test case to use a separate driver instance, create the driver within the test itself, perhaps by calling a method that centralizes all your initialization.
An important thing to remember is that the ParallelizableAttribute is used to tell NUnit that the test may be run in parallel. NUnit accepts that as a promise, but if you share state between tests they won't run correctly.

Related

NUnit Selenium tests inside the same TestFixture cannot be running in parallel

I have the question about the NUnit test setup (.NET Core 3.1 and NUnit 3) for Selenium tests in Visual Studio 2019.
In AssemblyInfo.cs, I add 2 lines.
[assembly: Parallelizable(ParallelScope.Children)]
[assembly: LevelOfParallelism(4)]
The Code is easy. Initialize the driver in the SetUp(). However, when using the test explorer to run 2 tests, 2 chrome windows are open. But they are not running in parallel ( Still not working using Setup, OneTimeSetup attributes)
If I initialize the driver in the TestMethod directly, it is fine , but it is the dupe code.
Does it mean NUnit Selenium Tests inside the same TestFixture cannot be running in parallel?
Thanks,
Ray
[TestFixture]
public class Account : BaseTest
{
[SetUp]
public void Setup()
{
_driver = new ChromeDriver();
_driver.Manage().Window.Maximize();
}
[Test]
[Category("UAT")]
[Order(1)]
public void Test1()
{
_driver.Navigate().GoToUrl("https://www.msn.com");
Assert.AreEqual("https://www.msn.com/", _driver.Url);
}
[Test]
[Category("UAT")]
[Order(0)]
public void Test2()
{
_driver.Navigate().GoToUrl("https://www.google.com");
Assert.AreEqual("https://www.google.com/", _driver.Url);
}
I had the same question, but I've managed to put together an example where VS2019 + Selenium + Parallelism does work, though I believe there is a limitation that seems likely to be what you are encountering (I'll speak to it at the end) based upon your example.
To make it work, I added an AssemblyInfo.cs file with the two attributes you noted:
[assembly: Parallelizable(ParallelScope.Children)]
[assembly: LevelOfParallelism(6)]
I tested having with the "Parallelizable" attributed to the class or in the AssemblyInfo.cs file and both worked. Including the "LevelOfParallelism" was required -- but in my tests, it was not required for parallel execution of non-Selenium unit tests.
My model involves executing my tests against multiple WebDrivers, which I am currently doing by passing an IEnumerable collection to each test using the [TestCaseSource] or [ValueSource] attributes. This allows me to reuse a WebDriver instance with each test to reduce the overall execution time (it is so expensive to spin up/down instances) and ensure correct clean-up, but I observed that although the tests are run in parallel, each WebDriver instance can only execute one test at a time. If you were using a different WebDriver instance with the second test, they would be executed in parallel.

Session management in Selenium webdriver

I have a class under which I have different methods one of them is login and other are related to the adding product to cart, checkout and shipping.
When I try to run the methods using TestNG in one go, it execute all the methods in different browsers session and I lost my login session.
I want some solution so that either all the methods execute in same browser or I can use the session of login method in other consecutive methods
Thanks in advance
I have same login for both
I am creating new instance of driver in both methods
Here is my code :
public class purchase {
#Test
public void login(){
System.setProperty("webdriver.chrome.driver", "{path}/chromedriver.exe");
WebDriver fd= new ChromeDriver();
fd.get("{domain}/login/");
/*{login script here}*/
}
#Test
public void purchaseItem(){
System.setProperty("webdriver.chrome.driver", "{path}/chromedriver.exe");
WebDriver fd1= new ChromeDriver();
fd1.get("{domain}/travel");
/*add item to cart*/
}
}
It means there might be 2 issue:
1. You have separate login in both the methods
2. You are creating new instance of driver in both methods.
Please share your code and will help you with resolving it.
You have to share the same driver instance between methods. Otherwise, you will have a new session each time, like you described it.
Here you go.. as per your comment - "I am creating new instance of driver in both methods"..
So, if you want to maintain the session, you have to use a single instance of driver in both methods.
You can achieve it using beforeClass method (Use it for driver initializing and login), and write your logic in different test methods.. So it will make sure complete test cycle for the class is 1 and then your session will not be lost.
Let me know if this helps.

Webdriver.quit() ,not working properly when called in #AfterSuite annotation

I have a two "test" in testng XML file, all are running parallel what my issue is that I called a Webdriver.quit() method in the #AfterSuite annotation, but it is only closing the last created browser instance not all opened browsers.
when I call in the #AfterTest annotation it gets closed.
below is my Testng file:
I created the driver in the base class and extend it in test classes.
code for creating driver instance.
#BeforeClass
#Parameters({"deviceName","platformName","platformVersion","udid","browserName","browserVersion","nodeIP","nodePort"})
public void startBrowser(String deviceName, String platformName, String platformVersion, String udid, String browserName, String browserVersion, String nodeIP, String nodePort)
{
DesiredCapabilities capabilities =DesiredCapabilities.chrome();
System.setProperty("webdriver.chrome.driver", "chromeDriverPath");
ChromeOptions options = new ChromeOptions();
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
driver = new ChromeDriver(capabilities);
}
AfterSuite is called once per suite. What you have shown above is one suite executing two tests in parallel. You need to make sure your driver is thread safe.
AfterTest gets called after <test> tag is over - so your webdriver if specific to a thread will get closed.
If you want to close all webdriver in aftersuite, you need to build a list of webdrivers and close each one individually.
Because you have 2 <test>, TestNG is creating 2 instances (with the same after suite method).
But testng consideres a suite method as single in the suite and will only call it once (the first it finds).
That's why it is working with after test method which looks to be the one you should use.
BTW, as you want to close the driver in an after suite method, I suppose you create it in a before suite method which is not supposed to work for the same reasons. Could you detail where/how you create the driver ?

JUnit - Postponed assert failure

I'm automating functional tests using JUnit. And I ran into a problem: if I follow the rule "one (significant) assert per test method" then I end up with a bunch of 6-line test methods per one test case (17 is the biggest number yet). If I put them all into one test method I have to comment out failing asserts or leave half of the test never launched.
I don't like the first way because it launches browser for too many times, and it appears that browser launch + login/logout are more "expensive" and time consuming than test run itself.
The second way is no better because it introduces a lot of manual work in any case of managing it.
So, my questions are:
1. What are the best practices for such cases?
2. Is there some way to postpone test failure till the end of test? I mean, less important assert, that doesn't stop test run but causes it to fail in the end nonetheless.
UPD: Yes, I'm using Selenium. And I have a parent class for every test class to unify their settings.
You can use #BeforeClass and #AfterClass to launch and shutdown the Browser once for each test class or you can create a Rule that launches your Browser and use it with #ClassRule.
ErrorCollector may be helpful.
#BeforeClass
public static void beforeClass(){
launchBrowser();
}
#Before
public void before() {
login();
}
#AfterClass
public static void beforeClass(){
killBrowser();
}
That could be the answer to your problem

How to get Selenium and TestNG to open one browser to run tests in multiple classes

I am using Selenium with TestNG to test a website. I have created tests using the Selenium IDE and exported them to TestNG with each test being a method in a class. Eg,
For login tests there is a Login class which has methods testLogin(), testLogin2() etc
For signup tests there is a Signup class has methods testSignup(), testSignup2(), etc
I am using Ant to run the tests which works fine except that each class will open up a browser and then run its methods, eg, if I have five classes, then five browsers will open simultaneously and then run the tests.
What I want is to get Ant/Selenium/TestNG to just open up one browser and then run all the tests (in the same browser) in all the classes that I have specified in testng.xml. Using the example above, I want one browser to open then run testLogin(), testLogin2(), testSignup(), testSignup2().
If this cannot be achieved, then I want to open a browser, run all tests in a class then close the browser then open another browser then run the set of test methods in the next class.
Any help appreciated. Thanks in advance.
Today I have found the answer that works for me. Give me a few minutes to gather all code samples :)
Init.java
//base class that will be called before all tests
#Test(groups = "init")
public class Init{
DefaultSelenium browser;
public void start(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
browser.open("url");
browser.click("xpath");
}
}
TemplateForClasses.java
/* - all public methods will be tests
* - all tests will be called after group init
* - just before suite will start we will start 1 browser instance
*/
#Test(dependsOnGroup="init")
public class TemplateForClasses{
DefaultSelenium browser;
#BeforeSuite
public void startBrowser(ITestContext itc){
browser = new DefaultSelenium(host,port,browser_type,url);
itc.setAttribute("browser",browser);
browser.start();
}
#AfterSuite
public void stopBrowser(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
browser.stop();
}
//any other: #Before, #After methods
}
FirstGroupOfTests.java
//all tests classes will inherit preferences set in TemplateForClasses
public class FirstGroupOfTests extends TemplateForClasses{
public void FirstTest(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
//browser.click("start");
}
}
idea:
start browser just once have tests
that run before every other tests(isBrowserRunning)
refer to browser from single test
This code was tested but currently I took it from the top of my head so possibly I will edit it tomorrow to make it more exact.
Update:
This result is based on testng.org documentation + some questions asked by me on stackoverflow + some answers found on several forums/groups
I must add I'm running testng programatically and I'm generating xml on the fly (as it is done on documentation.org). I am using it all in one package, I added package to the xml, included only classes Init + the ones that inherit from TemplateForClasses. If you need that xml, let me know.
I did this with Spring's dependency injection. And the init code is in a factory. I needed a way to have a Selenium instance shared not only between tests but between helper classes. Very seldom is selenium.someMethod() called directly in the tests. It more like helper.goToSomePage() or preferencesPage.changePassword(....).
It could be considered a bad idea to have a Selenium instance shared between tests, but the few bugs it brought were not that hard to find. The tests are run sequentially and the Selenium object need not be thread-safe. The state of the object must be kept consistent though.
For info, Spring is a Java framework and Dependency injection is only a part of it. Other DI frameworks like Guice can of course be used instead.
I too was stuck in the same problem for quite some time. I'll explain it in the simplest terms possible. Consider the following example:
Class A (contains the code selenium.start();)
|
|(inherited classes)
|--------class B }
|--------class C } Have some #Test methods
|--------class D }
Now everytime we run these test methods it will execute the code in the parent class constructor selenium.start(); Thats when the multiple browsers will all open up on your screen.
Now one by one the test methods will get executed - suppose tests in class B are executed they will be happening in one window, for class C another and so on.
So basically, all you have to do is remove the start() code from the parent constructor and put it somewhere in the classes B, C and D.
As long as you keep working with one selenium object everything will happen in one browser window. When you put start(); that browser will open (if it wasnt open) and a new session is created. stop(); and the session is terminated.
The flow of control goes like this=>
Class A, Class B
Class A, Class C
Class A, Class D
So if you can figure out a way to keep using the same selenium object with only 1 start() and 1 stop() for the entire execution sequence shown above, your test execution will happen in only one browser window.
If you put start() code in class A and stop code in each of B,C and D then you will have 3 windows open and one by one they will close as execution progresses.
If you put start() and stop() code individually in B,C and D then you will see one browser opening, executing test cases, closing. Another will then open, execute test cases for C, close etc.
Hope this helps. :-)