I am attempting to shave some time off my tests in Quarkus by saving off resources in static members of utilities in my test sources. Examples here being test containers and a Selenium web driver. I am saving these off so that ideally between tests/ profile changes, I can keep the same instances of these services that took a long time to instantiate.
However, it seems that when the tests re-run the test server, the static members that hold these are wiped out, thus killing the benefits I was trying to achieve.
How can I keep these static members from being wiped out, or otherwise keep some data between server instances?
Example of where I am trying to do this:
https://github.com/Epic-Breakfast-Productions/OpenQuarterMaster/blob/main/software/open-qm-base-station/src/test/java/com/ebp/openQuarterMaster/baseStation/testResources/lifecycleManagers/TestResourceLifecycleManager.java
public class TestResourceLifecycleManager implements QuarkusTestResourceLifecycleManager {
public static final String EXTERNAL_AUTH_ARG = "externalAuth";
private static MongoDBContainer MONGO_EXE = null;
private static KeycloakContainer KEYCLOAK_CONTAINER = null;
...
In this example, I realize the test resources might not get torn down as planned, but ideally that could be handled in a lifecycle event, but am still working on why the static members disappear. One step at a time.
Related
i was reading the blogs for "What about the code? Why does the code need to change when it has to run on multiple machines?".
And i came across one line which i am not getting it, can anyone please help me to understand it with simple or any example.
"There should be no static instances in the class. Static instances hold application data and when a particular server goes down, all the static data/state is lost. The app is left in an inconsistent state."
Assuming: Stating instance is an instance which can be at most once per process or context - e.g. in java there is at most one copy of a static class, with all data (or state) that the class contains.
So it is very simple memory model for a static class in a single node/jvm/process. Since there is a single copy of data, it is quite straightforward to reason about it. For example, you one may update the data and every next reader will see the updated information. This is a bit more complicated for multithreaded programs, but is still straightforward comparing to distributed systems.
Clearly in a distributed system, every node may have at most one static class with state. Which means if a system contains several nodes - a distributed system - then there are several copies of data.
Having several copies is a problem. It is hard to reason about such system - every node may have some unique data and data may differ on different node. It is very hard to reason about such data: how it is synced? Availability vs consistency?
For example, take a simple counter. In a single node system, a static instance may keep the score. If one writer increased the counter, the next reader will see the increased value (assuming multithreaded part is implemented correctly, which is not that complicated).
Same counter is a distributed system is much more complicated. A writer may write to one node, but a reader may read from another.
Basically, having state on nodes is a hard problem to solve. This is the primary reason to use some distributed storage layer e.g. Hbase, Cassandra, AWS DynamoDB. All these storage systems have predictable behaviour which helps to reason about correctness of programs.
For example, there are just two servers which accepts payments from clients.
Then somebody decided to create static class to be friendly with mutli threading:
public static class Payment
{
public static decimal Amount;
public static bool IsMoneyReceived;
public static string UserName;
}
Then some client, let's call him John, decided to buy something in shop. John sent money and static class has data about this purchase. Some service is going to write data into database from
Payment class, however, electicity was turned off. Load balancer knows that the server is not responding and redirects John requests to another server which knows nothing about data in
Payment class.
I'm having an issue trying to do as the title says. When i run this locally, it does launch 2 chrome instances, however it uses only one of the browsers for both tests, as opposed to using each browser for each test. Any ideas how how to set this up correctly?
public class BaseClass
{
public IWebDriver driver;
[SetUp]
public void BaseSetUp()
{
driver = new ChromeDriver();
driver.Manage().Window.Maximize();
}
[TearDown]
public void BaseTearDown()
{
driver.Quit();
}
}
[Parallelizable(ParallelScope.All)]
[TestFixture]
public class DerivedClass : BaseClass
{
[TestCase("https://www.python.org/", "Welcome to Python.org")]
[TestCase("https://www.w3.org/", "World Wide Web Consortium (W3C)")]
public void Test3(string url, string title)
{
driver.Navigate().GoToUrl(url);
Thread.Sleep(4500);
Assert.AreEqual(driver.Title, title);
}
}
You are indeed creating the driver twice, but you are storing both instances in the same member field, driver. The reason this happens is that NUnit uses the same instance of a test class for all the tests within that class. (A new feature will be available in a future release to use a separate instance, but that's no help to you right now.)
As the tests run in parallel, the first test to run performs its setup and stores the driver in that field. Then the second test starts and stores it's instance in the same field. It's not possible to predict exactly when - during the execution of the tests- that replacement will take place. However, it will most likely happen consistently as you re-run on the same machine.
In the case of the example, the solution is simple. If you want each test case to use a separate driver instance, create the driver within the test itself, perhaps by calling a method that centralizes all your initialization.
An important thing to remember is that the ParallelizableAttribute is used to tell NUnit that the test may be run in parallel. NUnit accepts that as a promise, but if you share state between tests they won't run correctly.
I am using NUnit for the Selenium C# project. In which I have many test methods. For getting data (from excel) I am using a public static method that returns IEnumerable<TestCaseData> which I am calling at test method level as TestCaseSource. I am facing challenges now, as I start executing on the test method it is invoking all the static methods which are there in the project.
Code looks like this:
public static IEnumerable<TestCaseData> BasicSearch()
{
BaseEntity.TestDataPath = PMTestConstants.PMTestDataFolder + ConfigurationManager.AppSettings.Get("Environment").ToString() + PMTestConstants.PMTestDataBook;
return ExcelTestDataHelper.ReadFromExcel(BaseEntity.TestDataPath, ExcelQueryCreator.GetCommand(PMTestConstants.QueryCommand, PMTestConstants.PMPolicySheet, "999580"));
}
[Test, TestCaseSource("BasicSearch"), Category("Smoke")]
public void SampleCase(Dictionary<string, string> data)
{
dosomething;
}
Can someone help me how can I restrict my data call method to the respective test method?
Your TestCaseSource is not actually called by the test method when you run it, but as part of test discovery. While it's possible to select a single test to execute, it's not possible to discover tests selectively. NUnit must examine the assembly and find all the tests before it's possible to run any of them.
To make matters worse, if you are running under Visual Studio, the discovery process takes place multiple times, first before the tests are initially displayed and then again each time the tests are run. This is made necessary by the architecture of the VS Test Window, which runs separate processes for the initial disovery and the execution of the tests.
That makes it particularly important to minimize the amount of work done in test discovery, especially when running under Visual Studio. Ideally, you should structure the code so that the variable parameters are recorded during discovery. The actual data access should take place at execution time. This can be done in a OneTimeSetUp method, a SetUp method or at the start of the test itself.
Finally, I'd say that your instinct is correct: it should be possible to set up a TestCaseSource, which only runs if the test you select is about to be executed. Unfortunately, that's a feature that NUnit doesn't yet have.
I've previously written some selenium tests using ruby/rspec, and found it quite powerful. Now, I'm using Selenium with PHPUnit, and there are a couple of things I'm missing, it might just be because of inexperience. In Ruby/RSpec, I'm used to being able to define a "global" setup, for each test case, where I, among other things, open up the browser window and log into my site.
I feel that PHPUnit is a bit lacking here, in that 1) you only have setUp() and tearDown(), which are run before and after each individual test, and that 2) it seems that the actual browser session is set up between setUp() and the test, and closed before tearDown().
This makes for a bit more clutter in the tests themselves, because you explicitly have to open the page at the beginning, and perform cleanups at the end. In every single test. It also seems like unnecessary overhead to close and reopen the browser for every single test, in stead of just going back to the landing page.
Are there any alternative ways of achieving what I'm looking for?
What I have done in the past is to make a protected method that returns an object for the session like so:
protected function initBrowserSession() {
if (!$this->browserSession) {
$this->setBrowser('*firefox');
$this->setBrowserUrl('http://www.example.com/');
//Initialize Session
$this->open('http://www.example.com/login.php');
// Do whatever other setup you need here
}
$this->browserSession = true;
}
public function testSomePage() {
$this->initBrowserSession();
//Perform your test here
}
You can't really use the setupBefore/AfterClass functions since they are static (and as such you won't have access to the instance).
Now, with that said, I would question your motivation for doing so. By having a test that re-uses a session between tests you're introducing the possibility of having side-effects between the tests. By re-opening a new session for each test you're isolating the effects down to just that of the test. Who cares about the performance (to a reasonable extent at least) of re-opening the browser? Doing so actually increases the validity of the test since it's isolated. Then again, there could be something to be said for testing a prolonged session. But if that was the case, I would make that a separate test case/class to the individual functionality test...
Although I agree with #ircmaxell that it might be best to reset the session between tests, I can see the case where tests would go from taking minutes to taking hours just to restart the browser.
Therefore, I did some digging, and found out that you can override the start() method in a base class. In my setup, I have the following:
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class SeleniumTestCase extends PHPUnit_Extensions_SeleniumTestCase
{
public function setUp() {
parent::setUp();
// Set browser, URL, etc.
$this->setBrowser('firefox');
$this->setBrowserUrl('http://www.example.com');
}
public function start() {
parent::start();
// Perform any setup steps that depend on
// the browser session being started, like logging in/out
}
}
This will automatically affect any classes that extend SeleniumTestCase, so you don't have to worry about setting up the environment in every single test.
I haven't tested, but it seems likely that there is a stop() method called before tearDown() as well.
Hope this helps.
I am using Selenium with TestNG to test a website. I have created tests using the Selenium IDE and exported them to TestNG with each test being a method in a class. Eg,
For login tests there is a Login class which has methods testLogin(), testLogin2() etc
For signup tests there is a Signup class has methods testSignup(), testSignup2(), etc
I am using Ant to run the tests which works fine except that each class will open up a browser and then run its methods, eg, if I have five classes, then five browsers will open simultaneously and then run the tests.
What I want is to get Ant/Selenium/TestNG to just open up one browser and then run all the tests (in the same browser) in all the classes that I have specified in testng.xml. Using the example above, I want one browser to open then run testLogin(), testLogin2(), testSignup(), testSignup2().
If this cannot be achieved, then I want to open a browser, run all tests in a class then close the browser then open another browser then run the set of test methods in the next class.
Any help appreciated. Thanks in advance.
Today I have found the answer that works for me. Give me a few minutes to gather all code samples :)
Init.java
//base class that will be called before all tests
#Test(groups = "init")
public class Init{
DefaultSelenium browser;
public void start(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
browser.open("url");
browser.click("xpath");
}
}
TemplateForClasses.java
/* - all public methods will be tests
* - all tests will be called after group init
* - just before suite will start we will start 1 browser instance
*/
#Test(dependsOnGroup="init")
public class TemplateForClasses{
DefaultSelenium browser;
#BeforeSuite
public void startBrowser(ITestContext itc){
browser = new DefaultSelenium(host,port,browser_type,url);
itc.setAttribute("browser",browser);
browser.start();
}
#AfterSuite
public void stopBrowser(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
browser.stop();
}
//any other: #Before, #After methods
}
FirstGroupOfTests.java
//all tests classes will inherit preferences set in TemplateForClasses
public class FirstGroupOfTests extends TemplateForClasses{
public void FirstTest(ITestContext itc){
browser = (DefaultSelenium) itc.getAttribute("browser");
//browser.click("start");
}
}
idea:
start browser just once have tests
that run before every other tests(isBrowserRunning)
refer to browser from single test
This code was tested but currently I took it from the top of my head so possibly I will edit it tomorrow to make it more exact.
Update:
This result is based on testng.org documentation + some questions asked by me on stackoverflow + some answers found on several forums/groups
I must add I'm running testng programatically and I'm generating xml on the fly (as it is done on documentation.org). I am using it all in one package, I added package to the xml, included only classes Init + the ones that inherit from TemplateForClasses. If you need that xml, let me know.
I did this with Spring's dependency injection. And the init code is in a factory. I needed a way to have a Selenium instance shared not only between tests but between helper classes. Very seldom is selenium.someMethod() called directly in the tests. It more like helper.goToSomePage() or preferencesPage.changePassword(....).
It could be considered a bad idea to have a Selenium instance shared between tests, but the few bugs it brought were not that hard to find. The tests are run sequentially and the Selenium object need not be thread-safe. The state of the object must be kept consistent though.
For info, Spring is a Java framework and Dependency injection is only a part of it. Other DI frameworks like Guice can of course be used instead.
I too was stuck in the same problem for quite some time. I'll explain it in the simplest terms possible. Consider the following example:
Class A (contains the code selenium.start();)
|
|(inherited classes)
|--------class B }
|--------class C } Have some #Test methods
|--------class D }
Now everytime we run these test methods it will execute the code in the parent class constructor selenium.start(); Thats when the multiple browsers will all open up on your screen.
Now one by one the test methods will get executed - suppose tests in class B are executed they will be happening in one window, for class C another and so on.
So basically, all you have to do is remove the start() code from the parent constructor and put it somewhere in the classes B, C and D.
As long as you keep working with one selenium object everything will happen in one browser window. When you put start(); that browser will open (if it wasnt open) and a new session is created. stop(); and the session is terminated.
The flow of control goes like this=>
Class A, Class B
Class A, Class C
Class A, Class D
So if you can figure out a way to keep using the same selenium object with only 1 start() and 1 stop() for the entire execution sequence shown above, your test execution will happen in only one browser window.
If you put start() code in class A and stop code in each of B,C and D then you will have 3 windows open and one by one they will close as execution progresses.
If you put start() and stop() code individually in B,C and D then you will see one browser opening, executing test cases, closing. Another will then open, execute test cases for C, close etc.
Hope this helps. :-)