How to manage the step definition code that needs to be reused again & again - selenium

My feature file contains a feature that needs to be reused by each and every step definition file again & again. How to manage the code
My feature is :-
"User is on home Page".
The above feature / scenario contains a code that needs to be reused again & again. In my code base , for every feature file i have written separate step definition.
My first step definition file :-
#Given ("^user is on HomePage$")
public void user_homePage()
{
configFileReader =new ConfigFileReader();
System.setProperty("webdriver.chrome.driver","D:\\chromedriver.exe");
driver=new ChromeDriver();
driver.get("https://parabank.parasoft.com/parabank/index.htm");
driver.manage().window().maximize();
}
Now the same feature needs to be used in another step definition file. i.e
here below
Before user clicks on 'Register' link , it should verify that user is on homePage. The description of 'User is on home page' is defined in first step definition file .
Now how to manage the code here :-
My Second Step definition file here below:-
import StepFiles.ParaBank_TC_01_Step; [ I have even imported first step definition file , so that feature "User is on home page" could be executed. ]
public class ParaBank_TC_02_Step {
public WebDriver driver;
ConfigFileReader configFileReader;
#When ("^user clicks on register link$")
public void click_register() throws InterruptedException
{
Thread.sleep(3000);
WebElement register_link= driver.findElement(By.xpath("//a[contains(text(),'Register')]"));
register_link.click();
}
Actual Result :-
1. When i write all the step definition for 2 features files in one file , then it executes perfectly fine , because feature 'User is on home page' is defined in one same file.
As i write separate step definition for feature 2 in another java file. It shows me "Null pointer exception" error because on feature "User is on home page" > driver is initialised in 1 step def file . It isn't executing for second step definition file.
Please help me out in understanding the root cause of this issue & provide the best possible solution.

In order to share state between steps, you can use dependency injection (DI). Cucumber offers support for several DI frameworks. We recommend you use either the one your application already uses / you are familiar with (in my case, that is Spring). Otherwise, we recommend PicoContainer as the most lightweight option.
You can find a little more information about using DI in the Cucumber docs and the related code on GitHub.
For more information on using PicoContainer, see this blogpost.
To use Spring, please have a look at my blogpost.
To use Guice, have a look at this blogpost.
Sidenote:
Feature-coupled step definitions (defining the step definitions for each feature in a separate file, in a way so that they cannot be reused across features), is considered an anti-pattern, as "This may lead to an explosion of step definitions, code duplication, and high maintenance costs."
(from the Cucumber docs).
The solution is to decouple your step definitions:
"
* Organise your steps by domain concept.
Use domain-related names (rather than feature- or scenario-related names) for your step & step definition files."
(from the Cucumber docs).
In order to do so, you will need to use DI.

If you are interested to implement PicoContainer, please follow below steps for good understanding :
Step 1. OrderSelectionStepDef & OrderDetailsStepDef would look like below (please change name as per your implementation)
/**
* Step Definition implementation class for Cucumber Steps defined in Feature file
*/
public class HomePageSteps extends BaseSteps {
TestContext testContext;
public HomePageSteps(TestContext context) {
testContext = context;
}
#When("^User is on Brand Home Page (.+)$")
public void user_is_on_Brand_Home_Page(String siteName) throws InterruptedException {
homePage = new HomePage().launchBrandSite(siteName);
testContext.scenarioContext.setContext(Context.HOMEPAGE, homePage);
}
#Then("^Clicking on Sign In link shall take user to Sign In Page$")
public void clicking_on_Sign_In_link_shall_take_user_to_Sign_In_Page() {
homePage = (HomePage) testContext.scenarioContext.getContext(Context.HOMEPAGE);
signInPage = homePage.ecommSignInPageNavigation();
testContext.scenarioContext.setContext(Context.SIGNINPAGE, signInPage);
}
For your reference
public class BaseSteps {
protected HomePage homePage;
protected PLPPage plpPage;
protected PDPPage pdpPage;
protected ShoppingBagPage shoppingBagPage;
protected ShippingPage shippingPage;
More implementation goes here.....
}
Step 2. Please add below 2 Classes under your framework -
First, Java file name - ScenarioContext.java
public class ScenarioContext {
private Map<String, Object> scenarioContext;
public ScenarioContext(){
scenarioContext = new HashMap<String, Object>();
}
public void setContext(Context key, Object value) {
scenarioContext.put(key.toString(), value);
}
public Object getContext(Context key){
return scenarioContext.get(key.toString());
}
public Boolean isContains(Context key){
return scenarioContext.containsKey(key.toString());
}
}
Second, Java file name - TestContext.java
public class TestContext {
public ScenarioContext scenarioContext;
public TestContext(){
scenarioContext = new ScenarioContext();
}
public ScenarioContext getScenarioContext() {
return scenarioContext;
}
}
Step 3. POM Dependency - picocontainer shall be as per your cucumber version
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-picocontainer</artifactId>
<version>${cucumber.version}</version>
</dependency>

Related

Cucumber- // Write code here that turns the phrase above into concrete actions

I Tried to execute the same test with different data but I am getting the following error when I tried to run the test
You can implement missing steps with the snippets below:
#When("^Enter the ADMIN and password(\\d+)$")
public void enter_the_ADMIN_and_password(int arg1) throws Throwable {
// Write code here that turns the phrase above into concrete actions
throw new PendingException();
}
Below is my Feature file
Scenario Outline: Login
Given Open chrome and login
When Enter the <Username> and <Password>
Then Click on Login
Examples:
|Username |Password |
|ADMIN |password123|
Below is the #when annotation part in Steps file
#When("^Enter the \"(.*)\" and \"(.*)\"$")
public void enter_the_Username_and_Password(String username,String Password) throws Throwable
{
System.out.println("This step enter the Username and Password on the login page."+username +Password);
loginObj.Login(driver);
}
It may be occurring due to cucumbers packages is misconfigured. Check the GLUE options
#RunWith(Cucumber.class)
#CucumberOptions(
features = { "src/test/resources/features/CucumberTests.feature" },
tags = { "#integration_test"},
glue = { "br.com.cucumber.integration.stepDefinition" },
format = { "pretty", "html:target/reports/cucumber/html", "json:target/cucumber.json", "usage:target/usage.jsonx", "junit:target/junit.xml" })
#ContextConfiguration(classes= AppConfiguration.class)
public class CumcumberTest extends BaseTestCase {
}
I think what you are describing as an "error" is in fact cucumber telling you that is not able to find the matching definition for the step "When Enter the Username and Password" and has auto-generated the example definition below based on the scenario outline data:
#When("^Enter the ADMIN and password(\\d+)$")
public void enter_the_ADMIN_and_password(int arg1) throws Throwable {
// Write code here that turns the phrase above into concrete actions
throw new PendingException();
}
The problem is that your step definition is using quotes on the parameters but your actual step has no quotes, hence cucumber is not able to match them. You either need to change your step in the feature file to include quotes:
When Enter the "<Username>" and "<Password>"
Or you need to change the step definition to not include quotes:
#When("^Enter the (.*) and (.*)$")
public void enter_the_Username_and_Password(String username,String Password) throws Throwable
{
System.out.println("This step enter the Username and Password on the login page."+username +Password);
loginObj.Login(driver);
}
I got the same error. For me the problem was that I created step definition in the package for back end tests instead of front end ones. If the step definition classes have same or similar names, it is possible to make a mistake. This would happen during chosing the wrong package while creating step definition from the feature file in Intellij.
I was getting this issue and discovered the reason was I accidentally gave the name of the class and not the package for the glue code:
The wrong argument value for my gluecode was my class as: gluecode="StepDefinition", when it should have been my package: gluecode="stepDefs"
My TestRunner.java looks as follows:
package testRunner;
import org.junit.runner.RunWith;
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
#RunWith(Cucumber.class)
#CucumberOptions(
features = "src/test/java/features",
glue = "stepDefs")
public class TestRunner {
}
I got the same error. In my case I was forgetting the .step. part in my file name.
So I've solved by modifying my file name from:
mystepfile.ts
to:
mystepfile.step.ts

Can "driver.findElement" be used as method and be called in a program

I am using "driver.findElement" multiple times. So can it be saved in a method and called multiple times?
driver.findElement(By.xpath("//tbody[#id='detailsstockid']/tr/td[12]/a/input")).click();
driver.findElement(By.id("supplier_name")).click();
driver.findElement(By.xpath("//select[#id='supplier_name']/option[7]")).click();
driver.findElement(By.id("catagory_name")).click();
driver.findElement(By.id("productname")).sendKeys("AutoProductNew");
driver.findElement(By.id("productcode")).sendKeys("ap02");
You just need a wrapper like this
public static WebElement findElement(By locator){
WebElement anElement = driver.findElement(locator);
return anElement;
}
findElement(By.id("id")).click();
Basically you can call whatever method available to the object
I think what you are trying to do is to simplify your code... maybe make it more human readable. This is a noble goal but I would suggest that you take a different approach. Without more info, I don't know what your typical script is like. From the example code you provided, I would assume that you are going to a few pages, clicking on things, filling in textboxes, etc. I would suggest that you look into the page object model.
There are a LOT of references on the web on this topic, here's a good one to start with.
http://www.seleniumhq.org/docs/06_test_design_considerations.jsp#page-object-design-pattern
The basic concept is create one Class file per page. [Page doesn't always literally mean an entire page. For example, I would create a separate page object for each popup dialog.] What this allows you to do is to consolidate all the code specific to that page into a single location. If you know what an API is and understand it, you will basically create an API for each page/dialog you touch. If done properly, it will greatly simplify and organize your code and will make creating new scripts drastically easier and they will be much easier to read. The benefits are numerous... one big one is code maintenance is made SO much easier. Imagine you have 100 scripts that all start by logging into the site. Let's assume the design of the login page changes... without the page object model, you will have to update all 100 scripts individually. If you are using the page object model, you open the LoginPage.java class, make the change to the Login() method, and you just fixed all 100 scripts.
You may be asking... how does this answer my question? What it does is turns
driver.findElement(By.id("supplier_name")).click();
into
somePage.clickSupplierName();
in your actual script. I think this is more along the lines of what you are asking without answering your specific question. The .findElement code still exists, but is "hidden away" in the page object.
A simple example. I wrote a script, with supporting page objects, to navigate to Amazon.com, search for Star Wars, return the # of found items, and then navigate to the Today's Deals page. The entire code is below.
program.java (Main script)
public class program
{
public static void main(String[] args)
{
WebDriver driver = new FirefoxDriver();
driver.manage().window().maximize();
driver.get("https://www.amazon.com");
// HOME PAGE
HomePage homepage = new HomePage(driver);
homepage.search("star wars");
// SEARCH RESULTS PAGE
SearchResultsPage searchResultsPage = new SearchResultsPage(driver);
System.out.println(searchResultsPage.getNumberOfResults());
// navigate to the Today's Deals page
searchResultsPage.clickTodaysDealsLink();
// TODAY'S DEALS PAGE
// do more stuff
driver.close();
driver.quit();
System.out.println("Done");
}
}
HomePage.java (Home page page object)
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
public class HomePage
{
WebDriver driver;
private By searchTextboxLocator = By.id("twotabsearchtextbox");
public HomePage(WebDriver driver)
{
this.driver = driver;
}
public void search(String searchString)
{
driver.findElement(searchTextboxLocator).sendKeys(searchString + "\n");
}
}
SearchResultsPage.java (Search results page page object)
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
public class SearchResultsPage
{
WebDriver driver;
private By todaysDealsLinkLocator = By.linkText("Today's Deals");
private By numberOfResultsLocator = By.id("s-result-count");
public SearchResultsPage(WebDriver driver)
{
this.driver = driver;
}
public String getNumberOfResults()
{
// grabs only the total count of search results from the string in the form "1-16 of 7,488,146 results for "
return driver.findElement(numberOfResultsLocator).getText().split(" ")[2];
}
public void clickTodaysDealsLink()
{
driver.findElement(todaysDealsLinkLocator).click();
}
}

Managing web driver code separately

Can you help me with this question.
I am working on a test automation framework and I want to keep my webdriver code separately and initiate from my test cases.
please help me in this. - Thanks
Go for the POM Model with Encapsulation . In the Example below I have created a Test Setup class which can then be used in all the test classes.
So this would be you TestSetup Class-
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.IE;
namespace dummy
{
[SetUpFixture]
public class TestSetup
{
public static IWebDriver driver;
[OneTimeSetUp]
public void testSetup()
{
if (driver == null)
{
//Local Tests
driver = new FirefoxDriver();
}
}
public static bool IsElementPresent(By by)
{
try
{
driver.FindElement(by);
return true;
}
catch (NoSuchElementException)
{
return false;
}
}
[OneTimeTearDown]
public void TearDown()
{
driver.Quit();
}
}
}
And Below can be you TestCase Class . Notice that you are just calling the test setup .So your test class will inherit all the functions from the Primary class-TestSetup
using System;
using OpenQA.Selenium;
using NUnit.Framework;
using System.Threading;
namespace dummy
{
[TestFixture]
[Parallelizable]
public class TESTCASE
{
IWebDriver driver = TestSetup.driver;
[Test]
public void siteVisit()
{
driver.Navigate().GoToUrl("http://google.com");
Assert.IsTrue(String.Equals("Google, driver.Title));
}
}
}
Hi if i have understood it correctly then please go for POM (Page object Framework)
1.The main advantage of Page Object Model is that if the UI changes for any page, it don’t
require us to change any tests, we just need to change only the code within the page objects
(Only at one place).
2.Page Object model is writing all the functionalities / reusable components of a page that
we want to automate in a separate class.
3.As per google wiki Page object **"Within your web app’s UI there are areas that your tests
interact with. A Page Object simply models these as objects within the test code. This
reduces the amount of duplicated code and means that if the UI changes, the fix need only
be applied in one place."**
Example : google home page
1.Say now if we consider our google home page as Home page.
2.For the above pages we will create class as HomePage.class.
3.In class we will identify and write reusable methods which are specific to Home page.
4.'google home page' which will have many options like Search, Sign In, +You, Images etc.
5.Now all functionalities that we want to automate should have reusable methods/components
for each page.
6.as our main page is google page we can navigate to other pages by clicking on any link
from the google page. When ever we are navigating to other page, we need to return that page
object. Else Return the current page object as this action doesn't navigate to a other page
represented by another Page Object.
advantages.
1.There is clean separation between test code and page specific code such as locators
(or their use if you’re using a UI map) and layout.
2.There is single repository for the services or operations offered by the page rather than
having these services scattered through out the tests.
In both cases this allows any modifications required due to UI changes to all be made in one place.Click for more on POM

Using SpecFlow, Selenium and FluentAutomation causing problems

I am currently using Specflow with Selenium and FluentAutomation, and am running into significant problems with maintaining state between steps in specflow.
See my example code below:
[Binding]
public class RegistrationSteps : FluentTest
{
[Given(#"I create an account")]
public void GivenICreateAnAccount()
{
new HomePage(this)
.Go()
.StartRegistration()
.EnterDetailsAndClickSubmit(); // takes me to deposit page
}
[When(#"Deposit '(.*)' dollars in my account")]
public void GivenDepositMoneyInMyAccount(int amount)
{
new DepositPage(this)
.EnterDetailsAndClickSubmit(amount);
}
}
My problem is:
In the first step the page is loaded using Go() and everything happens fine
In the second step my tests continue, here I expect I am on a different page, based in the Submit in the previous
Because I am no on a different PageObject it gets confused, I don't use Go because the previous step shouldve brought me here, and at this stage it wont find the expected elements
So my question is, how can I use one browser session and several PageObjects across multiple Specflow tests?
According to the FluentAutomation doc, you should do something like this:
[Binding]
public class RegistrationSteps : FluentTest
{
private PageObject _currentPage;
[Given(#"I create an account")]
public void GivenICreateAnAccount()
{
_currentPage = new HomePage(this)
.Go()
.StartRegistration()
.EnterDetailsAndClickSubmit(); // takes me to deposit page
}
[When(#"Deposit '(.*)' dollars in my account")]
public void GivenDepositMoneyInMyAccount(int amount)
{
_currentPage = _currentPage
.EnterDetailsAndClickSubmit(amount);
}
}
Provided that you return the page object that is switched to in the EnterDetailsAndClickSubmit method of your concrete page object like:
Public PageObject EnterDetailsAndClickSubmit() {
// [.. enter details here and click submit ..]
return this.Switch();
}

Is Page Object Model linking compatible with Cucumber's Gherkin?

With Test Automation's Page Object Model we link pages together like this:
WebDriver driver = new WebDriver()
HomePage homePage = new HomePage(driver);
LoginPage loginPage = homePage.GoToLoginPage();
WelcomePage welcomePage = loginPage.Login();
etc
etc
The big benefit of this is if the Devs change the homepage so it no longer links to the loginpage, I can update my homepage class and see all the tests I need to update (with errors) before even running a test.
With Gherkin however, each row above would form a separate 'Step' and therefore a separate method. Therefore, how can this linking be done?
Is the only way to place instances of the page object classes (e.g. homePage, loginPage, etc) into a cross gherkin statement persistant store (e.g. like a specflow POCO or 'World')?
Ok so having asked numerous dev and test automation experts, it seems the solution is to continue with linking [e.g. WelcomePage welcomePage = loginPage.loginWithValidUser(validUser)] is the way to go.
To persist instance of page objects across steps (e.g. welcomePage in example above) you can use dependency injection tool (creating functionality similar to World extensions in Ruby's implementation of cucumber).
Here is more info:
https://cukes.info/docs/reference/java-di
However, most projects will benefit from a Dependency Injection module
to organize your code better and to share state between Step
Definitions.
More info from SpecFlow (the .net official cucumber implementation):
http://specflow.org/getting-started/beyond-the-basics/
And finally, I have created a whole blog around this area that might help people out, since gherkin/page object interaction is a subject of great interest to me:
http://www.seligmanventures.com/dev-blog/test-automation-page-object-model-with-gherkin
When it comes to most websites (where url's can be used), in my opinion it is best practice to simply use the url instead of an action to get to that same url.
For instance:
# Suggested by OP:
driver = Selenium::Webdriver.for :chrome, prefs: prefs
homepage = Homepage.new(driver)
login = homepage.go_to_login
welcome = login.log_in_as('dave4429')
# My Suggestion:
homepage = Url.new('/')
login = Url.new('/login')
welcome = Url.new('/welcome')
This means that you start from a url instead of having to start at the homepage for every test. You would still have the methods that you suggested, but they would be used in other areas, in order to make sure that the user can access the page through means other than the url.
However, this is not a one stop shop solution. With mobile and desktop applications, your only option may be to go through the home screen, in which case, the method you suggested is definitely the one to go for.
"Page objects themselves should never make verifications or assertions. This is part of your test and should always be within the test’s code, never in an page object." - Selenium HQ
The example I gave was a very basic one, and I would most likely wrap these into modules and classes, to enable coding like this:
google = Project::Pages::Google.new
google.search_for('Hello, World!')
expect(google.found_result?).to_equal(true)
Edit
In addition to this, you seem to have a misconception about how Cucumber works with Gherkin.
You can have multiple lines of code per step, as the step itself is a description of the actions within the step.
For instance:
Given I am logged in as "dave4429"
When I have submitted the "Contact Us" form with the following data:
| dave4429#example.com | David McBlaine | I want to find out more about your Data Protection services, can I talk to a staff member or get a PDF? |
Then an email should be sent to "support#example.com" with the details specified
The definition for the "When" may look like this:
When(/^I have submitted the "Contact Us" form with the following data:$/) do |table|
rows = table.raw
row = rows[0]
contact_us.fill_form({email: row[0], username: row[1], message: row[2]})
contact_us.submit_message
expect(browser.title).to_equal("Message Sent!")
end
It all depends on how much you break down the steps within the definition.
Edit #2
It's also clear to me that you want to do method chaining, something in the way of contact_us.fill_form({email: row[0], username: row[1], message: row[2]}).submit_message, which again, isn't out of the question while using the techniques that I'm suggesting, but the question of whether this chaining should be for each individual page, or whether everything should be included in one class or module, can only be answered by your needs.
It's just my opinion that this would put too much into a single class, and that breaking down that class will allow for more control to be given to the testers, and less redundant code will be written.
Another option I have seen recently would be to store the Page Object instances as Static variables that can be accessed from any class?
After much discussion on this topic, an equally plausible alternative is to not return instances of new pages when using the page object pattern with gherkin. You will lose the benefit of linking that you normally get with POM, but the code will arguably read better and be less complex. Posting this alternative answer, so as a test community we can vote which method is peoples preference.
This can be a bit tricky with Cucumber and Selenium. I've developed a pattern that involves extension methods to the IWebDriver interface for Selenium allowing me to navigate to specific pages using the page objects. I register the IWebDriver object with the SpecFlow dependency injection framework, and then my step definition classes are free to initialize whichever page objects they need.
Registering Selenium Web Driver With SpecFlow
You just need to plug in to the before/after scenario hooks to manage the web driver object:
[Binding]
public class WebDriverFactory
{
private readonly IObjectContainer container;
public WebDriverFactory(IObjectContainer container)
{
this.container = container;
}
[BeforeScenario]
public void CreateWebDriver()
{
var driver = new ChromeDriver(...);
// Configure Chrome
container.RegisterInstanceAs<IWebDriver>(driver);
}
[AfterScenario]
public void DestroyWebDriver()
{
var driver = container.Resolve<IWebDriver>();
if (driver == null)
return;
// Capture screenshot if you want
// var photographer = (ITakeScreenshot)driver;
driver.Quit();
driver.Dispose();
}
}
Then It's a matter of gluing step definitions and page objects together using some extensions on the IWebDriver interface.
Selenium Page Objects
Keep your page objects navigating to one another. For instance the HomePage allows you to navigate to the "Create blog post" page, and returns the page object for that page:
public class HomePage
{
private readonly IWebDriver driver;
private readonly WebDriverWait wait;
private IWebElement CreatePostLink => driver.FindElement(By.LinkText("Create New Blog Post"));
public HomePage(IWebDriver driver)
{
this.driver = driver;
wait = new WebDriverWait(driver, 30);
}
public AddEditBlogPostPage ClickCreatePostLink()
{
CreatePostLink.Click();
wait.Until(d => d.Title.Contains("Create new blog post"));
return new AddEditBlogPostPage(driver);
}
}
And subsequently, the AddEditBlogPostPage returns the BlogPostListingPage when you create a new blog post:
public class AddEditBlogPostPage
{
private readonly IWebDriver driver;
private IWebElement Title => driver.FindElement(By.Id("Title"));
private IWebElement PostDate => driver.FindElement(By.Id("Date"));
private IWebElement Body => driver.FindElement(By.Id("BodyText"));
private IWebElement SaveButton => driver.FindElement(By.XPath("//button[contains(., 'Save Blog Post')]"));
public AddEditBlogPostPage(IWebDriver driver)
{
this.driver = driver;
}
public BlogPostListingPage CreateBlogPost(BlogPostDataRow data)
{
Title.SendKeys(data.Title);
PostDate.SendKeys(data.Date.ToShortDateString());
Body.SendKeys(data.Body);
SaveButton.Click();
return new BlogPostListingPage(driver);
}
}
Step Definitions To Glue Things Together
The step:
When I create a new blog post:
| Field | Value |
| Title | Selenium Page Objects and Cucumber |
| Date | 11/1/2019 |
| Body | ... |
Would have this definition:
[Binding]
public class BlogPostSteps
{
private readonly IWebDriver driver;
public BlogPostSteps(IWebDriver driver)
{
this.driver = driver;
}
[When(#"I add a new blog post:")]
public GivenIAmAddingANewBlogPost(Table table)
{
var addBlogPostPage = driver.GoToCreateBlogPostPage();
var blogPostData = table.CreateInstance<BlogPostDataRow>();
addBlogPostPage.CreateBlogPost(blogPostData);
}
}
The driver.GoToCreateBlogPostPage(); is an extension method on IWebDriver that kicks off the navigation from one page object to another:
public static class SeleniumPageNavigationExtensions
{
public static AddEditBlogPostPage GoToCreateBlogPostPage(this IWebDriver driver)
{
var homePage = new HomePage(driver);
return homePage.ClickCreatePostLink();
}
}
This gives you the flexibility to keep your page objects "pure" and devoid of SpecFlow, Cucumber and Gherkin. You can use these same extension methods and page objects in other tests that do not utilize Gherkin or behavior driven development. This allows for easy reuse of your test classes. Your test projects should be just as purposefully architected as the actual application it tests.