I'm in the process of developing an automation framework using Selenium Webdriver following the PageObject pattern where there are 2 layers:
Test classes (Test specification)
PageObject classes (Test implementation)
My test classes contain TestNG annotation methods (#Test, #BeforeTest, #AfterTest etc) where I'm creating objects of the PageObject classes and calling the methods in those classes, which will actually interact with the AUT.
The way I'm passing data to the tests is by including:
All the locators (css/xpath/class/id locators) of elements in the
application.
All the test-data needed
in a properties file (For ex: testDataSet1.properties). I'm creating a HashMap<String, String> of the properties file and passing it as a parameter to all the pageobjects which will read the data from the hashmap and use them in the script. I'm not passing test data at all from the test classes.
What my problem is, I am not sure whether it is appropriate to read test-data in PageObjects. I get a feeling that this approach is very inflexible because I am tying test data to PageObjects instead of tying it to the tests. So when I need to perform data-driven tests / run the automation suite in multiple iterations, this will not work. Because, currently if I need to use different data each time I need to manually go and change the key specified in the hashmap which will fetch me a different value from the properties file.
Any suggestions?
I personally like having the page object act as a the web page itself, providing me actions to perform and validation methods to use. For example, I keep all of my CSS, ID, etc. selectors within the page object itself. This way, if there is ever an update to the webpage which breaks the tests, I simply go to the corresponding Page Object for that web page and update the selector there.
As far as test data, I have some other files (classes, properties, etc.) that I use to pull test data from eg. test users. The test classes themselves pull this test data and pass it into the Page Objects (when needed), or the Page Object methods for validation purposes.
An example of a test data class (mine is a bit more complex, but this is a simple example):
public TestUser(){
username = getUniqueUser();
password = "";
name = "Test User";
email = getUniqueEmail(username);
}
tl;dr;
I keep the html selectors IN their respective Page Objects (can use sub-page objects for re-use if needed).
I keep the test data mixed between classes, property files, and test classes.
Related
I've been seraching for a while how should I test my data access layer with not a lot of success. Let me list my concerns:
Unit tests
This guy (here: Using InMemoryConnection to test ElasticSearch) says that:
Although asserting the serialized form of a request matches your
expectations in the SUT may be sufficient.
Does it really worth to assert the serialized form of requests? Do these kind of tests have any value? It doesn't seem likely to change a function that should not change the serialized request.
If it does worth it, what is the correct way to assert these reqests?
Unit tests once again
Another guy (here: ElasticSearch 2.0 Nest Unit Testing with MOQ) shows a good looking example:
void Main()
{
var people = new List<Person>
{
new Person { Id = 1 },
new Person { Id = 2 },
};
var mockSearchResponse = new Mock<ISearchResponse<Person>>();
mockSearchResponse.Setup(x => x.Documents).Returns(people);
var mockElasticClient = new Mock<IElasticClient>();
mockElasticClient.Setup(x => x
.Search(It.IsAny<Func<SearchDescriptor<Person>, ISearchRequest>>()))
.Returns(mockSearchResponse.Object);
var result = mockElasticClient.Object.Search<Person>(s => s);
Assert.AreEqual(2, result.Documents.Count()).Dump();
}
public class Person
{
public int Id { get; set;}
}
Probably I'm missing something but I can't see the point of this code snippet. First he mocks an ISearchResponse to always return the people list. then he mocks an IElasticClient to return this previous search response to any search request he makes.
Well it doesn't really surprise me the assertion is true after that. What is the point of these kind of tests?
Integration tests
Integration tests do make more sense to me to test a data access layer. So after a little search i found this (https://www.nuget.org/packages/elasticsearch-inside/) package. If I'm not mistaken this is only about an embedded JVM and an ES. Is it a good practice to use it? Shouldn't I use my already running instance?
If anyone has good experience with testing that I didn't include I would happily hear those as well.
Each of the approaches that you have listed may be a reasonable approach to take, depending on exactly what it is you are trying to achieve with your tests. you haven't specified this in your question :)
Let's go over the options that you have listed
Asserting the serialized form of the request to Elasticsearch may be a sufficient approach if you build a request to Elasticsearch based on a varying number of inputs. You may have tests that provide different input instances and assert the form of the query that will be sent to Elasticsearch for each. These kinds of tests are going to be fast to execute but make the assumption that the query that is generated and you are asserting the form of is going to return the results that you expect.
This is another form of unit test that stubs out the interaction with the Elasticsearch client. The system under test (SUT) in this example is not the client but another component that internally uses the client, so the interaction with the client is controlled through the stub object to return an expected response. The example is contrived in that in a real test, you wouldn't assert on the results of the client call as you point out but rather on the output of the SUT.
Integration/Behavioural tests against a known data set within an Elasticsearch cluster may provide the most value and go beyond points 1 and 2 as they will not only incidentally test the generated queries sent to Elasticsearch for a given input, but will also be testing the interaction and producing an expected result. No doubt however that these types of test are harder to setup than 1 and 2, but the investment in setup may be outweighed by their benefit for your project.
So, you need to ask yourself what kinds of tests are sufficient to achieve the level of assurance that you require to assert that your system is doing what you expect it to do; it may be a combination of all three different approaches for different elements of the system.
You may want to check out how the .NET client itself is tested; there are components within the Tests project that spin up an Elasticsearch cluster with different plugins installed, seed it with known generated data and make assertions on the results. The source is open and licensed under Apache 2.0 license, so feel free to use elements within your project :)
I have heard about both terms Injectable and Embeddable many times, but I am not getting actual meaning of it.
Please help me to understand both clearly.
Injectable means that something can be created and added to the main script while the script is running.
Embeddable means something can be added to a script or code before running it i.e before compilation or running of the script.
For better understanding lets take a website with a textbox as a context.
Now, In the textbox, suppose its very basic one. So, I can add a javascript into the textbox and when I will submit, it will run my JS script. This way, I am injecting my own script into the main page.
Now, suppose, I add an Iframe of another website to the HTML file of my website. In this way, when the website will be viewed, it contains the iframe. In this way, the Iframe is embedded to the website.
Injectable means that the object can be created and injected at run time. This is a hint to the compiler that this object will be managed outside the scope of the compilation and can be used at runtime to determine if the object was intended to be injected.
Embeddable means that the object can be serialized and stored in a column instead of as a separate table when the containing object is persisted. That also implies the lifetime of the embedded object is the same as the lifetime of the containing object.
I have a big issue With Webdriver (Selenium 2).
In my test code, I find all the elements in the beginning of my test, and do some actions on them (like click(), checking attributes, etc.). My problem is that my page is refreshed and re-load my elements, and Webdriver don't know to recognize the elements again.
I know that I can find my elements again, but in some functions I don't know my XPath/ids, and I get just the WebElements, not XPath/IDs.
Am I right in saying that it's no possible to read elements into variables and re-use them?
WebElement element = driver.findElement(By.id("xyz"));
The above line will store the element object in element. You can certainly pass this element to other functions to make use of it over there.
We generally follow a pattern called PageObject patterns where we create all objects of a page as members of a class and instantiate them at once. This way we can use them any where in our project. For example all objects in Login page will be created as public static variables in a class called LoginPage. The constructor of LoginPage class will find elements and store them.
Next time any where you want to access an object of LoginPage, we access them as below(asuming that you have created elements userName and submit)...
LoginPage.userName.sendKeys("buddha");
LoginPage.submit.click();
However as Robie mentioned there is a chance for this objects to become unaccessible using the previously created object after page refresh. You can use the below modified approach for ensuring these objects are always found.
Instead of creating the objects as a member variable, create a getmethod for each object that you may need to use.
class LoginPage
{
public static WebElement getUserName()
{
return driver.findElement(By.id("xyz"));
}
}
Once LoginPage is defined that way, next time you want to use userName, you use below syntax.This way you don't have to give locators to the functions that needs to use these objects.
LoginPage.getUserName().sendKeys("buddha");
By using this approach, you can ensure that the objects are always accessible.
Buddha is incorrect in the following statement:
You can reuse it any number of times, however, it only works as long as the id doesn't change.
As you have correctly observed, if the page reloads, then elements become stale, even if the original object is still displayed on screen. In fact, refreshing of HTML via AJAX calls can also make objects stale even if the URL has not changed.
This is how Selenium works, and you have to understand this when deciding how to implement a test framework.
You can store elements, reuse them and pass them to functions, but understand when they will become stale and need to be refound.
In my current project, I have a very AJAX heavy application in which objects are continually becoming stale, so have extended WebElement to find and store it's HTML Id when constructed, then refinds by id if a stale element exception occurs and re performs the method that failed. However, this was achieved using Ruby and very specific to my application as I know every object has a unique HTML Id. I do not believe this approach would work for most applications under test.
I would also question whether storing elements in public static variables populated on construction, is actually following the Page Object pattern. I have never seen it implemented this way before, and can see lots of potential pitfalls. Lazy instantiation may be a better approach when following the Page Object pattern.
I'm designing a web service running on Google App Engine that scrapes a number of websites and presents their data via a RESTful interface. Based on some background reading, I think I'd like to attempt Test Driven Development (TDD) and develop my tests before I write any business code.
My problem is caused by the fact that my list of scraped elements includes timetables and other records that change quite frequently. The limit of my knowledge on TDD is that you write tests that examine the results of code execution and compare these results to a hardcoded result set. Seeing as the data set changes frequently, this method seems impossible. Assuming that this is true, what would be the best approach to test such an API? How would a large-scale web API be tested (Twitter, Google, Netflix etc.)?
You have to choose the type of test:
Unit tests just test proper operation of your modules (units). You provide input data and test that code outputs proper results. If there are system dependent classes you try to mock them or in case of GAE services, you use google provided local services. Unit tests can be run locally on your machine or on CI servers. There are two popular unit test libs for java: Junit & TestNG.
Integration tests check that various modules (internal & external) work together - they basically check that APIs between modules are working. They are usually run on real servers and call real external services. They are technology specific and are harder to run.
In your case, I'd go with unit tests and provide sets of different input data which you logic should parse and act upon. Since your flow is pretty simple (load data from fixed Url, parse it) you could also embed loading of real data into unit tests (we do this when we parse external sources).
From what you are describing you could easily find yourself writing integration tests. If your aim is to test the logic for processing what is returned from the scraped data (e.g. you know that you are going to get a timetable in a specific format coming in and you now have logic to process that data) you will need to create a SEAM between your web services logic and your processing logic. Once you have done this you should be able to mock the data that is returned from the web service call to always return the same table data and then you can write consistent unit tests against it.
public class ScrapingService : IScrapingService
{
public string Scrape(string url)
{// scraping logic}
}
public interface IScrapingService
{
string Scrape(string url);
}
public class ScrapingProcessor
{
private IScrapingService _scrapingService
// inject the dependency
pubilc ScrapingProcessor(IScrapingService scrapingService)
{
_scrapingService = scrapingService;
}
public void Process(string url)
{
var scrapedData = _scrapingService.Scrape(url)
// now process the scrapedData
}
}
To test you can now create a FakeScrapingService that implements the IScrapingService interface and then return whatever data you like from the Scrape method. There are some very good Mocking frameworks out there that make this type of thing easy. My personal favorite is NSubstitue.
I hope this explanation helps.
How would I test the following code?
Public Sub SetSerialIdForDevice()
Try
Dim component As Object = container.getComponentRef("componentInterface")
If component IsNot Nothing Then
component.SetupDeviceSerialID(container.serialNumbers)
Else
serialfound = False
End If
Catch ex As Exception
'' error handling
End Try
End Sub
Project references (or components as they're called here) are loaded at runtime into a singleton 'container.' We call the component that interfaces with a device by using the container.getComponentRef("< name of component we're looking for >"). We then invoke a method on this component to set the serial id, which is stored in a property of the container object.
In the SetupDeviceSerialID() method we call properties native to the 'container' (such as if it's an internal debug mode) as well as some other objects. What would be the best way to test this scenario where we have objects within objects? Would we mock all the calls, properties, and objects in order to isolate the test for SetupDeviceSerialID()?
Or do we mock the 'componentInterface' that's returned and mock the call for SetupDeviceSerialID() and then test the properties that were changed within SetupDeviceSerialID()?
EDIT
I've been thinking about testing this incorrectly (obviously) and through the answers I've come to realize that I was trying to cram testing for methods executed deeper in the code, into tests for the SetSerialIdForDevice() method.
So as a result,
If a serialID is found, we would set serialfound = true inside of SetupDeviceSerialID().
Is that something we would test here (since we will be testing for serialfound=false), or in a test for SetupdDeviceSerialID()? And would we create a test to see if SetupDeviceSerialID() actually exists on the "componentInterface" component?
I would mock container to have the getComponentRef return a mock object which the method can test against. Mocking each "componentInterface" class needs to be something that happens in their own dedicated unit tests. Don't combine testing responsibilities because its convenient, keep everything as its own unit so no unit test is dependent on another test.
The mores Seams you can put into your code, the easier it becomes to test.
If you can replace the return value of the getComponentRef method with a Test Double, you can write one test that verifies that this method has been invoked correctly and then move on to writing other unit tests that verifies something else.
Ideally, you should only be writing one test that tests any particular piece of behavior.
Assuming you can replace the component variable with a Test Double, you could then verify that the SetupDeviceSerialID method is called correctly.
That, as well as some tets that exercise the error paths, should then conclude the test suite for the SetSerialIdForDevice method.
You can then move on to write a new set of tests that verifies that a particular 'component' implementation works as intended, but those would be separate tests that are independent of the tests that exercise the SetSerialIdForDevice method.