Test case management and selenium? - selenium

Just wondering how other people manage their test cases that are written for selenium test automation? I've been investigating maybe integrating with testlink to show the results and all, but I already have Jenkins set up for my environment and running my tests. What I'm really looking for is some way to nicely document my tests, like what steps each test performs for non-programmers.
I'm using Selenium with python, and Jenkins to run the tests.

I've tried two ways:
1) You can use Cucumber to write test steps like this:
#sanity #home_page #test_628
Scenario: Launch Support FAQs & Guides from Home cog
Given I navigate to Home page
When I click "Support" user logo link
Then I should see Support FAQs & Guides app launched
Each step written on Gherkin language (Given, When, Then) you can implement then using Selenium (I have to write on Ruby now, so I use Watir instead of Selenium).
see https://github.com/cucumber/cucumber/wiki/Python
2) But before that I used Robot Framework + Python + Selenium + Jenkins.
You can write test like this:
Go To Google Page [Documentation] Go to google page and search something
Open Browser to Google Page
Input Search selenium
Submit Search
This is done using human readable keywords. It prints very nice reports and can be easily integrated with Jenkins.
see http://www.wallix.org/2011/07/26/how-to-use-robotframework-with-the-selenium-library/

To document a test case and that too for non-programmers, Behaviour-Driven Development (BDD) is the best way. Some of the BDD tools are Cucumber, JBehave etc.
For example, lets assume the we have an website to manage some employees and we have a test case in which admin wants to add a new employee.
So the above test case can be documented using Cucumber as shown below:
Feature: Admin : Employees : Add Employees : Add Employee Details
Scenario: Verify that admin user is able to add new employee
Given Joe is an admin user
Given Joe is logged into the admin account
When Joe clicks "Add Emplyee" button
And Joe enters "Test Employee" in "Employee Name" text box
And Joe enters "12/05/1988" in "Birth Date" text box
And Joe selects "Male" from "Gender" radio button
And Joe clicks on "Submit" button
Then Joe gets message that says "Employee is added successfully."
The above test case can be easily understood by a non-programmer and he can perform each steps as performed by the test.
Hierarchy of folders for this test case will be feature/Admin/Employees/Add Employees/Add Employee Details
Hope this helps :)

I don't use TestLink. I think it is much easier to use a Jenkins job server to schedule my tests and it can do nearly the same job as testlink can and, im my opinion, is easier for other people to understand. I check my test code into Subversion and Jenkins checks out the latest code before it runs my tests. Jenkins has plugins to handle JUnit and TestNG report formats and has email capability, etc.
NOTE: The only Gotcha is that if you use RemoteWebDriver and a grid, the tests need to be ran by a Jenkins slave server that is running in a foreground process ( not in the background as a service). This is the case at least on Windows and might not be a problem on Linux.

We use Cucumber to create readable tests and have the results pulled back to Enterprise Tester, our Test Management platform. We can also pull Selenium tests and jUnit tests from Jenkins into Enterprise Tester, also for visibility across both manual and automated tests and traceability back to stories / requirements.

Related

Using Selenium IDE for a test-case which includes backend call?

I add a new category in the admin panel and want to ensure that the category is available in the dropdown on the user's part of the website. Recorded test in the Selenium IDE works fine. But the thing is, the task that I execute is of course not a pure frontend thing - the category is saved in the database and is loaded from it to show it to the user. So if something goes wrong on the database-side, the test will fail.
My question is: is it bad practice to do such tests that depend on backend-behavior ? Should I go for Selenium Webdriver ?
If you use Selenium Webdriver, your test will not change in a main thing. It still will check database side. Selenium Webdriver is just anouther tool for testing that is more flexible and allows to make more complex test then in Selenium IDE.
I don't think that it is bad practice, because it is just one of the tests that chould be executed to enshure you that this part of your project works correctly. In this case I would check back-end part(get all categories from DB or admin's panel and check that there is no extra or missing ones) and than check user's panel(all categories are the same as set in DB and admin's panel).

Specflow - same test to be repeated for multiple logins

I am automating a web application - we are using specflow for writing the specs and selenium/nunit for automation.
The issue I am having is this.
Each scenario needs to be run on multiple browsers (IE, Chrome, Firefox, safari etc) and quite a few tests(if not all) need to run for multiple logins. How do I design my specflow scenarios to satisfy both needs.
I thought about using tags for browsers and Scenario Outlin/Examples for multiple logins. Is this the only way. How do you address this need in your project?
My example scenario:
#IE10
#IE11
#Chrome
Scenario Outline: Check audit trail for XXXX function
Given I am on the <role> homepage
When I do some ation YYYY
Then The expected result is ZZZZ
Examples:
|role|
|doctor|
|nurse|
|pharmacist|
You can solve this issue with the SpecFlow+Runner.
Have a look at the answer here: https://stackoverflow.com/a/39344970/3155323
Full disclosure: I am one of the developers of SpecFlow+.
You can pass these as terminal argument like:
gradle clean test - Dapp.browser.type="chrome"

About testing techniques

I want some Testing information about my application.
Which testing method is suitable for my application page.
In my page contains 200 check boxes. on click on check box, one new page open with different URL.
*Note: all check boxes having different URL.
So, please anyone help me out to find which testing method is suitable.
and how can i test my this page with less effort.
For testing techniques, I think here are some expectation.
Techniques should focus on functional testing and reduce efforts of regression testing. From my experience, You should focus on Manual and Automation techniques.
Manual Efforts
If page have more than 200 check box, First question is it necessary to have 200 check box on one page, It will not good user experience. You can filed defect against requirement and product development team. Discussion begin
To verify look and feel of 200 checkbox and page, I will always focus on some QA notes that help everyone team to understand testing efforts that include browser specification and if page is responsive than which are different size of screen you are going to test
I will prefer to write Cucumber Scenario in Gherkin Language using Given/When and Then Cucumber Source
Scenario should be written in way that can help you to automate
Automation Efforts
I would recommand you to use selenium and choose any programming lanague(C#,PHP,Java,JavaScript,perl,Ruby,Python and other many) for automation.
You already have scenario and You can automate easily
Few things should be part of automation like deep verification that page is loaded successfully, title of page are matching and take snapshot if page is not loaded or during exception. Automation code should execute against any browser.
This is good starting point.
One possibility for you is to use RSpec and capybara-webkit, but i don't know if you are familiar with the Rubylanguage as you didn't talk about any programming language you'd like to use.
In order to achieve this workflow (click on a checkbox and check for the url), you should do something like this
describe "A test to", :js => true do
it "click on a checkbox and check for the url" do
visit("http://your_url") //to visit your page
page.check('the ID or the NAME of the checkbox') //to click on the checkbox
within_window(switch_to_window(windows.last)) do //to focus on the new opened page
expect(current_url).to eq('http://the_expected_url') //to check the url
end
end
end

TestNG reporting

I want to Email my TestNG report to other users as well but when they are clicking on the screenshot link. They are not able to see the screenshot as it is stored in my local machine.
String SaveandReturn = ScreenShot.takeScreenShot("SaveAndReturnverification");
Reporter.log("<a href=\"" + SaveandReturn + "\"><p align=\"left\"> Add Sub Service in LFA Services screenshot at " + new Date()+ "</p>");
}
How can I attach the screenshot so that other people can also view the report.
If you are able to implement the following it would be quite useful if you have a lot of people who need to see the results of the tests at a given time, its what I have implemented in my enviornment.
I have created a web server using XAMPP. I then created a web page which has links to all the TestNG reports.
Might not suit your environment but its a good solution if it does.
I would suggest you to go for Extentreports which has good UI when compared to TestNG. it is an extended version of testNG reporting framework.
It also suits your requirement of sending the screenshots by keeping at local if you have hosted in your local machine using MongoDB.

Selenium Test Case vs. Test Suite vs. general usage

How do I know what should be a test case and what a test suite in Selenium?
Is there any general rule for it? I've read the seleniumhq site any several others, but
they only have some basic examples while I want to test a whole website.
My questions are for example:
Say I'm testing some multi-step web form. Should I make it one test suite and each
step (in web form) would be a single test case or all steps should be one test case?
Say I've built a web forum and I want to test several features in it. Do I make one
test suite and each test case tests each feature (or several cases per each feature) OR
I'll have many test suites and each suite tests one feature with a few test cases.
What to do if I have a form which contains 5 checkboxes - each of them can be obviously clicked
or not. This may have some consequences when I submit the form. So - theoretically there are 2^5=32
possible execution flows. Should I test all 32? Or maybe should I just test each checkbox separately
to simplify things. When can/should I simplify, when not? (assuming that checkboxes MAY be
somehow related).
Should each feature have test cases which test both positive and negative results?
For example should I just focus on correct workflows - i.e. submit valid form and see if the
website did what I asked for (worked) OR also submit empty form and check if error message
appeared.
Can you answer these giving some practical examples (if needed)? - maybe using some (StackOverflow?)
site as example site.
Answer to 1 and 2:
I think this is more an issue about test design than selenium. Consider Selenium as a tool which controls the browser/website like a user would do. It simulates a user clicking through the page. To know what a test case is and what a test suite is you should think of the functionalities of your web application you want to test. Let's say you have web shop than one test case could test the following use case:
user puts articles in cart
user enters his data (name etc)
user gets a summary of his order
user confirms the order
It depends on your application which workflows or functionality you want to test.
I would consider a test suite for a whole project so one suite for one web application. And this application has a lot of test cases. Every test case is a use case.
When building a test suite, consider some design patterns like ui-mapping, page object design and consider the advantages of a test management system (like TestNG in Java).
here are some links to that:
http://www.shino.de/2011/07/26/on-the-pageobject-pattern/
http://www.theautomatedtester.co.uk/tutorials/selenium/page-object-pattern.htm
http://www.cheezyworld.com/2010/11/09/ui-tests-not-brittle/
http://hedleyproctor.com/2011/07/automating-selenium-testing-with-testng-ant-and-cruisecontrol/
Answer to 3 and 4:
It is similar to 1 and 2. It is always a question WHAT you want to test. Or a question what your project leader wants you to test (or customer). Every functionality which is important and should work should be tested.