How do I know what should be a test case and what a test suite in Selenium?
Is there any general rule for it? I've read the seleniumhq site any several others, but
they only have some basic examples while I want to test a whole website.
My questions are for example:
Say I'm testing some multi-step web form. Should I make it one test suite and each
step (in web form) would be a single test case or all steps should be one test case?
Say I've built a web forum and I want to test several features in it. Do I make one
test suite and each test case tests each feature (or several cases per each feature) OR
I'll have many test suites and each suite tests one feature with a few test cases.
What to do if I have a form which contains 5 checkboxes - each of them can be obviously clicked
or not. This may have some consequences when I submit the form. So - theoretically there are 2^5=32
possible execution flows. Should I test all 32? Or maybe should I just test each checkbox separately
to simplify things. When can/should I simplify, when not? (assuming that checkboxes MAY be
somehow related).
Should each feature have test cases which test both positive and negative results?
For example should I just focus on correct workflows - i.e. submit valid form and see if the
website did what I asked for (worked) OR also submit empty form and check if error message
appeared.
Can you answer these giving some practical examples (if needed)? - maybe using some (StackOverflow?)
site as example site.
Answer to 1 and 2:
I think this is more an issue about test design than selenium. Consider Selenium as a tool which controls the browser/website like a user would do. It simulates a user clicking through the page. To know what a test case is and what a test suite is you should think of the functionalities of your web application you want to test. Let's say you have web shop than one test case could test the following use case:
user puts articles in cart
user enters his data (name etc)
user gets a summary of his order
user confirms the order
It depends on your application which workflows or functionality you want to test.
I would consider a test suite for a whole project so one suite for one web application. And this application has a lot of test cases. Every test case is a use case.
When building a test suite, consider some design patterns like ui-mapping, page object design and consider the advantages of a test management system (like TestNG in Java).
here are some links to that:
http://www.shino.de/2011/07/26/on-the-pageobject-pattern/
http://www.theautomatedtester.co.uk/tutorials/selenium/page-object-pattern.htm
http://www.cheezyworld.com/2010/11/09/ui-tests-not-brittle/
http://hedleyproctor.com/2011/07/automating-selenium-testing-with-testng-ant-and-cruisecontrol/
Answer to 3 and 4:
It is similar to 1 and 2. It is always a question WHAT you want to test. Or a question what your project leader wants you to test (or customer). Every functionality which is important and should work should be tested.
Related
I add a new category in the admin panel and want to ensure that the category is available in the dropdown on the user's part of the website. Recorded test in the Selenium IDE works fine. But the thing is, the task that I execute is of course not a pure frontend thing - the category is saved in the database and is loaded from it to show it to the user. So if something goes wrong on the database-side, the test will fail.
My question is: is it bad practice to do such tests that depend on backend-behavior ? Should I go for Selenium Webdriver ?
If you use Selenium Webdriver, your test will not change in a main thing. It still will check database side. Selenium Webdriver is just anouther tool for testing that is more flexible and allows to make more complex test then in Selenium IDE.
I don't think that it is bad practice, because it is just one of the tests that chould be executed to enshure you that this part of your project works correctly. In this case I would check back-end part(get all categories from DB or admin's panel and check that there is no extra or missing ones) and than check user's panel(all categories are the same as set in DB and admin's panel).
I want some Testing information about my application.
Which testing method is suitable for my application page.
In my page contains 200 check boxes. on click on check box, one new page open with different URL.
*Note: all check boxes having different URL.
So, please anyone help me out to find which testing method is suitable.
and how can i test my this page with less effort.
For testing techniques, I think here are some expectation.
Techniques should focus on functional testing and reduce efforts of regression testing. From my experience, You should focus on Manual and Automation techniques.
Manual Efforts
If page have more than 200 check box, First question is it necessary to have 200 check box on one page, It will not good user experience. You can filed defect against requirement and product development team. Discussion begin
To verify look and feel of 200 checkbox and page, I will always focus on some QA notes that help everyone team to understand testing efforts that include browser specification and if page is responsive than which are different size of screen you are going to test
I will prefer to write Cucumber Scenario in Gherkin Language using Given/When and Then Cucumber Source
Scenario should be written in way that can help you to automate
Automation Efforts
I would recommand you to use selenium and choose any programming lanague(C#,PHP,Java,JavaScript,perl,Ruby,Python and other many) for automation.
You already have scenario and You can automate easily
Few things should be part of automation like deep verification that page is loaded successfully, title of page are matching and take snapshot if page is not loaded or during exception. Automation code should execute against any browser.
This is good starting point.
One possibility for you is to use RSpec and capybara-webkit, but i don't know if you are familiar with the Rubylanguage as you didn't talk about any programming language you'd like to use.
In order to achieve this workflow (click on a checkbox and check for the url), you should do something like this
describe "A test to", :js => true do
it "click on a checkbox and check for the url" do
visit("http://your_url") //to visit your page
page.check('the ID or the NAME of the checkbox') //to click on the checkbox
within_window(switch_to_window(windows.last)) do //to focus on the new opened page
expect(current_url).to eq('http://the_expected_url') //to check the url
end
end
end
Suppose I have a web page with several links on it. Also it has few buttons which execute some JavaScript.
So should I create one Java class for testing each of these links and elements or should I test all the links in just one test method and other elements in another one(so ending up with two Scripts).
Is there a another way of gouping these tests.
Thank you.
I have found that writing test cases based on actions is much more useful than writing based on pages.
Obviously, we would love to have everything automated. But realistically, this isn't possible. So we test what is most important...which happens to be: 1. The primary purposes of the product you are testing, and 2. The security of the product.
To make this easier to understand, lets say I have a Checkout page.
If I were to test based on a page, I would make sure every link on the page would work. I would verify that I can type in a new number in the quantity field, and make sure that the page verifies that the credit card number I type in is correct.
This method would somewhat test Security, but beyond that, it would test the UI. What would happen if I clicked on the Checkout button, and I was sent to the right page, but the item I was trying to checkout disappeared? That is a huge problem, but the test based on the page would work.
However, if I were to test based on actions (go to this page, add to cart, type in personal information, and then checkout), now I have made sure that the most important part of your program works, checked security, and even a little UI testing.
All in all, write your testing to do what the average user would do. No normal person is going to sit on the same page, testing out every little feature on that page.
It depends on whether you like to see 1/1 tests passed or 2/2 tests passed.
Just wondering how other people manage their test cases that are written for selenium test automation? I've been investigating maybe integrating with testlink to show the results and all, but I already have Jenkins set up for my environment and running my tests. What I'm really looking for is some way to nicely document my tests, like what steps each test performs for non-programmers.
I'm using Selenium with python, and Jenkins to run the tests.
I've tried two ways:
1) You can use Cucumber to write test steps like this:
#sanity #home_page #test_628
Scenario: Launch Support FAQs & Guides from Home cog
Given I navigate to Home page
When I click "Support" user logo link
Then I should see Support FAQs & Guides app launched
Each step written on Gherkin language (Given, When, Then) you can implement then using Selenium (I have to write on Ruby now, so I use Watir instead of Selenium).
see https://github.com/cucumber/cucumber/wiki/Python
2) But before that I used Robot Framework + Python + Selenium + Jenkins.
You can write test like this:
Go To Google Page [Documentation] Go to google page and search something
Open Browser to Google Page
Input Search selenium
Submit Search
This is done using human readable keywords. It prints very nice reports and can be easily integrated with Jenkins.
see http://www.wallix.org/2011/07/26/how-to-use-robotframework-with-the-selenium-library/
To document a test case and that too for non-programmers, Behaviour-Driven Development (BDD) is the best way. Some of the BDD tools are Cucumber, JBehave etc.
For example, lets assume the we have an website to manage some employees and we have a test case in which admin wants to add a new employee.
So the above test case can be documented using Cucumber as shown below:
Feature: Admin : Employees : Add Employees : Add Employee Details
Scenario: Verify that admin user is able to add new employee
Given Joe is an admin user
Given Joe is logged into the admin account
When Joe clicks "Add Emplyee" button
And Joe enters "Test Employee" in "Employee Name" text box
And Joe enters "12/05/1988" in "Birth Date" text box
And Joe selects "Male" from "Gender" radio button
And Joe clicks on "Submit" button
Then Joe gets message that says "Employee is added successfully."
The above test case can be easily understood by a non-programmer and he can perform each steps as performed by the test.
Hierarchy of folders for this test case will be feature/Admin/Employees/Add Employees/Add Employee Details
Hope this helps :)
I don't use TestLink. I think it is much easier to use a Jenkins job server to schedule my tests and it can do nearly the same job as testlink can and, im my opinion, is easier for other people to understand. I check my test code into Subversion and Jenkins checks out the latest code before it runs my tests. Jenkins has plugins to handle JUnit and TestNG report formats and has email capability, etc.
NOTE: The only Gotcha is that if you use RemoteWebDriver and a grid, the tests need to be ran by a Jenkins slave server that is running in a foreground process ( not in the background as a service). This is the case at least on Windows and might not be a problem on Linux.
We use Cucumber to create readable tests and have the results pulled back to Enterprise Tester, our Test Management platform. We can also pull Selenium tests and jUnit tests from Jenkins into Enterprise Tester, also for visibility across both manual and automated tests and traceability back to stories / requirements.
I'm currently writing some functional tests, and I started wondering what's the best philosophy between these two.
Situation
My application has some secured page, that need the user's group to have the right credentials to have access. These user are split into 2 groups : the 'collaborator group', and the 'accountable group'. Credentials are given to the groups.
Possible philosophies
Solution 1: Tests the credentials a.k.a. Test the features.
For each secured page, I test the
access with 2 users : one with the
right credential, and only this one,
and one without the right credential.
Pros: Tests only the fact that the page is secured against a specific credential
Cons: Doesn't test the "final" application behavior, as wanted (and user) by the client.
Solution 2: Test the groups a.k.a. test the requirements
For each secured page, I test the
access with a user of each group, and
check that only the allowed groups
gain access to the secured page.
Pros: Tests the "final" application behavior, as wanted (and user) by the client.
Cons:
Tests are tieds with the tests fixtures
Tests will have to change if the business rules changes or if more groups are created.
Thank you.
I think the second solution is the good one. The credentials will be tested as far as they are associated with a group.
Pros: Tests the "final" application behavior, as wanted (and user) by the client.
This is the most important part. Functional tests aims to test the final application in every possible cases. If you want to test the fact that your credentials have the same behavior with a user or a group, you'd better use unit tests.
Cons: Tests will have to change if the business rules changes or if more groups are created.
Your tests cases will always have to be updated if the business of your application changes. As you do with your unit tests. If your modify the code of a function, you check if your unit tests are still able to control each case. It's the same way with functional tests.
Maintaining your tests (and the fixtures they need to run) is a very tedious task, but it's the only way to ensure you're code is strong.
Hope it helped.
I would do both tests. The first one like you pointed out, does not need updating, but is testing the crucially important fact that users without entitlements do not have access. The second is the more comprehensive test and like #TimotheeMartin pointed out, tests will always need to be updated when the code changes.