I'm starting with Selenium and FitNesse. My goal is to create an acceptance tests suite which can be run easily in different browsers.
At first I wrote a very short wrapper around Seleniums WebDriver. The wrapper functions are used in Scenario Tables, Script Tables and Query Tables. So far, everything is fine.
My idea was to create a wiki page hierarchy called ParameterizedSuites, which has sub-suites and tests in any possible depth. Now I wanted to create one suite called FirefoxSuite defining an variable ${BROWSER} with value "firefox" and include the ParameterizedSuites only. In ParameterizedSuites.SuiteSetUp a WebDriver instance will be instantiated using ${BROWSER}.
But my idea does not work. It seems that I must include every SetUp, TearDown, Suite and Test page seperately? Is my idea completely wrong or am I missing something?
Does anybody something similar?
Hope my description is clear enough to understand the scenario.
Kind regards,
Andi
I think what you want is SymbolicLinks. You can use Symbolic Links to include a subtree of the wiki into a different place in the wiki.
For more information on SymbolicLinks, check your local users guide or the copy online: http://www.fitnesse.org/FitNesse.UserGuide.FitNesseWiki.SymbolicLinks
A common way to do this is to actually create your ParameterizedSuite and then symbolically link it into the Pages that define the configuration and supply specialized SetUp and TearDown pages. This is sort of the oposite of what you are describing, but I think what you are doing could work as well, provided you use SymbolicLinks to make the pages part of the wiki structure of your parameterized tests.
FrontPage
SuiteOne
TestOne
ConfigurationOne
SetUp
TearDown
SuiteOneSymolicLink
ConfigurationTwo
SetUp
TearDown
SuiteOneSymolicLink
In the structure above, SuiteOne can be run via the symbolic link in ConfigurationOne or ConfigurationTwo. Variables defined in the ConfigurationOne or ConfigurationTwo pages will inherit to SuiteOne when is is run via the SuiteOneSymbolicLink in each configuration hierarchy. Additionally, the SetUp and TearDown from the Confiugration Pages will be inherited as well.
Related
NUnit (and the like) has method attributes which allow tests to be run multiple times with different arrange values. Is something similar possible with SpecFlow?
What I am aiming for is a way to run the same scenario tests in a feature file with as many browser drivers as I can, in one test run.
You can use scenario outlines. In example of scenario outline you can mention driver name and you code logic should take action according to driver. Please see more details about scenario ouyline below
https://github.com/cucumber/cucumber/wiki/Scenario-outlines
Examples are one solution, but in your case a little cumbersome, as you have to specify them at every scenario.
In your case, please have a look at the targets feature of the SpecFlow+Runner. With that you can "multiply" your scenarios for different configurations. If you put the web driver that should be used in this configuration, you can test as many webdriver as you want.
Have a look at this example: https://github.com/techtalk/SpecFlow.Plus.Examples/tree/master/SeleniumWebTest
Full Disclosure: I am one of the developers of SpecFlow & SpecFlow+
Use scenario outlines and this tool if you want to use browsers as tags:
https://github.com/unickq/Unickq.SeleniumHelper
What is the correct (Most effective) Cucumber Project Layout when Considerring Page Object Modelling?
After allot of research I have come up with the following design:
Maven Project
NEW PROJECT SETUP:
I agree with the main idea you present, however the page object model also refers to the utilites. One of the desired goals of the page object model is to keep the selenium code out of the test itself, so most of those references will go to the pages, then its locators and actions would access the driver class, preferably through the utilities. That does not mean that the test program cannot make direct references to the utilities, but it should do so for non-selenium reasons only. There are exceptions, of course. In the case of Cucumber or any other BDD-based framework, you would only refer to what you are now calling "main" as "steps" and each test would have its own story file, accessing the one, or more, steps files. The rest remains the same. The idea behind that is it allows you to create and maintain a library of related steps that existing and future story files can refer to.
Hope this has helped you and/or others to better understand the flow. Also, disclaimer - most of this is my opinion - there are likely many ways to diagram the relationships, but what I described is what I use.
Upon further examination, I see that I missed the lower half (I am visually impaired). The testrunner is typically at the very top of the chain in this environment. It runs as a single JUnit or TestNG test to run ALL your stories.
And now my browser is messed-up and I cannot re-scroll back up to confirm that diagram again to comment more.
I've drawn a crude layout of what I was trying to describe. I hope it explains my answer to your question more clearly.
Here's the basic project tree:
Here's src/test/java expanded:
And finally, src/test/resources expanded
The only thing in \src\main\resources is some extra stuff that JBehave uses to allow some customization of their reports, called FTL.
I have 2 environments - test and production. The only thing that is different is the base URL. All other aspects should be identical - objects, pages, etc.
What I am looking to do is to create the option in Selenium to tell it which environment I am looking to test when I click start. I don't have an issue doing the coding to get it to know which base URL to use once the testing starts, but I am not sure if I can or how I can, make Selenium ask me which environment to use. For example, I would click start and a pop-up would appear with a drop down showing test or production.
Is something like that possible? I hate to create two entirely separate tests and test suites just because of the base URL.
Thanks,
Jamie
Can any one please tell me is there any kind of tools or eclipse base plugins available for generate relevant test cases for SalesForce platform related Apex classes. It seems with code coverage they are not expecting out come like we expect with JUnit, they want to cover whether, test cases are going through the flows of the source classes (like code go through).
Please don't get this post in wrong, I don't want anyone is going to write test cases for my codes :). I have post this question due to nature of SalesForce expecting that code coverage should be. Thanks.
Although Salesforce requires a certain percentage of code coverage for your test cases, you really need to be writing cases that check the results to ensure that the code behaves as designed.
So, even if there was a tool that could generate code to get 100% coverage of your test class, it wouldn't be able to test the results of those method calls, leaving you with a false sense of having "tested code".
I've found that breaking up long methods into separate, sometimes static, methods makes it easier to do unit testing. You can test each individual method, and not worry so much about tweaking parameters to a single method so that it covers all execution paths.
it's now possible to generate test classes automatically for your class/trigger/batch. You can install "Test Class Generator" app from AppExchange and see it working.
This would really help you generating test class and saves lot of your development time.
We use TestNG and Selenium WebDriver to test our web application.
Now our problem is that we often have several tests that need to run in a certain order, e.g.:
login to application
enter some data
edit the data
check that it's displayed correctly
Now obviously these tests need to run in that precise order.
At the same time, we have many other tests which are totally independent from the list of tests above.
So we'd like to be able to somehow put tests into "groups" (not necessarily groups in the TestNG sense), and then run them such that:
tests inside one "group" always run together and in the same order
but different test "groups" as a whole can run in any order
The second point is important, because we want to avoid dependencies between tests in different groups (so different test "groups" can be used and developed independently).
Is there a way to achieve this using TestNG?
Solutions we tried
At first we just put tests that belong together into one class, and used dependsOnMethods to make them run in the right order. This used to work in TestNG V5, but in V6 TestNG will sometimes interleave tests from different classes (while respecting the ordering imposed by dependsOnMethods). There does not seem to be a way to tell TestNG "Always run tests from one class together".
We considered writing a method interceptor. However, this has the disadvantage that running tests from inside an IDE becomes more difficult (because directly invoking a test on a class would not use the interceptor). Also, tests using dependsOnMethods cannot be ordered by the interceptor, so we'd have to stop using that. We'd probably have to create our own annotation to specify ordering, and we'd like to use standard TestNG features as far as possible.
The TestNG docs propose using preserve-order to order tests. That looks promising, but only works if you list every test method separately, which seems redundant and hard to maintain.
Is there a better way to achieve this?
I am also open for any other suggestions on how to handle tests that build on each other, without having to impose a total order on all tests.
PS
alanning's answer points out that we could simply keep all tests independent by doing the necessary setup inside each test. That is in principle a good idea (and some tests do this), however sometimes we need to test a complete workflow, with each step depending on all previous steps (as in my example). To do that with "independent" tests would mean running the same multi-step setup over and over, and that would make our already slow tests even slower. Instead of three tests doing:
Test 1: login to application
Test 2: enter some data
Test 3: edit the data
we would get
Test 1: login to application
Test 2: login to application, enter some data
Test 3: login to application, enter some data, edit the data
etc.
In addition to needlessly increasing testing time, this also feels unnatural - it should be possible to model a workflow as a series of tests.
If there's no other way, this is probably how we'll do it, but we are looking for a better solution, without repeating the same setup calls.
You are mixing "functionality" and "test". Separating them will solve your problem.
For example, create a helper class/method that executes the steps to log in, then call that class/method in your Login test and all other tests that require the user to be logged in.
Your other tests do not actually need to rely on your Login "Test", just the login class/method.
If later back-end modifications introduce a bug in the login process, all of the tests which rely on the Login helper class/method will still fail as expected.
Update:
Turns out this already has a name, the Page Object pattern. Here is a page with Java examples of using this pattern:
http://code.google.com/p/selenium/wiki/PageObjects
Try with depends on group along with depends on method. Add all methods in same class in one group.
For example
#Test(groups={"cls1","other"})
public void cls1test1(){
}
#Test(groups={"cls1","other"}, dependsOnMethods="cls1test1", alwaysrun=true)
public void cls1test2(){
}
In class 2
#Test(groups={"cls2","other"}, dependsOnGroups="cls1", alwaysrun=true)
public void cls2test1(){
}
#Test(groups={"cls2","other"}, dependsOnMethods="cls2test1", dependsOnGroups="cls1", alwaysrun=true)
public void cls2test2(){
}
There is an easy (whilst hacky) workaround for this if you are comfortable with your first approach:
At first we just put tests that belong together into one class, and used dependsOnMethods to make them run in the right order. This used to work in TestNG V5, but in V6 TestNG will sometimes interleave tests from different classes (while respecting the ordering imposed by dependsOnMethods). There does not seem to be a way to tell TestNG "Always run tests from one class together".
We had a similar problem: we need our tests to be run class-wise because we couldn't guarantee the test classes not interfering with each other.
This is what we did:
Put a
#Test( dependsOnGroups= { "dummyGroupToMakeTestNGTreatThisAsDependentClass" } )
Annotation on an Abstract Test Class or Interface that all your Tests inherit from.
This will put all your methods in the "first group" (group as described in this paragraph, not TestNG-groups). Inside the groups the ordering is class-wise.
Thanks to Cedric Beust, he provided a very quick answer for this.
Edit:
The group dummyGroupToMakeTestNGTreatThisAsDependentClass actually has to exist, but you can just add a dummy test case for that purpose..