I have the following scenario:
Scenario Outline: Searching for something on Google
Given user goes to google.com
When user search for "<something>"
Then <somethingHappens>
I would like to prevent Then <somethingHappens> from being executed if a condition in When user search for "<something>" is not met.
Let say that google does not find something, how do I stop the execution of the test?
I found some post saying that this is not the way Cucumber works. Instead, I'd rather create another scenario that would match what I'm looking for. Is that correct?
Thanks in advance for your help
There are few ways you gan go:
The correct way is to have two scenarios. Because the behavior (aka logic) is not defined for the case that notheing has been found. I would rework your scenario to:
Scenario Outline: Searching for something on Google
Given user goes to google.com
When user search for "<something>"
And there are results returned
Then <somethingHappens>
and
Scenario Outline: Searching for something on Google
Given user goes to google.com
When user search for "<something>"
And there are no results returned
Then <nothingHappens>
Hence your logic would look more like a test..
You can use the capabilities of unit-test framework that is currenty driving your Cucumber test. For example if you are using JUnit you coud have the step definition like this:
#When("user search for something")
public void when(){
boolean condition = testForConditionMet();
Assume.assumeTrue(condition)
}
The above code would make your test skip if condition is false.
If you would like to fail the test then use Assert.assertTrue(condition) instead of Assume.
I would not recommend to use the second approach since you are stuck to unit-test framework in that case.
Related
I'm new to Gherking and trying to write my first scenarios as best as I can, but I regularly find myself in situations where I'm really tempted to add an "Else" to my scenario. "Given, When, Then" becomes "Given, When, Then, Else". I know that the "Else" keyword is not defined and so not implemented in Gherkin tools but it doesn't care to me because I don't use these tools.
Do you think it is correct to write this :
Example :
Scenario : Application starts
Given I start the application
When I already have an open session
Then I see the home screen
Else I see the login screen
Or is it better to write two different scenarios :
Scenario : Application started by authenticated user
Given I have an open session
When I start the application
Then I see the home screen
Scenario : Application started by unauthenticated user
Given I don't have an open session
When I start the application
Then I see the login screen
In short no, but here are options to handle multiple variants of a scenario:
If it was only tailing elements of the scenario steps that differed you could of moved early steps in to a common 'Background' section, making repeated differing scenarios shorter and clearer.
But from your example it is all steps differing slightly so you can:-
accept the repitition of multiple scenarios
Or
parametise the differences and use data tables in the 'given' and 'then' sections to give before and after values.
Or (my prefernece)
Use the 'scenario outline' syntax that uses an examples table to provide sets of data fixtures with their expected results. These replace in the scenario steps as runtime. The scenario is then 'played out' once for each row in the examples table.
So:
Scenario : Application started by authenticated user
Given I have an open session
When I start the application
Then I see the home screen
Scenario : Application started by unauthenticated user
Given I don't have an open session
When I start the application
Then I see the login screen
Becomes:
Scenario Outline: Application Start and login
Given Application started by <AuthenticationStatus> user
And I have <SessionState> session
When I start the application
Then I see the <FirstScreen> screen
Examples:
|AuthenticationStatus |SessionState |FirstScreen|
|Authenticated |open |home |
|Un-Authenticated |not open |login |
IMHO for 2 scenarios it might not be worth the loss of readabiltiy but for more than that I think it's definitely worth it.
For something that REALLY needs to be set up before every scenario then BeforeScenario is used.
Sometimes there are things that only need to be set up for some scenarios, but for a significant proportion of them. For example, if the scenario needs a "regular user account" to exist, then it goes on to login as that user and do some stuff.
You can make an #BeforeScenario #RegularUser method that will run for each Scenario tagged as #RegularUser. So scenarios end up looking like:
#RegularUser
Scenario login as a regular user
Given I am on the login page
When I login to a regular user account
Then the welcome screen is displayed
The alternative is to
Scenario login as a regular user
Given a regular user exists
And I am on the login page
When I login to a regular user account
Then the welcome screen is displayed
"a regular user exists" would be associated with the method that creates the regular user.
With the first approach, I can make #AfterScenario #RegularUser that will delete the user after the scenario ends. So that seems "a good thing".
But the 2nd approach kind of looks nicer to me from a BDD viewpoint. But its limitation is that then there is nothing happening at the end of the scenario to delete the user.
Which approach is the better practice?
In my opinion, i would go with first option.
The problem is that, both options are not so good, because normally you have context of this scenario added:
In order to ..
As a Regular User
I need to ..
That should show what was you intention of running this scenario. The process of creating RegularUser don't adding value to scenario itself.
Then, if you need to confirmation in scenario of having regular user - then is OK. But in here in my opinion, it is not the point - then second option is not so good - we have context to add informations like that.
That is why, I think first option (not ideal, but better that second) is good solution.
I don't know in Behat functionality to set up user by context, but tag option should be more elastic.
I'm writing acceptance tests in Gherkin where I want to test for multiple changes in the UI of a web app based on an initial action. Here's an example:
Scenario: Cancel editing a new text asset
Given the user "test_user#fakedomain.com" is logged in
When the user navigates to "/build/"
And the user clicks the "Sandbox" link
And the user inputs "Test story for canceling editing of a new text asset" for the "title" field
And the user inputs "Test User" for the "byline" field
And the user inputs "My summary, so exciting!" for the "summary" textarea
And the user clicks on "Untitled Section" in the section list
And the user clicks the "Text" icon in the "center" container
And the user inputs the following text in the rich text editor:
"""
Test text for asset. This is cool.
"""
And the user clicks the "cancel" button
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
And the "Image" icon is present
And the "Text" icon is present
When the user refreshes the browser
And the user clicks on "Untitled Section" in the section list
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
When the user opens the asset drawer
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
Note that there are multiple groups of When/Then steps, to test for responses of the initial action. While most implementations of steps ignore the prefix keyword, and I'm pretty sure that I can get this test to run, is there a better way to test the different outcomes? Is it better practice to write multiple scenarios with the same setup but different "Then" statements?
Remember that you should test only ONE behaviour/feature at a time.
The rule of thumb is that you should use only one When step:
Given some state before action
And some other state before action
...
When only one action
Then assert correct output
And assert correct output
...
You see - only one line of When, without any Ands under When. If you use many When steps instead, you create test script, not a specification. Your tests will be hard to understand and you will notice that you add more and more steps when the underlying implementation changes.
You also need to keep underlying logic hidden, because you don't want to change it every time you change something irrelevant. Example:
And the user inputs "My summary, so exciting!" for the "summary"
textarea
What if you change the summary field from a textarea to an input type? You have to change the scenario (maintenance nightmare) or left you scenario lying (worse than not having scenario). You should write instead:
When the user describes it as "so exciting!"
But still, the structure of the whole scenario is bad. Ask yourself the question: what I want to check? If I were a person that wants to understand the business logic of the feature I would like to see something like:
Scenario: Cancel editing a new text asset
Given I edit the Sandbox details with some data
When I cancel editing
Then Sandox details should be empty
That's it!
How to achieve it? Move all irrelevant logic deeper, use the PageObject pattern etc. And read about Specification By Example
In contrast to the previous answer, there is no strict rule on how one can use define the steps. Cucumber official documentation at https://cucumber.io/docs/gherkin/reference/ recommends to use only one When due to the fact that only behavior is listed in one acceptance criteria. This is just a recommendation, but not a rule. And Cucumber documentation is mostly focused on the Acceptance criteria specification, but not the test automation.
So, it is definitely possible to follow how it best suits your requirement when it comes to automation testing.
I'd recommend using fewer "And" keywords by merging different steps. Below points I recommend (It is a recommendation, but not a rule :) ):
Use only one Given, When and Then keywords in one flow of a scenario, use "And" keyword if you need to specify extra steps at the appropriate event
If you come across using too many "And" keywords, try to merge multiple such steps
In reality, we cannot just use only one When in the test automation, because:
Number of tests in the automation increases
Total execution time for the automation goes up
We may be doing a lot of redundant actions across multiple scenarios if we use only one When. Consider that you need to perform 5 steps as the initial condition for testing 10 different actions. Here, you will perform these 5 steps for 10 times when you use only one "When" - This consumes too much time, and causes the tests to be unstable, and causes more load on your application
Due to the increased number of tests, we need to spend more time to analyze the results, more time to maintain
I also recommend testing the behavior based on the requirement. If your requirement is about verifying something in the "test area" rather than the "input area", your step should indicate that. Remember that if the requirement is changed, development is code changed, and hence the test automation code is also changed.
How do I know what should be a test case and what a test suite in Selenium?
Is there any general rule for it? I've read the seleniumhq site any several others, but
they only have some basic examples while I want to test a whole website.
My questions are for example:
Say I'm testing some multi-step web form. Should I make it one test suite and each
step (in web form) would be a single test case or all steps should be one test case?
Say I've built a web forum and I want to test several features in it. Do I make one
test suite and each test case tests each feature (or several cases per each feature) OR
I'll have many test suites and each suite tests one feature with a few test cases.
What to do if I have a form which contains 5 checkboxes - each of them can be obviously clicked
or not. This may have some consequences when I submit the form. So - theoretically there are 2^5=32
possible execution flows. Should I test all 32? Or maybe should I just test each checkbox separately
to simplify things. When can/should I simplify, when not? (assuming that checkboxes MAY be
somehow related).
Should each feature have test cases which test both positive and negative results?
For example should I just focus on correct workflows - i.e. submit valid form and see if the
website did what I asked for (worked) OR also submit empty form and check if error message
appeared.
Can you answer these giving some practical examples (if needed)? - maybe using some (StackOverflow?)
site as example site.
Answer to 1 and 2:
I think this is more an issue about test design than selenium. Consider Selenium as a tool which controls the browser/website like a user would do. It simulates a user clicking through the page. To know what a test case is and what a test suite is you should think of the functionalities of your web application you want to test. Let's say you have web shop than one test case could test the following use case:
user puts articles in cart
user enters his data (name etc)
user gets a summary of his order
user confirms the order
It depends on your application which workflows or functionality you want to test.
I would consider a test suite for a whole project so one suite for one web application. And this application has a lot of test cases. Every test case is a use case.
When building a test suite, consider some design patterns like ui-mapping, page object design and consider the advantages of a test management system (like TestNG in Java).
here are some links to that:
http://www.shino.de/2011/07/26/on-the-pageobject-pattern/
http://www.theautomatedtester.co.uk/tutorials/selenium/page-object-pattern.htm
http://www.cheezyworld.com/2010/11/09/ui-tests-not-brittle/
http://hedleyproctor.com/2011/07/automating-selenium-testing-with-testng-ant-and-cruisecontrol/
Answer to 3 and 4:
It is similar to 1 and 2. It is always a question WHAT you want to test. Or a question what your project leader wants you to test (or customer). Every functionality which is important and should work should be tested.
I am currently evaluating possible solutions to limitations in Microsoft's Test Manager 2010 TestStep Editor control and I was wondering whether anyone knows if and if so, how you can write richtext to the Action and Expected Result fields of a step? As far as I saw it takes a 'ParameterizedString' as input, but I am not sure what parameters MSDN refers to (or the MTM TestRunner can handle) when saying "[...]Represents a string that has embedded parameters.[...]".
The Expected Results string is a list of string namees delimited by spaces, semi-colons or commas. You can enter something like: #Test, #Cart, #Book or #Test #Cart #Book.
You can't add HTML to the Expected Result parameters or the action; however, I am not sure I would describe this as a limitation. What action were you thinking you would need to be able to do this. Actions are typically "Click this" or "Go to here" Did you just want a stronger visual queue for certain parts of the action?
Here is an article that describes the intent with the expected results section:
The real power of the test case though, comes in the bottom portion of
the screen, on the Steps tab. This is where you list out the manual
steps the tester will use to test the application. Start typing where
it says "Click here to add a step." In the Action column, you add the
action that the tester should try and perform, such as "Open Internet
Explorer." In the Expected Results column, you list what should happen
when the action is performed, such as "Internet Explorer opens to its
default home page." To continue adding new steps to the test case, hit
Enter to go to a new line.
Not every action step requires an expected result. When an expected
result is specified, it turns that step into a verification step. With
a verification step, the tester will have to specify whether that
particular step was successful or not.
You have the ability to provide parameters to your test steps. This
allows you to run the test multiple times, each time using a different
value. An example of this would be testing different login values on a
Web site. Instead of creating a new test case for each login, you can
create one test case, and pass multiple login values into it.