Is it ok to have multiple groups of Given/When/Then in a single gherkin scenario - testing

I'm writing acceptance tests in Gherkin where I want to test for multiple changes in the UI of a web app based on an initial action. Here's an example:
Scenario: Cancel editing a new text asset
Given the user "test_user#fakedomain.com" is logged in
When the user navigates to "/build/"
And the user clicks the "Sandbox" link
And the user inputs "Test story for canceling editing of a new text asset" for the "title" field
And the user inputs "Test User" for the "byline" field
And the user inputs "My summary, so exciting!" for the "summary" textarea
And the user clicks on "Untitled Section" in the section list
And the user clicks the "Text" icon in the "center" container
And the user inputs the following text in the rich text editor:
"""
Test text for asset. This is cool.
"""
And the user clicks the "cancel" button
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
And the "Image" icon is present
And the "Text" icon is present
When the user refreshes the browser
And the user clicks on "Untitled Section" in the section list
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
When the user opens the asset drawer
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
Note that there are multiple groups of When/Then steps, to test for responses of the initial action. While most implementations of steps ignore the prefix keyword, and I'm pretty sure that I can get this test to run, is there a better way to test the different outcomes? Is it better practice to write multiple scenarios with the same setup but different "Then" statements?

Remember that you should test only ONE behaviour/feature at a time.
The rule of thumb is that you should use only one When step:
Given some state before action
And some other state before action
...
When only one action
Then assert correct output
And assert correct output
...
You see - only one line of When, without any Ands under When. If you use many When steps instead, you create test script, not a specification. Your tests will be hard to understand and you will notice that you add more and more steps when the underlying implementation changes.
You also need to keep underlying logic hidden, because you don't want to change it every time you change something irrelevant. Example:
And the user inputs "My summary, so exciting!" for the "summary"
textarea
What if you change the summary field from a textarea to an input type? You have to change the scenario (maintenance nightmare) or left you scenario lying (worse than not having scenario). You should write instead:
When the user describes it as "so exciting!"
But still, the structure of the whole scenario is bad. Ask yourself the question: what I want to check? If I were a person that wants to understand the business logic of the feature I would like to see something like:
Scenario: Cancel editing a new text asset
Given I edit the Sandbox details with some data
When I cancel editing
Then Sandox details should be empty
That's it!
How to achieve it? Move all irrelevant logic deeper, use the PageObject pattern etc. And read about Specification By Example

In contrast to the previous answer, there is no strict rule on how one can use define the steps. Cucumber official documentation at https://cucumber.io/docs/gherkin/reference/ recommends to use only one When due to the fact that only behavior is listed in one acceptance criteria. This is just a recommendation, but not a rule. And Cucumber documentation is mostly focused on the Acceptance criteria specification, but not the test automation.
So, it is definitely possible to follow how it best suits your requirement when it comes to automation testing.
I'd recommend using fewer "And" keywords by merging different steps. Below points I recommend (It is a recommendation, but not a rule :) ):
Use only one Given, When and Then keywords in one flow of a scenario, use "And" keyword if you need to specify extra steps at the appropriate event
If you come across using too many "And" keywords, try to merge multiple such steps
In reality, we cannot just use only one When in the test automation, because:
Number of tests in the automation increases
Total execution time for the automation goes up
We may be doing a lot of redundant actions across multiple scenarios if we use only one When. Consider that you need to perform 5 steps as the initial condition for testing 10 different actions. Here, you will perform these 5 steps for 10 times when you use only one "When" - This consumes too much time, and causes the tests to be unstable, and causes more load on your application
Due to the increased number of tests, we need to spend more time to analyze the results, more time to maintain
I also recommend testing the behavior based on the requirement. If your requirement is about verifying something in the "test area" rather than the "input area", your step should indicate that. Remember that if the requirement is changed, development is code changed, and hence the test automation code is also changed.

Related

Is it possible to add an "Else" after "Given, When, Then"?

I'm new to Gherking and trying to write my first scenarios as best as I can, but I regularly find myself in situations where I'm really tempted to add an "Else" to my scenario. "Given, When, Then" becomes "Given, When, Then, Else". I know that the "Else" keyword is not defined and so not implemented in Gherkin tools but it doesn't care to me because I don't use these tools.
Do you think it is correct to write this :
Example :
Scenario : Application starts
Given I start the application
When I already have an open session
Then I see the home screen
Else I see the login screen
Or is it better to write two different scenarios :
Scenario : Application started by authenticated user
Given I have an open session
When I start the application
Then I see the home screen
Scenario : Application started by unauthenticated user
Given I don't have an open session
When I start the application
Then I see the login screen
In short no, but here are options to handle multiple variants of a scenario:
If it was only tailing elements of the scenario steps that differed you could of moved early steps in to a common 'Background' section, making repeated differing scenarios shorter and clearer.
But from your example it is all steps differing slightly so you can:-
accept the repitition of multiple scenarios
Or
parametise the differences and use data tables in the 'given' and 'then' sections to give before and after values.
Or (my prefernece)
Use the 'scenario outline' syntax that uses an examples table to provide sets of data fixtures with their expected results. These replace in the scenario steps as runtime. The scenario is then 'played out' once for each row in the examples table.
So:
Scenario : Application started by authenticated user
Given I have an open session
When I start the application
Then I see the home screen
Scenario : Application started by unauthenticated user
Given I don't have an open session
When I start the application
Then I see the login screen
Becomes:
Scenario Outline: Application Start and login
Given Application started by <AuthenticationStatus> user
And I have <SessionState> session
When I start the application
Then I see the <FirstScreen> screen
Examples:
|AuthenticationStatus |SessionState |FirstScreen|
|Authenticated |open |home |
|Un-Authenticated |not open |login |
IMHO for 2 scenarios it might not be worth the loss of readabiltiy but for more than that I think it's definitely worth it.

Behat using BeforeScenario vs Given

For something that REALLY needs to be set up before every scenario then BeforeScenario is used.
Sometimes there are things that only need to be set up for some scenarios, but for a significant proportion of them. For example, if the scenario needs a "regular user account" to exist, then it goes on to login as that user and do some stuff.
You can make an #BeforeScenario #RegularUser method that will run for each Scenario tagged as #RegularUser. So scenarios end up looking like:
#RegularUser
Scenario login as a regular user
Given I am on the login page
When I login to a regular user account
Then the welcome screen is displayed
The alternative is to
Scenario login as a regular user
Given a regular user exists
And I am on the login page
When I login to a regular user account
Then the welcome screen is displayed
"a regular user exists" would be associated with the method that creates the regular user.
With the first approach, I can make #AfterScenario #RegularUser that will delete the user after the scenario ends. So that seems "a good thing".
But the 2nd approach kind of looks nicer to me from a BDD viewpoint. But its limitation is that then there is nothing happening at the end of the scenario to delete the user.
Which approach is the better practice?
In my opinion, i would go with first option.
The problem is that, both options are not so good, because normally you have context of this scenario added:
In order to ..
As a Regular User
I need to ..
That should show what was you intention of running this scenario. The process of creating RegularUser don't adding value to scenario itself.
Then, if you need to confirmation in scenario of having regular user - then is OK. But in here in my opinion, it is not the point - then second option is not so good - we have context to add informations like that.
That is why, I think first option (not ideal, but better that second) is good solution.
I don't know in Behat functionality to set up user by context, but tag option should be more elastic.

How to catagorize or group selenium webdriver tests

Suppose I have a web page with several links on it. Also it has few buttons which execute some JavaScript.
So should I create one Java class for testing each of these links and elements or should I test all the links in just one test method and other elements in another one(so ending up with two Scripts).
Is there a another way of gouping these tests.
Thank you.
I have found that writing test cases based on actions is much more useful than writing based on pages.
Obviously, we would love to have everything automated. But realistically, this isn't possible. So we test what is most important...which happens to be: 1. The primary purposes of the product you are testing, and 2. The security of the product.
To make this easier to understand, lets say I have a Checkout page.
If I were to test based on a page, I would make sure every link on the page would work. I would verify that I can type in a new number in the quantity field, and make sure that the page verifies that the credit card number I type in is correct.
This method would somewhat test Security, but beyond that, it would test the UI. What would happen if I clicked on the Checkout button, and I was sent to the right page, but the item I was trying to checkout disappeared? That is a huge problem, but the test based on the page would work.
However, if I were to test based on actions (go to this page, add to cart, type in personal information, and then checkout), now I have made sure that the most important part of your program works, checked security, and even a little UI testing.
All in all, write your testing to do what the average user would do. No normal person is going to sit on the same page, testing out every little feature on that page.
It depends on whether you like to see 1/1 tests passed or 2/2 tests passed.

Richtext in TFS2010 Testcase Teststeps' texts?

I am currently evaluating possible solutions to limitations in Microsoft's Test Manager 2010 TestStep Editor control and I was wondering whether anyone knows if and if so, how you can write richtext to the Action and Expected Result fields of a step? As far as I saw it takes a 'ParameterizedString' as input, but I am not sure what parameters MSDN refers to (or the MTM TestRunner can handle) when saying "[...]Represents a string that has embedded parameters.[...]".
The Expected Results string is a list of string namees delimited by spaces, semi-colons or commas. You can enter something like: #Test, #Cart, #Book or #Test #Cart #Book.
You can't add HTML to the Expected Result parameters or the action; however, I am not sure I would describe this as a limitation. What action were you thinking you would need to be able to do this. Actions are typically "Click this" or "Go to here" Did you just want a stronger visual queue for certain parts of the action?
Here is an article that describes the intent with the expected results section:
The real power of the test case though, comes in the bottom portion of
the screen, on the Steps tab. This is where you list out the manual
steps the tester will use to test the application. Start typing where
it says "Click here to add a step." In the Action column, you add the
action that the tester should try and perform, such as "Open Internet
Explorer." In the Expected Results column, you list what should happen
when the action is performed, such as "Internet Explorer opens to its
default home page." To continue adding new steps to the test case, hit
Enter to go to a new line.
Not every action step requires an expected result. When an expected
result is specified, it turns that step into a verification step. With
a verification step, the tester will have to specify whether that
particular step was successful or not.
You have the ability to provide parameters to your test steps. This
allows you to run the test multiple times, each time using a different
value. An example of this would be testing different login values on a
Web site. Instead of creating a new test case for each login, you can
create one test case, and pass multiple login values into it.

How to Verify whether a Robot is Entering Information

I have a web form which the users fill and the info send to server and stored on a database. I am worried that Robots might just fill in the form and I will end up with a database full of useless records. How can I prevent Robots from filling in my forms? I am thinking maybe something like Stackoverflow's robot detection, where if it thinks you are a robot, it asks you to verify that you are not. Is there a server-side API in Perl, Java or PHP?
There are several solutions.
Use a CAPTCHA. SO uses reCAPTCHA as far as I know.
Add an extra field to your form and hide it with CSS (display:none). A normal user would not see this field and therefore will not fill it. You check at the submission if this field is empty. If not, then you are dealing with a robot that has carefully filled out all form fields. This technique is usually referred to as a "honeypot".
Add a JavaScript timer function. At the page load it starts a value at zero and then increases it as time passes. A normal user would read and fill out your form for some time and only then submit it. A robot would just fill out and submit the form immediately upon receiving it. You check if the value has gone much from zero at the submission. If it has, then it is likely a real user. If you see just a couple of seconds (or even no value at all due to the robots not executing JavaScript) then it is likely a robot. This will however only work if you decide you will require your users have JavaScript on in order to perform "write" operations.
There are other techniques for sure. But these are quite simple and effective.
You can use reCAPTCHA (same as stackoverflow) - they have libraries for a number of programming languages.
I've always preferred Honeypot captcha (article by phil haack), as its less invasive to the user.
Captchas bring accessibility problems and will be ultimately defeated by software recognition.
I recommand the reading of this short article about bot traps, which include hidden fields, as Matthew Vines and New in town already suggested.
Anyway, you are still free to use both captcha and bot traps.
CAPTCHA is great. The other thing you can do that will prevent 99% of your robot traffic yet not annoy your users is to validate fields.
My site, I check for text in fields like zip code and phone number. That has removed all of the non-targeted robot misinformation.
You could create a two-step system in which a user fills the form, but then must reply to an e-mail to "activate" the record within a set period of time - say 24 hours.
In the back end, instead of populating your current table with all the form submissions, you could put them into a temporary table that automatically deletes any row that is older than your time allotment. Unless you have a serious bot problem, then I would think that the table wouldn't get that big, especially if the first form is just a few fields.
A benifit of this approach is that you don't have to use captcha or some other technology like that that might create some accessibility problems.