This is my first more "complex" SpecFlow scenario written in Gherkin. Please provide me any feedback about it's quality:
Scenario: CompanyUpload
Given admin (user) log in to admin-site
When user enters Uploading page
And user downloads Company Upload template
And template is updated by script
And user opens Company upload form
And user fills it with proper data
And user submit filled form
Then new job should be added to the queue (visible in table form)
And job status should be "In Queue" for not longer than 10 minutes
And job status should be updated to "Processing" and shouldn't take more than 4 minutes
And final job status should be "Succeeded"
This scenario is already coded with Selenium Webdriver actions. Now, I would like to cover it with SpecFlow to create acceptance tests library.
Features should focus on behavior not on implementation. So putting incidental detail in your features is not a good idea. In your example I'd say that your when steps are somewhat fragmented. Instead using 6 when steps to describe how the user uploads something, use one or two to describe what the user does. For instance.
When user uploads document using company upload template
Something like this describes the necessary detail but not more than that.
Related
I'm writing some end-to-end tests for my application using Cucumber and Selenium. I'm keeping every scenario totally independent of one another, as advised on the Cucumber website. However, my application has session based authentication, so every time a new scenario is run, it will required a login process in order to first access the site. Right now my approach is to put the login scenario as a background scenario for all other scenarios, like so:
Background: User is Logged In
Given I am on the login screen
When I enter my login details
And I click submit
Then I should be logged in
However, this feels like a lot of duplicated 'code'. Moreover, having each scenario run independently requires a new WebDriver instance being created and a browser being run for each scenario, which feels a bit inefficient?
Can anyone advise firstly on how I can avoid duplicating the background scenario in every other scenario (if possible) and secondly if having a separate WebDriver instance for each scenario is the correct approach?
First of all each of your scenarios is going to have to login and create a new session. Thats the price you pay to do end to end testing. The cost of this in runtime should be relatively small with a standard login process as most login screens are simple, and have short rendering times and very little db access. It really is a very bad idea to tray and save runtime here by trying to share sessions between scenarios.
To start tidying your cukes you could follow Daniel's answer, but instead of nesting the steps I would recommend extracting the code to a helper method and calling that instead.
To do this elegantly with power, i.e. deal with different users with roles and extra attributes you need to do a bit more. You can see a detailed example of this here (https://github.com/diabolo/cuke_up).
To use this effectively follow the commit history and focus mostly on the features folder.
This has some framework code that allows you to register/create users and then use them in lots of different configurations. There is some underlying code that is a little complex, which gives you the power to create users who know their own passwords, as well as add other attributes like roles.
The end result is that you can write a step definitions like
Given I am registered
Given I am an admin
Given I am logged in
Given I am logged in as an admin
which are implemented as
Given 'I am registered' do
#i = create_user
end
Given 'I am an admin' do
#i = create_user role: 'admin'
end
Given 'I am logged in' do
#i = create_user
login as: #i
end
Given 'I am logged in as an admin' do
#i = create_user role: 'admin'
login as: #i
end
note: the variable #i is used to pass the user from one step to the next
You don't have to worry about repetition here as all the steps make calls to the same helper methods. You can use the patterns shown here in a much wider context to simplify your features/
This example is in Ruby. You can group up the steps used for Login in login_steps.rb file.
In the .feature file you'll need to write a step like "Given the user is logged in". You can pass in login data in this step as well, if you want. Then in the login_steps.rb file, you create:
Given(/^the user is logged in$/) do
step('I am on the login screen')
step('I enter my login details')
step('I click submit')
step('I should be logged in')
end
I'm sure you can find the equivalent in any other language. Now you can write a the background like:
Background: Given the user is logged in
and it will be used before each scenario of that specific .feature file
As for the Webdriver, as far as I'm aware, you create a session when the test starts and you quit when it ends.
Hope it helps!
Let's say I use Background feature which has the below steps:
Background: My pre-requisites
Given Step one
When Step Two
Then Step Three
....
Then Step Fifteen
So basically my Background has 15 common steps that is required for a set of scenario to run.
My question is : Is it possible to use the Background title My pre-requisites in other feature files, instead of writing down all the 15 steps as Background? Or is there any other way to handle this more gracefully.
You can create a before hook to code the background logic and reuse it across any feature file. Even tag the hook so u can filter out scenarios where u do not want the before hook to run.
Alongside the existing answer, you could also write a new step that compresses the functionality of the 15 steps into one or two steps that express what you are setting up.
The questions revolving around it are these:
Does the business need to know how the user will interact with the site to get to the point where you are ready to begin the test?
Can you bypass how the user will interact with the site to get there without information surrounding the test getting lost in translation between the dev team and the business?
For the former, I would go with a Background. The latter, I would go with a Hook.
For example:
I need to register a user in order to log in as them in my test. The Givens will start at me logging in, so I'll register the user through an API in a Hook.
I need to register a user in order to test the functionality of a new user on exiting the form, and various items surrounding registration, so I'll register the user, starting with some Background steps.
I am first time using gatling and I am stack with such task
I need to provide working code of the following scenarios: dont know how to describe logging in/out
Basic flow: user1 is login in > continue to landing page > navigating through pages
Flow2 : user2 which is already logged in is downloading some pages
Flow 3: user1 and user2 are logging in and out for about 500 time each during 10 minutes.
You can use "Scenarios" and "Chain" them.
Each flow you are mentioning will become a scenario.
To give you a hint:
Start with defining User Requests: for e.x.: Login, Logout, Go To Landing Page, Navigate Some Pages, Download Something etc.
val loginRequest = exec(http("Login").get(...))
Then define Scenario like:
val basicFlow = scenario("My basic flow").exec(loginRequest).exec(browseRequest).exec(logout)
val flow2 = scenario("My 2nd Flow").exec(loginRequest).exec(browseRequest).exec(downloadRequest).exec(logout)
Then you can chain your scenarios in Simulation/setUp like:
setUp(basicFlow(injectAtOnce(1)), flow2(rampUsers(20) over (3 minutes))
You can then improve the load (setUp method), loop your requests etc.
The Gatling Documentation is a good place to start. The following pages will help elaborate more on what I explained briefly above:
More about Scenarios and Control flows like repeat etc.
Simulation and Load setup
If all this seems overwhelming, I would suggest start your record your individual scenarios and then refactor the generated code.
Once you are familiar with the concepts, you can try their helpful Cheatsheet
Edit#1
Gatling has restructured their documentation, updated the links. Please note that the code is as of Gatling 2.0.3, current version is 2.3, so it might not work out of the box, but the concept remains the same.
I'm writing acceptance tests in Gherkin where I want to test for multiple changes in the UI of a web app based on an initial action. Here's an example:
Scenario: Cancel editing a new text asset
Given the user "test_user#fakedomain.com" is logged in
When the user navigates to "/build/"
And the user clicks the "Sandbox" link
And the user inputs "Test story for canceling editing of a new text asset" for the "title" field
And the user inputs "Test User" for the "byline" field
And the user inputs "My summary, so exciting!" for the "summary" textarea
And the user clicks on "Untitled Section" in the section list
And the user clicks the "Text" icon in the "center" container
And the user inputs the following text in the rich text editor:
"""
Test text for asset. This is cool.
"""
And the user clicks the "cancel" button
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
And the "Image" icon is present
And the "Text" icon is present
When the user refreshes the browser
And the user clicks on "Untitled Section" in the section list
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
When the user opens the asset drawer
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
Note that there are multiple groups of When/Then steps, to test for responses of the initial action. While most implementations of steps ignore the prefix keyword, and I'm pretty sure that I can get this test to run, is there a better way to test the different outcomes? Is it better practice to write multiple scenarios with the same setup but different "Then" statements?
Remember that you should test only ONE behaviour/feature at a time.
The rule of thumb is that you should use only one When step:
Given some state before action
And some other state before action
...
When only one action
Then assert correct output
And assert correct output
...
You see - only one line of When, without any Ands under When. If you use many When steps instead, you create test script, not a specification. Your tests will be hard to understand and you will notice that you add more and more steps when the underlying implementation changes.
You also need to keep underlying logic hidden, because you don't want to change it every time you change something irrelevant. Example:
And the user inputs "My summary, so exciting!" for the "summary"
textarea
What if you change the summary field from a textarea to an input type? You have to change the scenario (maintenance nightmare) or left you scenario lying (worse than not having scenario). You should write instead:
When the user describes it as "so exciting!"
But still, the structure of the whole scenario is bad. Ask yourself the question: what I want to check? If I were a person that wants to understand the business logic of the feature I would like to see something like:
Scenario: Cancel editing a new text asset
Given I edit the Sandbox details with some data
When I cancel editing
Then Sandox details should be empty
That's it!
How to achieve it? Move all irrelevant logic deeper, use the PageObject pattern etc. And read about Specification By Example
In contrast to the previous answer, there is no strict rule on how one can use define the steps. Cucumber official documentation at https://cucumber.io/docs/gherkin/reference/ recommends to use only one When due to the fact that only behavior is listed in one acceptance criteria. This is just a recommendation, but not a rule. And Cucumber documentation is mostly focused on the Acceptance criteria specification, but not the test automation.
So, it is definitely possible to follow how it best suits your requirement when it comes to automation testing.
I'd recommend using fewer "And" keywords by merging different steps. Below points I recommend (It is a recommendation, but not a rule :) ):
Use only one Given, When and Then keywords in one flow of a scenario, use "And" keyword if you need to specify extra steps at the appropriate event
If you come across using too many "And" keywords, try to merge multiple such steps
In reality, we cannot just use only one When in the test automation, because:
Number of tests in the automation increases
Total execution time for the automation goes up
We may be doing a lot of redundant actions across multiple scenarios if we use only one When. Consider that you need to perform 5 steps as the initial condition for testing 10 different actions. Here, you will perform these 5 steps for 10 times when you use only one "When" - This consumes too much time, and causes the tests to be unstable, and causes more load on your application
Due to the increased number of tests, we need to spend more time to analyze the results, more time to maintain
I also recommend testing the behavior based on the requirement. If your requirement is about verifying something in the "test area" rather than the "input area", your step should indicate that. Remember that if the requirement is changed, development is code changed, and hence the test automation code is also changed.
I'm a newbie to jMeter, so please bear with me.
I've been assigned the task of testing how an e-commerce website responds under load. I've managed to set up basic tests in jMeter that basically just repeatedly visit the home page, but I'd like to simulate something a bit more realistic:
User arrives on home page
User goes to catalogue page
User views product
User adds product to cart
User returns to catalogue, selects another product, adds to cart
User removes first product from cart
User proceeds to checkout
User completes checkout process.
I'm having trouble finding adequate documentation to explain how to do this. I figured out that I need a cookie manager in my test so that the user session will be maintained, but I haven't figured out how to get the user to traverse the site in a realistic use pattern (such as the one described above). Can anyone help out with this, give me some pointers as where to look for good examples, etc?
This should be no problem, record or manually create the necessary steps as HTTP Samplers, then add them into a Runtime Controller for example to execute them iteratively.
The individual steps will be executed in the order they are in the tree and, in case Cookies are used to handle session state, you might need to add the Cookie Manager to the top of the tree which will handle cookie headers for each user.
Add some timers to simulate user's think time and scale up by increasing the number of virtual users in the thread group.
Use some listener like the Aggregate Report to view the response times for every step.
Try to read http://jmeter.apache.org/usermanual/index.html at first.
Also you'll encounter the problem that Jmeter can't process dynamic pages:
http://wiki.apache.org/jmeter/JMeterFAQ#Does_JMeter_process_dynamic_pages_.28e.g._Javascript_and_applets.29
Does JMeter process dynamic pages (e.g. Javascript and applets)?
No. JMeter does not process Javascript or applets embedded in HTML pages.
JMeter can download the relevant resources (some embedded resources are downloaded automatically if the correct options are set), but it does not process the HTML and execute any Javascript functions.
If the page uses Javascript to build up a URL or submit a form, you can use the Proxy Recording facility to create the necessary sampler. If this is not possible, then manual inspection of the code may be needed to determine what the Javascript is doing.