I have a question about getting test data for automated testing, here is my problem:
I need to prepare automated testing scripts for a clothes shop.
And I need to know what is the best practice for getting test data needed for the following scenario:
Scenario description: Checking if a user can correctly add a product to the basket.
Given I am on the "WOMEN'S DRESSES" page
When I add "XXX" product to the bag
Then I can see "XXX" product displayed in the basket
My question is: how to ensure that the "XXX" product is always available and what is the best practice for this?
Do I have to always connect to the env database and check if the "XXX" product is available and if not then insert it into DB?
Or maybe should modify a little BDD scenario and get the list of currently available products on the "WOMEN'S DRESSES" page, chose any product, add it to the bag and check if it is correctly added to the basket? ( but in that case what to do if there are no products available for the testing env ?)
I want to have efficient and strong automated tests.
There are a few options for this. Some developers and I looked into a similar problem we had for a car-marketplace. We actually found out that the "dirty" way was the best way for us.
We simply did a BeforeFeature database query and picked up all brands which were available in the test environment. We added these brands to FeatureContext and ScenarioContext, depending on the required context. Because of the Feature/Scenario context, you can use these values during the run of that particular feature/scenario. During the "When" step, in the code, we created a list of all the brands available in the database.
Then, we passed every brand through specflow to the code and checked the list if it contained the brand. If the list did contain the brand, a click had to take place on that brand and we checked in the last step if we were landed on the page of that brand.
You could do something similar with the dresses.
So our specflow looked somewhat like:
Scenario Outline: Check available brands till vehicle detail page
Given I navigate to Vehichle Search Page
When I search '<Brand>', if available in test environment
Then I am navigated to the vehicle detail page
Examples:
|Brand|
|Audi |
|BMW |
|etc |
Just an addition:
This way of writing test keeps your test very BDD. Business can read what your are testing. Also, as long as the data model remains as-is, your tests are very maintainable. You can add unlimited amounts of dresses to your table and you will only get test results if the brands are available. So if the database is swiped clean, sure, not a single test will be executed, but you won't get any false positives or unnecessary negatives.
Also, you can choose to add brands to the database if they weren't found in the first place. You could do this within the "When" step. If the dress is not found in the list, you can add it with a query in the database at that moment.
In your case you have to add one more precondition:
GIVEN: XXX product is available on the "WOMEN'S DRESSES" page.
If product is not available - it's another test case )
Create XXX product just before test using most suitable way:
api call
web UI
DB update (not recommended)
or just mock api response for XXX product
Related
Let's say I have a Main.story file which contains-
Login Scenario
Search Scenario
AddToCart Scenario
UpdateQuantity Scenario
Checkout Scenario
But now what if I only want to run-
Login Scenario → Search Scenario → AddToCart Scenario → Checkout Scenario and skipping UpdateQuantity Scenario
How could I possibly achieve this without removing/deleting anything from the story file.
BDD style scenarios are meant to be completely independent. In this case, you would have 2 scenarios:
Scenario: I can check out with an updated order quantity
Given I login
And I search
And I add to the cart
And I update the quantity
When I checkout
Then I get a confirmation email (or whatever)
Scenario: I can purchase items
Given I login
And I search
And I add to the cart
And I update the quantity
When I checkout
Then I get a confirmation email (or whatever)
If there's a concern about each of those steps actually being several steps and that looking really ugly in the scenario with 20 givens, so long as each of those actions is tested separately, you can use a compound step. That's a step that, in its definition, calls other steps (through code, not Gherkin). You take the same actions, but the Gherkin has far fewer entries.
The important part is that no scenario should require action from a previous scenario in BDD.
To further clarify, there is a way to re-order things if you really must, but it's very bad practice. If you choose to go that road, see this question:
How do i execute story files in specific order in serenity BDD Jbehave
When a piece of data that I want to assert on in a later test is derived from an earlier step, is there an accepted method of storing and recalling that data later on in the scenario?
For example:
Scenario: Recently viewed product
Given I am on a product category page
And I click on the first product
When I am on a product category page
Then the recently viewed products should list the product I just viewed
On the step And I click on the first product can I somehow persist the product name so that I can assert that I am actually seeing that product in the last step? The reason I can't specify an actual product name in the feature file is that this could and will become out of sync with live product data.
Thanks!
It's entirely up to you how you implement this. The simplest way is in your "I click on the first product" step definition store the clicked product information in a static variable (or as a global variable) and then use it in "the recently viewed products should list the product I just viewed" step to verify it's present.
Store the values you need to persist between steps as FeatureContext class properties.
Don't forget to clear them when the Scenario ends, or before each new Scenario.
I have to integrate my webshop with an external system. This system provides several functionalities, described in their API. This is what I want to do:
Import/sync products (this is available as CSV via request)
Update stock of products (also available as webrequest)
So I have to hook into the Magento stock check and I want some script which updates the products like everyday.
Where should I implement my changes? What is the best way to do this? I can imagine Magento already have some API stock functionalities, but can`t seem to find the right documentation.
Thanks in advance!
Firstly you could use the Magento web service via either SOAP and XML-RPC:
http://www.magentocommerce.com/api/soap/introduction.html#Introduction-SOAP
There's an API for updating stock levels:
http://www.magentocommerce.com/api/soap/catalogInventory/cataloginventory_stock_item.update.html
However, this is usually quite slow and can take some time to update large collections of products, in which case you are better going for a solution which uses the Magento object model directly, or something custom.
Depending upon your level of development expertise there's a few options available. One library to checkout is Magmi:
http://sourceforge.net/apps/mediawiki/magmi/index.php?title=Magmi_Wiki
this is useful for updating lots of product data / stock levels quite quickly.
If you do no require an automated solution, you could always use the standard Magento import/export profiles to import stock levels from a CSV file, a quick search for 'Magento Import Export Profiles' should give you a good starting place:
http://www.seenbest-web-design.com/techclub/importing-a-csv-of-products-to-magento/
http://www.woolleydesign.net/2011/02/updating-store-inventory-in-magento/
I am fully functionally using Magmi now. I have written a plugin to automatically create configurable products, so if anyone is ever interested let me know.
There is already a plugin to process configurable products, my plugin is dependent on that plugin.
Basically, you will only be able to create configurable products if you especially have added this to your CSV, with a column named 'type'. In the column type you can set simple or configurable. With my module that won`t be nessecary. By some logic I automatically recognize the simple products and create a new configurable containing those products.
A little explanation on why I need this:
The stock system of our client updates a list of product (CSV) every few hours, containing entries of products. They sell clothing, and every size, and every color is a new entry. So one product can have 10 entries, entry 1: t-shirt green, size 12; entry 2: t-shirt green, size 13, etc.
I explain this, just in case anybody would ever need something like this, my plugin will probably help you. It will although need a few modification for your situation, since this is very specific to how your list of products is composed.
Anyway, thanks for your help!
I have set up SpecFlow to execute a number of Selenium tests to test a website. This all works fine!
I am just considering if there might be room for some optimization on the structuring of the SpecFlow features.
All the scenarios in a specific feature should use the same login. This is something I have hardcoded in the StepDefinition using the [BeforeScenario()] hook at the moment, since I don't really wan't pollute the scenarios with login info. It isn't relevant for the test.
But at the same time, I would like to remove the hardcoded part, and move this into my feature.
My question is two part.
Can I specify login credentials in my feature description. Sort of like a Given:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Is this good practice?
Does it make sense to do it like this, on the feature level I mean. The scenarios are not dependent on a specific order, and they would execute faster if I didn't need to log in for each scenario.
Thanks for your input.
For steps that are common to all scenarios in a feature you can use Backgrounds:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Too much abuse of background steps can be a bad practice because it introduces coupling between your scenarios.
Another solution is to put the login part directly into your "I Place an order" step.
This will remove all the noise about login stuff since it's implicit that you need to log in to place an order.
I'd also suggest to call it "I've placed an order" instead of "I place an order".
Given steps are usually pre-condition that describes what happened prior using the functionnality (when steps)
When I first read your post, I thought of Dan North: Who's domain is it anyway? and that influences me to think that we should be trying to write our tests so that they stick to a single knowledge domain. As you talk about specific users and navigation and orders in a list, well its very like the example he gives in that your specification is crossing domains. This kind of leads me towards almost making your specification less specific, i.e. its not about navigating and checking a list, do you have an order or not!
Scenario: See list of ongoing order
Given I am logged in with username abc and password xyz
And I place an order
Then should have at least one order
I wanted to give you another example link, to a post I cant find, which talks about the benefits that a team has got by actually defining some example users and what value that gives their testing. So in your business domain you might have a user called Jack who will have placed a large order in the system, and another prospective client Jill with no orders. In this way tests such as
Given I am Jack
When I search for my orders
Then I will find 1
Given I am Jill
When I search for my orders
Then I will find 0
can guarantee that you are only testing your Search functionality, and less about getting the setup in place.
I'd also suggest that you have a look at Liz Keogh: Acceptance Criteria vs Scenarios who asserts that more general broad definitions are less valuable than very specific examples. (It is Specification By Example after all :-) )
So to answer your question, I think having the specified user is a good thing, but doing it how you are doing it is heading a very complicated route.
The scenario examples that are given are completely not in line with the Feature.
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in as John Doe
Scenario: This is taking to long
And I have placed an order more than 29 days ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to cancel it
Scenario: Track package
And I have been notified my order has been send
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to navigate to the postman's site and see its status
Scenario: Review item
And I have received my package
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to review the item
Scenario: Contact Seller
And I have placed an order two weeks ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to ask the seller what is going on
As you can see the When and the Then is starting to duplicate itself and you might want to consider tucking those away.
Is it acceptable to write a "Given When Then When Then" test in Gherkin?
A real-life example is as follows all AllPlayers.com
Scenario: Successfully register a user
Given I am on homepage
And I am not logged into an account
When I follow "create a new account"
And I fill in "First Name" with "Bobby"
And I fill in "Last Name" with "Bricks"
And I fill in "E-mail" with "bbricks#example.com"
And I select "Jun" from "Birthday Month"
And I select "22" from "Birthday Day"
And I select "1985" form "Birthday Year"
And I select "Male" from "Gender"
And I fill in "Password" with "123testing"
And I fill in "Confirm Password" with "123testing"
And I solve the captcha math problem
And I click "Create new account"
Then I should see "the user dashboard"
And I should see the Registration Wizard
When I push "Proceed to next step"
Then the "First Name" field should contain "Bobby"
And the "Last Name" field should contain "Bricks".
I know it works using behat, so parsing it isn't a problem. I'm just trying to write better tests. I could write in the first then And the Registration Wizard should be filled out with data but that doesn't seem specific enough...
Suggestions?
It depends on the target audience of the feature as written. It seems highly likely that the gherkin you've got there was not written with a stakeholder (i.e. somebody not-techie but has a vested interest in the business and the website). BDD is really about the conversation about requirements and expectations - and Gherkin is a tool which gives a standard/recognised way that everyone should be able to read that you can write the requirements and expectations; in a way that serves as automated tests for a developer and perhaps test scripts for a tester.
Trying to take my developer hat off now - I would say that a business stakeholder would rather read, and understand easily...
Scenario: Should be able to successfully register on website
Given I am new to the website
And I want to register for a user account
When I go to the registration form
And I complete all the required registration details correctly
Then I will be registered on the website
And I will be automatically logged in
You can still build the same test behind the scenes of this specification - but this specification has larger readership, it is a more easily understood requirement that anyone should understand. I'm not saying what you have got has no value - far from it. It will be a very valid test. But it is quite developer specific, and highly coupled to the UI implementation (if you refactor/redesign the UI, you now need to refactor your Requirements...).
I started off having plenty of gherkin specifications much like yours - and I still use them on occasion. Once your testing framework has built up a little gherkin is a really great way of kind of writing data-driven/configurable unit tests; and they still have great value to my development process. But I do try to separate the more "pure" specifications from my "developer" ones - but folder and tags/categories.
Edit: I guess in summary what I'm getting at is... what you have is a great "test", but a fairly bad "requirement". Stick with it though!
Yes, more than one When/Then cycle is appropriate in a Gherkin scenario when the real-world scenario calls for it.
SaxonMatt's answer makes the excellent point that scenarios are best written in stakeholder language rather than in the language of UI manipulation, and that doing so often reduces the length of a scenario, but that misses the exact point of the question. Let's take the bull by the horns.
Gherkin was designed for acceptance tests: tests which test that stakeholder-level requirements have been completely implemented, i.e. that the software actually provides value to stakeholders. Sometimes providing value takes more than one action-response cycle. Consider the following scenario:
Scenario: Guest buys a product
# This scenario starts with the user not logged in, which doesn't require a step
Given there is a product named "Elliptical Juicer"
When I go to the product page for "Elliptical Juicer"
And I add the product to my shopping cart
Then I should see 1 product in my shopping cart
When I request to check out
Then I should see the account creation form
When I create an account
Then I should see the checkout form with 1 product, "Elliptical Juicer"
When I check out
Then I should see the checkout success page with 1 product, "Elliptical Juicer"
And I should receive a checkout confirmation email with 1 product, "Elliptical Juicer"
(Note that when I have more than one When/Then cycle in a scenario I like to separate them with blank lines so they stand out.)
There are several reasons why this scenario is best written with multiple When/Then cycles:
Before the user checks out, they should see one product in their shopping cart (only as a digit in the site header, so the step doesn't mention the product name). There is no way to test this requirement at the end of the scenario. (Well, the test could collect the information immediately after the user adds the product to their cart and assert the expected count at the end of the scenario, but that would be pointlessly sneaky and confusing.) Instead, assert the correct count at the natural place in the scenario, as soon as it is visible to the user.
Similarly, Then I should see the account creation form and Then I should see the checkout form with 1 product, "Elliptical Juicer" can test important requirements at the points in the scenario at which it is natural to test them.
Suppose we didn't care about what the user sees during the process, only whether they get to the end of the scenario with their product on the way. We might then omit the intermediate Then steps:
Given there is a product named "Elliptical Juicer"
When I go to the product page for "Elliptical Juicer"
And I add the product to my shopping cart
And I request to check out
And I create an account
And I check out
Then I should see the checkout success page with 1 product, "Elliptical Juicer"
And I should receive a checkout confirmation email with 1 product, "Elliptical Juicer"
And I create an account comes as a surprise, doesn't it? It requires the reader to infer that a guest user is asked to create an account during checkout. It's clearer to say so explicitly, as in the first version of the scenario that I gave.
Suppose none of the above concerns convinced us and we wrote a separate Gherkin scenario for each point in the overall scenario where we needed to assert that requirements have been met:
Scenario: Guest adds a product to their shopping cart
Given there is a product named "Elliptical Juicer"
When I go to the product page for "Elliptical Juicer"
And I add the product to my shopping cart
Then I should see 1 product in my shopping cart
Scenario: Guest with a product in their shopping cart attempts to check out
Given I have a product in my shopping cart
When I request to check out
Then I should see the account creation form
Scenario: Guest creates an account
Given I have a product named "Elliptical Juicer" in my shopping cart
And I am on the account creation form
When I create an account
Then I should see the checkout form with 1 product, "Elliptical Juicer"
Scenario: Newly registered user checks out
Given I am a user
And I have a product named "Elliptical Juicer" in my shopping cart
And I am on the checkout form
When I check out
Then I should see the checkout success page with 1 product, "Elliptical Juicer"
And I should receive a checkout confirmation email with 1 product, "Elliptical Juicer"
That's awful! First, none of the scenarios is what a stakeholder would think of as a scenario. Second, when one of the intermediate states changes, two steps will have to change: the step which asserts the intermediate state and the Given step which sets up the intermediate state for the next scenario. Each of those Given steps is an opportunity to set up the wrong state, i.e. make an integration error. This set of scenarios has much less value as an integration test suite than did the single scenario. You might almost have written a series of unit tests.
It's true that writing every scenario end-to-end is likely to lead to some duplication. Just as you tolerate duplication more in unit tests than you would in regular code, tolerate duplication even more in Gherkin scenarios than you would in unit tests. Don't compromise on understandability. Break up scenarios and use Givens only at crucial points (such as creation of a product in the example above), and do so knowing that you're diluting your scenarios' integration-testing power.
Also, keep in mind that acceptance tests should be only part of your automated test suite. Write only enough acceptance tests to cover critical scenarios, and cover the details with unit tests. Often enough, the solution to duplication among acceptance tests is to replace one with a unit test.
I would also say No.
In a separate post of mine Daniel F found this fantastic article. Here is the relevant section:
Given-When-Then steps must appear in order and cannot repeat. A Given may not follow a When or Then, and a When may not follow a Then. The reason is simple: any single When-Then pair denotes an individual behavior. This makes it easy to see how, in the test above, there are actually two behaviors covered: (1) searching from the search bar, and (2) performing an image search. In Gherkin, one scenario covers one behavior. Thus, there should be two scenarios instead of one. Any time you want to write more than one When-Then pair, write separate scenarios instead. (Note: Some BDD frameworks may allow disordered steps, but it would nevertheless be anti-behavioral.)
https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/
I would say no.
When a test fails it should tell you where in your system the failure has occurred.
Long tests like in your example tend to be brittle and require a higher level of maintenance.
You need to define what your test is testing (which should be one thing) reading your test
it could be a form validation test.
it could be a registration test.
it could be a user dashboard test.
It would require an amount of time to investigate where the failure is and where that relates to in the code.
I would also say No.
The Given is a precondition for setup.
The When is an action (which can be a do nothing)
The Then form asserts.
If you need more actions then break the test down.
This will become far more useful once the first Then's fail for localising the problems.