Let's say I create a story dedicated to giving the weather in a given place, and a story dedicated to listing the possible activities in a given place, and a allowing to buy tickets for them.
Let's say a user launches the second story by entering "what can I do in Paris ?". Bot answers with a list of things and asks the user to input the one he'd like to buy a ticket for.
But, instead of entering a number, the user inputs "What is the current weather in Paris ?". Will the first story be triggered ?
A wit.ai blog post states :
We strengthened the initial overfitting: when a user follows the
scenario of a story, the bot will 100% stick to the story.
Should we repost the intent as a context key so that the next action prediction scope stays within the same story ? Should all end of a story flush the context of all keys (which in our tests seems to be a very good idea not to stick to a story) ?
Thanks for any insights.
Related
I try to understand the logic behind the weather bot in the wit.ai quick-start.The story is build with a trait intent (supposed to be something like "what is the weather ?") and an entity is declared to get where the weather is wondered. So, I would think this story is only triggered when weather is asked AND location is given.
This is not the case since the illustration of jump / bookmarks just below this first step deals with unset location.
Hence my question : how does Wit.ai decide which story to trigger ?
Actually, there is no difference between :
What is the weather in Barcelona ? and What is the weather ?
from a decision regarding the next action to choose from the Wit engine standpoint.
Previous contexts and current context (and contained key) are what matters for this task.
As the quick-start puts it:
Only trait entities have their value influence the prediction. For
non-trait entities, the value is ignored with regards to action
prediction.
My advice on this is to create a flushContext action which returns an empty context and to trigger it everytime a give story comes to an end.
I am designing a bot with wit.ai and encounter a conflict issue.
In my story the user wants instructions to access a service. I've created an entity service.
There are 2 services available "mailing" and "chat channels" but I want the user to be able to ask instructions for both at once.
Therefore, my entity service has "keywords" for search strategy and "mailing", "chat channels" and "both" as values.
But when I branch the story after the question the bot asks "to which I've added quick answers "mailing", "chat channels" and "both", but at this point I have a conflict between the branches.
When I test the bot on wit.ai, whatever my answer he goes to the "both" branch.
I'm struggling to find out why there is a conflict between the keywords, if someone could help me it would be great.
I've tried to train the bot with small variations of each answer but this doesn't solve my issue.
Here is the solution we came up with for this issue:
instead of creating a new story, start by creating the entityof type traitin the understanding tab
enter all possible values in the understanding tab
once the entity has been properly created, you can use it in a story
I am experimenting with a flow-based conversation with wit.ai. I took the example from https://wit.ai/l5t/FlowBased_Example and noticed a strange thing. If you start the bot right from the page above and say immediately "no", the bot calls the answer-tv function and says "Ok. And do you watch sports online?". Checking out the stories of the bot it can be clearly seen that the answer-tv function is could be called only deep in the conversation, the shortest path being:
-> I want to take the survey!
<- Do you watch sports on TV?
-> No
<- [answer-tv]
<- Ok. And do you watch sports online?
How is it possible the the bot has jumped the first steps, found an arbitrary point in the conversation where the user would respond "no" and continued from there? How can one avoid this behavior?
Picture all these stories as connected graphs. Where the entry point is any one of the stories. The vertices are user messages, bot actions and bot replies. The edges are the bookmark links and branches in a story. To understand the flow, draw a graph on sheet yourself and visualise the above conversation. You can regulate the bot behaviour by structuring the graph appropriately.
I have set up SpecFlow to execute a number of Selenium tests to test a website. This all works fine!
I am just considering if there might be room for some optimization on the structuring of the SpecFlow features.
All the scenarios in a specific feature should use the same login. This is something I have hardcoded in the StepDefinition using the [BeforeScenario()] hook at the moment, since I don't really wan't pollute the scenarios with login info. It isn't relevant for the test.
But at the same time, I would like to remove the hardcoded part, and move this into my feature.
My question is two part.
Can I specify login credentials in my feature description. Sort of like a Given:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Is this good practice?
Does it make sense to do it like this, on the feature level I mean. The scenarios are not dependent on a specific order, and they would execute faster if I didn't need to log in for each scenario.
Thanks for your input.
For steps that are common to all scenarios in a feature you can use Backgrounds:
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in with username abc and password xyz
Scenario: See list of ongoing order
Given I place an order
When I navigate to MyOrders page
Then I can see at least one order in the list
Too much abuse of background steps can be a bad practice because it introduces coupling between your scenarios.
Another solution is to put the login part directly into your "I Place an order" step.
This will remove all the noise about login stuff since it's implicit that you need to log in to place an order.
I'd also suggest to call it "I've placed an order" instead of "I place an order".
Given steps are usually pre-condition that describes what happened prior using the functionnality (when steps)
When I first read your post, I thought of Dan North: Who's domain is it anyway? and that influences me to think that we should be trying to write our tests so that they stick to a single knowledge domain. As you talk about specific users and navigation and orders in a list, well its very like the example he gives in that your specification is crossing domains. This kind of leads me towards almost making your specification less specific, i.e. its not about navigating and checking a list, do you have an order or not!
Scenario: See list of ongoing order
Given I am logged in with username abc and password xyz
And I place an order
Then should have at least one order
I wanted to give you another example link, to a post I cant find, which talks about the benefits that a team has got by actually defining some example users and what value that gives their testing. So in your business domain you might have a user called Jack who will have placed a large order in the system, and another prospective client Jill with no orders. In this way tests such as
Given I am Jack
When I search for my orders
Then I will find 1
Given I am Jill
When I search for my orders
Then I will find 0
can guarantee that you are only testing your Search functionality, and less about getting the setup in place.
I'd also suggest that you have a look at Liz Keogh: Acceptance Criteria vs Scenarios who asserts that more general broad definitions are less valuable than very specific examples. (It is Specification By Example after all :-) )
So to answer your question, I think having the specified user is a good thing, but doing it how you are doing it is heading a very complicated route.
The scenario examples that are given are completely not in line with the Feature.
Feature: As a user I want to be able to see my ongoing orders, and interact with them.
Background:
Given I am logged in as John Doe
Scenario: This is taking to long
And I have placed an order more than 29 days ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to cancel it
Scenario: Track package
And I have been notified my order has been send
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to navigate to the postman's site and see its status
Scenario: Review item
And I have received my package
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to review the item
Scenario: Contact Seller
And I have placed an order two weeks ago
When I navigate to MyOrders page
Then I can see my order in the list
and I am able to ask the seller what is going on
As you can see the When and the Then is starting to duplicate itself and you might want to consider tucking those away.
I'm thinking about the best way to do the following kind of tests with selenium and I wanted asking how you would do.
I have 2 different clients in the system that I have to check, the first one offers something to sell and the second one has to do an offer. The offer can be accepted or declined.
Example.
Person 1: I want to sell a car so I put my car with all its details onto the system.
Person 2: I want to buy a car so I search for a car, I see the car of person 1 so I do an offer. On my dashboard is showed that I did an offer.
Person 1: I access to my dashboard and I notice that there is an offer, I see the offer, I decline because I don't like it.
Person 2: I access to my dashbord and I notice that the offer has been declined, I make another offer.
Person 1: I access my dashbord and I notice the new offer, I like it, I accept.
How would you organize the tests with selenium??
I would consider your example as 5 separate user stories. You can create a separate automated test for each of these stories. For this purpose, you would have to define preconditions for the test and the desired outcome.
For example, for the 1st story there are no preconditions, other than the trivial ones, like a user with such and such username exists and has privileges to sell cars. You will have to script the process of putting car details into your system using Selenium. The desired outcome is the car is in the system (can be searched for or shows up in the full list of cars for sale) and its attributes match what was entered.
For the 2nd story a precondition is that there is a car for sale. You will have to ensure this either by submitting car for sale via the process you developed for the 1st story or by entering it bypassing UI, for example via business services layer or inserting directly into the database. The desired outcome is that the dashboard for user 2 shows the offer.
And so on.