I am currently evaluating possible solutions to limitations in Microsoft's Test Manager 2010 TestStep Editor control and I was wondering whether anyone knows if and if so, how you can write richtext to the Action and Expected Result fields of a step? As far as I saw it takes a 'ParameterizedString' as input, but I am not sure what parameters MSDN refers to (or the MTM TestRunner can handle) when saying "[...]Represents a string that has embedded parameters.[...]".
The Expected Results string is a list of string namees delimited by spaces, semi-colons or commas. You can enter something like: #Test, #Cart, #Book or #Test #Cart #Book.
You can't add HTML to the Expected Result parameters or the action; however, I am not sure I would describe this as a limitation. What action were you thinking you would need to be able to do this. Actions are typically "Click this" or "Go to here" Did you just want a stronger visual queue for certain parts of the action?
Here is an article that describes the intent with the expected results section:
The real power of the test case though, comes in the bottom portion of
the screen, on the Steps tab. This is where you list out the manual
steps the tester will use to test the application. Start typing where
it says "Click here to add a step." In the Action column, you add the
action that the tester should try and perform, such as "Open Internet
Explorer." In the Expected Results column, you list what should happen
when the action is performed, such as "Internet Explorer opens to its
default home page." To continue adding new steps to the test case, hit
Enter to go to a new line.
Not every action step requires an expected result. When an expected
result is specified, it turns that step into a verification step. With
a verification step, the tester will have to specify whether that
particular step was successful or not.
You have the ability to provide parameters to your test steps. This
allows you to run the test multiple times, each time using a different
value. An example of this would be testing different login values on a
Web site. Instead of creating a new test case for each login, you can
create one test case, and pass multiple login values into it.
Related
I've been playing around with IBM API Connect Test and Monitor (looks like a free to use rival to Postman) and would like to create an integration test (a test with multiple steps).
Looking at the example APIs it seems like it's possible, e.g. /api/examples/retail/products returns a list of products and /api/examples/retail/product (singular) looks up information on each of the products. However I can't work out how to connect the calls together. Is this definitely possible and if so how do I construct the test?
It certainly is possible to create an integration test and I can try to talk you through the steps using the example API you mentioned.
create a test in the normal way, go into "Compose" and then you may as well use the wizard option to choose "I got this, start from scratch"
Add a request and choose GET, then specify https://us-east.apitest.apiconnect.ibmcloud.com/app/api/examples/retail/products as the URL. Set the variable field to something like "products_data" or whatever you like. Save that request by clicking the tick button.
Add another request and choose "Each", set the expression to be the content of the variable you just defined, e.g. products_data.content.products. This will create a loop you can nest other requests inside. Again save and close with the tick button.
Add another request, this time a "Set (variable)". The name can be just id or whatever you like. Set the mode to string, and the value to ${_1}. Click on the tick to close and then use the mouse to drag the variable piece so it's nested under the "for each" loop.
Add one last GET request where the URL is the single item endpoint, e.g. https://us-east.apitest.apiconnect.ibmcloud.com/app/api/examples/retail/product. Inside Query Params, add the id query parameter and select "Variable" as the value, referencing id (the variable we just declared in step 4). Save and close the request and drag it into the for loop as well, underneath the variable.
Now you can run the test and it should work, generating multiple calls to the single product endpoint.
This will also work for login methods where you post to a sign-in page and obtain a session token, then use it for a set of other requests against secure endpoints.
Great question - integration testing is a powerful mechanism to simulate an actual customers interaction with your API vs. testing a single endpoint.
Here is a 2-part tutorial series that walks you through an example in detail:
Part 1: https://medium.com/apitest/how-to-never-go-hungry-again-ad0a18453fc
Part 2: https://medium.com/apitest/dessert-how-to-never-go-hungry-with-apis-3fd804515364
Hope this helps! Learn more about the product here: http://ibm.biz/apitest
I've noticed a problem that seems to crop up with several of my otherwise cleanly organized Selenium test classes (though the problem could equally apply to unit or other feature/acceptance tests). I'll have a bunch of tests that are all logically grouped together:
login with valid password
login with valid password and go to page Foo
login with valid password and go to page Bar
login with valid password and then log out
etc.
All of these tests have "login" as part of them, so I add that to the setUp method:
self.login("fake#example.com", "testuser", "testpassword")
But then I need to add one more test to the suite:
fail to login with invalid password
And then I'm not sure what to do, because that test doesn't do the same login action as the others.
At this point, I'm left with several different options ... none of which I really like:
Start a whole other suite just for that last test (even though it logically belongs with the others)
Leave the setUp logic, but make my last test logout as its first action (which results in the test doing a pointless login/logout before even starting)
Leave the setUp logic, but add a line to the setup to skip the login if we're on the last test (ie. if self._testMethodName == "last test" ..., but this just feels hacky)
Take the login out of the setUp (but then I have login lines in every other test in the suite, which don't really add anything to the test)
My question is: which of the above approaches is the "best practice" (or is there a fifth option that I'm missing which is the real best practice)?
If you're testing the login page of the application, then why you're placing the tests inside the setup() method.
I believe setup method is used to set the environment for your tests. eg. Opening the browser, Loading the website/ application etc.,
You could use different Test method for testing the login page, and you can pass various inputs to test it.
I'm trying to test whether logging into an account is successful using Selenium IDE.
Is there a particular command that I can use in Selenium IDE to test whether I can successfully log into my account on facebook for instance? Any help would be very much appreciated. Thanks!
There isn't a single command to fill out an entire form. However, the type command can be used for each field:
Replace username with whatever the actual ID of each field is (which can be seen by right-clicking the page and viewing the page source). You can use many other HTML attributes in this manner, such as name and class; have a look at the documentation for more examples.
Then, simply repeat this procedure for each field you wish to fill in. To actually log in, you'll need the click command targeting the Login button. This is used the same way as the type command above, but doesn't need anything in the Value field.
Once you're actually in, you can check if it was successful by using the assertText command, targeting something you know the value of (your profile name, for example). If Selenium-IDE can't find the element (or the name doesn't match), it'll let you know that the test has failed.
I'm writing acceptance tests in Gherkin where I want to test for multiple changes in the UI of a web app based on an initial action. Here's an example:
Scenario: Cancel editing a new text asset
Given the user "test_user#fakedomain.com" is logged in
When the user navigates to "/build/"
And the user clicks the "Sandbox" link
And the user inputs "Test story for canceling editing of a new text asset" for the "title" field
And the user inputs "Test User" for the "byline" field
And the user inputs "My summary, so exciting!" for the "summary" textarea
And the user clicks on "Untitled Section" in the section list
And the user clicks the "Text" icon in the "center" container
And the user inputs the following text in the rich text editor:
"""
Test text for asset. This is cool.
"""
And the user clicks the "cancel" button
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
And the "Image" icon is present
And the "Text" icon is present
When the user refreshes the browser
And the user clicks on "Untitled Section" in the section list
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
When the user opens the asset drawer
Then the following text is not present:
"""
Test text for asset. This is cool.
"""
Note that there are multiple groups of When/Then steps, to test for responses of the initial action. While most implementations of steps ignore the prefix keyword, and I'm pretty sure that I can get this test to run, is there a better way to test the different outcomes? Is it better practice to write multiple scenarios with the same setup but different "Then" statements?
Remember that you should test only ONE behaviour/feature at a time.
The rule of thumb is that you should use only one When step:
Given some state before action
And some other state before action
...
When only one action
Then assert correct output
And assert correct output
...
You see - only one line of When, without any Ands under When. If you use many When steps instead, you create test script, not a specification. Your tests will be hard to understand and you will notice that you add more and more steps when the underlying implementation changes.
You also need to keep underlying logic hidden, because you don't want to change it every time you change something irrelevant. Example:
And the user inputs "My summary, so exciting!" for the "summary"
textarea
What if you change the summary field from a textarea to an input type? You have to change the scenario (maintenance nightmare) or left you scenario lying (worse than not having scenario). You should write instead:
When the user describes it as "so exciting!"
But still, the structure of the whole scenario is bad. Ask yourself the question: what I want to check? If I were a person that wants to understand the business logic of the feature I would like to see something like:
Scenario: Cancel editing a new text asset
Given I edit the Sandbox details with some data
When I cancel editing
Then Sandox details should be empty
That's it!
How to achieve it? Move all irrelevant logic deeper, use the PageObject pattern etc. And read about Specification By Example
In contrast to the previous answer, there is no strict rule on how one can use define the steps. Cucumber official documentation at https://cucumber.io/docs/gherkin/reference/ recommends to use only one When due to the fact that only behavior is listed in one acceptance criteria. This is just a recommendation, but not a rule. And Cucumber documentation is mostly focused on the Acceptance criteria specification, but not the test automation.
So, it is definitely possible to follow how it best suits your requirement when it comes to automation testing.
I'd recommend using fewer "And" keywords by merging different steps. Below points I recommend (It is a recommendation, but not a rule :) ):
Use only one Given, When and Then keywords in one flow of a scenario, use "And" keyword if you need to specify extra steps at the appropriate event
If you come across using too many "And" keywords, try to merge multiple such steps
In reality, we cannot just use only one When in the test automation, because:
Number of tests in the automation increases
Total execution time for the automation goes up
We may be doing a lot of redundant actions across multiple scenarios if we use only one When. Consider that you need to perform 5 steps as the initial condition for testing 10 different actions. Here, you will perform these 5 steps for 10 times when you use only one "When" - This consumes too much time, and causes the tests to be unstable, and causes more load on your application
Due to the increased number of tests, we need to spend more time to analyze the results, more time to maintain
I also recommend testing the behavior based on the requirement. If your requirement is about verifying something in the "test area" rather than the "input area", your step should indicate that. Remember that if the requirement is changed, development is code changed, and hence the test automation code is also changed.
Suppose I have a web page with several links on it. Also it has few buttons which execute some JavaScript.
So should I create one Java class for testing each of these links and elements or should I test all the links in just one test method and other elements in another one(so ending up with two Scripts).
Is there a another way of gouping these tests.
Thank you.
I have found that writing test cases based on actions is much more useful than writing based on pages.
Obviously, we would love to have everything automated. But realistically, this isn't possible. So we test what is most important...which happens to be: 1. The primary purposes of the product you are testing, and 2. The security of the product.
To make this easier to understand, lets say I have a Checkout page.
If I were to test based on a page, I would make sure every link on the page would work. I would verify that I can type in a new number in the quantity field, and make sure that the page verifies that the credit card number I type in is correct.
This method would somewhat test Security, but beyond that, it would test the UI. What would happen if I clicked on the Checkout button, and I was sent to the right page, but the item I was trying to checkout disappeared? That is a huge problem, but the test based on the page would work.
However, if I were to test based on actions (go to this page, add to cart, type in personal information, and then checkout), now I have made sure that the most important part of your program works, checked security, and even a little UI testing.
All in all, write your testing to do what the average user would do. No normal person is going to sit on the same page, testing out every little feature on that page.
It depends on whether you like to see 1/1 tests passed or 2/2 tests passed.