Test cases are created in iterations with our sprint cycles. They are then added to a test plan via requirements-based query for our integration and system testing.
At the end of every sprint and leading up to integration and system testing, the requirements-based query is run repeatedly against the same same static test suite to add any newly created test cases to the test plan.
The requirements-based query will see the new items and add them to the test plan, however, anywhere that displays the total count of test cases in the test plan or suites does not update automatically. This includes counts shown in TFS web GUI ("Show count from child tests"), MTM, and any dashboard charts.
The only way to get that count to update is to manually click on each test suite generated from the requirements-based suite.
Repro Steps:
Create Test Plan - "Test Plan A"
Create Static Suite in newly created Test Plan - "Static Suite B"
Right click on "Static Suite B" and select "New requirements-based suite"
Fill out criteria for requirements-based suite. My query: Top Level: 'Work Item Type - In Group - Feature/Requirement/Bug Categories' and 'Release Number = XX.XX'. Linked Work Items: 'Work Item Type = Test Case'. 'Return all top level items' is selected.
Run query. Results display in grid, including top level items that do not yet have test cases but are assigned to upcoming release.
Select all results in the grid.
Select "Create Suites". Suites are created for all items included in the release, including items that do not yet have test cases. Note the total count of test cases in "Static Suite B". This count will display in any charts for this test plan and/or suite.
Add a test case for an item that is included in the release but did not have a test case during prior step. Note the top level item for upcoming step.
Navigate back to "Test Plan A".
Right click on "Static Suite B" and select "New requirements-based suite"
Query used in Step 4 displays. Do not make any changes to the query.
Run query again. Results display in grid for all top level items, including those that do not yet have test cases but are assigned to upcoming release. Newly created test case should display in the results.
Select all results and select "Create Suites". No new suites are created since they were created in Step 6 above. Do not select existing requirements based suite for the test case that was added in Step 8.
Check the total count of test cases in "Static Suite B" and any test plan/suite charts. Observe count has not incremented per new test case.
Navigate to "Static Suite B" and select the requirements-based suite where test case was added in Step 7. Observe test case displays in the requirements-based suite as expected.
Check the total count of test cases in "Static Suite B" and any test plan/suite charts. Observe count is not incremented per new test case.
Screenshot of requirements-based query and results (obfuscated)
Screenshot of counts that do not update (obfuscated)
Has anyone else encountered this and have a fix/work-around?
Using local TFS Version 16.122.27102.1.
Related
I am trying to decide on the structure my test cases/suites should have in TFS 2018 (and MTM).
I want to divide based on sprints eg Sprint 1 one test plan, Sprint 2 another test plan etc.
These plans might have the same test cases/ suites.
The available options I found in order to achieve this is to:
Clone the test plan A and create a test plan B by selecting which test suites i want to bring to the new plan.
Create test suites by referencing
Add existing test cases to test plan
The first way is good because the new plans are separate from the old ones so if i delete or make changes to the new test cases it won't update or delete the original ones in the previous sprint. The problem with this way though, is that if i want to add in a later phase some tests from a previous sprint there is no way to clone it in that specific new sprint (it always create a new one as far as I know)
The other two ways are good because you can add anytime new test cases or test suites from the previous to the new sprint but in case you want to delete a test case /suite from the new test plan, it will delete this test suite/case from the previous test plan as well. And i would like to avoid that so that i know what exactly i have run in previous sprints.
A fourth way which i don't think it is a good way, it is to just have a single test plan with different folders for each sprint and just copy test cases between the folders.
What is the best way to achieve what I want ?
There will be only one test plan. And it will have test suites and all your test cases under them.
Test Suite1 (12 test cases)
Ready (5)
NotReady (7)
Every time, you just move the test cases between these Ready, NotReady folders. And every time you will execute only test cases in "Ready" folder.
In sprint 1, if you want to execute test cases 1,2,5,6 only. Then move only 1,2,5,6 test cases to "ready" folder and the rest in to "NotReady" folder.
I am creating Test Plans using Query Based Suites and am getting multiple items.
Create a Query based suite with very simple rules -> Run Query
Let's say 6 items are returned. I select all 6 and click "Create Suite"
The Suite that is created has 18 items (each of the 6 items is copied 3 times)
Any ideas how I can get this to stop?
No no no, the issue is it adds it to a different configurations:
So in order to see 1 test instead of 3 you need to click right mouse button on the configuration and un-assign all configs that you don't need.
Where to adjust default settings I don't know :(
This question is very similar to this post however I am trying to identify the specific id of the test result during the execution of the test. This is so that I can mark the screen shot with the specific id and attach it to the test run result after the execution. (This is, in turn, because the REST API does not return any results until the execution is complete.)
I am looking for a way to get the id's of the individually highlighted items during their execution:
Currently, I retrieve id's such as the release id during its execution like this:
Environment.GetEnvironmentVariable("RELEASE_RELEASEID")
But I do not know how to get information about the tests at that time.
When I run test cases individually, values are updated in Global Properties.
When I run test suite which contains 2 test cases by using test runner, at the time values aren't updated in Global Properties.
I am still able to see the old values which were generated when I ran the individual test cases.
Basically, say I have a report A and I want to add a link column to it, and I set its target to a page in the application...the page its referring to has another report (lets call it B)
Is it possible for me to somehow make it so that table A's link column opens the page with Report B with rows that have the same column value for one of their columns?
Example here:
A
Name Num1 Num2
--------------
A 5 3
B 3 3
C 4 2
B
Name Quantity Serial
--------------------
D 2 3
E 1 8
F 4 6
So if I click the link column for row A, I want it to open report B and only shows rows where its Num2 = Serial, so only row D would show since it is the only one that equals 3
Using Report Linking to Filter Other APEX Report Outputs
I had some fun with this one although. The "A" and "B" stuff was pretty dry, so I decided to create a data set that was more engaging, and perhaps clearer to understand for the rest of us... :) This is how the data-ecosystem was broken down, and the way I fulfilled the OP requirements.
The Test Schema:
Welcome to the manufacturing facilities of the "Recipe Stack" Food Works. The schema design and ERD (Entity Relation Diagram) is below, with the sample data used for this demonstration:
The data relations are as follows:
The staff at the Stack Food-Works keeps an inventory of all ingredients for the types of meals and prepared foods they manufacture. Each ingredient has a unique ID (INGREDIENT_ID) and the staff tracks the amounts of each item in their pantry.
Each ingredient can be used in multiple recipes, but they will be used exactly once for a given recipe.
The ENTREE_RECIPE table has a COMPOSITE KEY which means it is the combination of the two pieces of this composite key (ENTREE_NAME and INGREDIENT_SEQ) that should be unique.
There is a FOREIGN KEY relation between the INGREDIENT_ID values of both tables.
Report Display Requirements (APEX and SQL Design Elements)
Selecting an item from the FIRST report is used as the input and the restriction/filter criteria of the second report.
User Case #1:
User Selects a Ingredient ID from the list of available ingredients in the pantry.
Input from (1) filters output of the RECIPE REPORT. This is a list of all the recipes that have the chosen ingredient in their formula.
User Case #2:
User Selects an ENTREE_NAME from the RECIPE REPORT. The ENTREE_NAME is used to deliver a third report: the RECIPE FORMULA which is the full recipe for the entree item that was selected from the previous report.
Testing Tools
(You usually need these these for the more complex pages, so it's a good start to use or develop them for the easier ones...)
I made my own, but you can also invoke the SESSION link on the developer's tool bar at the bottom of the APEX page on your running application (when it is displayed).
Here's my idea; it's a header region that also has a button to RESET input values so that I can clear the cache and retest or try other examples. I'll show later how you can use this link to see what is going on. You can see it in the discussion of testing at the bottom of this guide.
APEX Report/Page Design and SQL Parametrization
My columnn linking scheme looks similar to the previous post such as the one from FTaveras. This is how my linking works. What's different is that I do not go to another page, I simply go BACK to the same page I came from. Redirects and Branches apparently don't care if they are simply returning to the same location.
What is different on the return trip is that page parameters that were originally null or unpopulated NOW have a value. That value now brings life to the reports on the page that were empty.
Step 1: The PANTRY REPORT
Output: Query all items from the FOOD_SUPPLIES table.
Inputs: Supply links by INGREDIENT_ID to filter the RECIPE_REPORT output.
How to do it: (hint) To accomplish this, define the report column/field value in your report layout design page as a "linked" column and assign its value as a page item. The page item will be referenced in the SQL query of the next report...
Step 2: The RECIPE REPORT
Output: Query all records from the ENTREE_RECIPE table which have the INGREDIENT_ID from Step 1 within their formula.
Input: Supply links by ENTREE_NAME to filter the RECIPE_FORMULA output.
How to do it: (hint) Include the page item defined from Step 1 within the SQL query of this report:
SELECT * FROM entree_recipe
WHERE ingredient_id = :P3_INGREDIENT_ID
Step 3: The FORMULA REPORT
Output: Query all records from the ENTREE_RECIPE table which have the ENTREE_NAME selected from Step 2.
Debug and Test Run
Most will be able to get this far without any problems. If not, here are a couple of examples of how you can debug and test your work. There may be some built-in tools and packages that already exist within Apex, so any suggestions on alternate approaches are welcome in the comments...!
Using the APEX Developer SESSION Output
After selecting the inputs for the trial run, click on the SESSION link on the developer toolbar at the bottom of the page. This is an example output:
Note that the page items that were set for that session are displayed. The inputs I used for this test were:
INGREDIENT_ID: 6432
ENTREE_NAME: peach cobbler (fresh)
Extra Credit: This one is an alternate approach. It may be useful to design something like this as an add-on to any app you design. You do not need to remove it from your app when you push it to production because there is a "conditional display/suppress" feature for page regions. (check it out)
Simply set a global parameter as a "mode" on your Apex app. Set the value to "DEBUG" or "TEST" or whatever and key all your instances of this page region to display only when the global parameter is set to it.
Wrap Up and Discussion
Hopefully, you've enjoyed your visit to the "Stack Food Works" (no tasting or sampling from the line, please).
This has been more of a holistic approach to Apex app design. It helps to have a methodology to map out each step, and a way to check your work at both the beginning and the end of your development process. Using smaller examples like this demo to apply these methods provides a chance to understand Apex development as a Software Creation PROCESS.
Yes it's possible.
On page B, add and hidden Item name for example P(#)_SERIAL where (#) is your current page number.
Modify your query and add one line like AND SERIAL=:P(#)_SERIAL.
On Page A go to "Report Attributes" tab on the report region, click edit.
On the column link section configure your link to page B and set the hidden item on page B to the value on report column of page A.
Name: Item 1 [P(#)_SERIAL] Value: #Num2#
Demo