We need to export the latest run with every step and its attachments in AzureDevOps. It is a 300 Test Cases run/execution report that needs to be approved, as we have many use cases, it is not plausible to go one run by one run and download every attachment step and print it by hand.
I tried exporting TestPlans, the export from Test Plan does not apply in this case as it does not care about the runs steps and its attachments. I do not see a solution with the new AzureDevOps run Page,
Would like to know if I am missing any configuration option or if there is an API or even product to export the data. e.g. Example one of many Runs to save in Report Many
I saw other similar questions but still missing how to export runs reports with its steps and attachment. Image below,
Example of one of the 300 runs to export
Many Thanks
UPDATE: As I do not see any an out of the box solution I am trying to get the information from this API ResultsAPI but I am not able yet to get the ActionResultModel that includes the stepIdentifier. Also problems downloading the attachments.
In this AzureDevOps version there is no way to export the run steps result. Also the results API is not retrieving the step result information, I did some test with the API and it is not possible to get the testActionResultModel in the response. The maximum that you can get is the iterations list, but there is no link between the iterations and the steps.
https://learn.microsoft.com/en-us/rest/api/azure/devops/test/action%20results/list?view=azure-devops-rest-5.1#testactionresultmodel
Thank you
You have a try with the export option in Test Plans. First you need to create a test plan and add all your manual test cases using query to this test plan.
You can follow the detailed steps of Creating test plans and test suites.
Go to Test plans under Test Plans of your project. Then click the 3 dots beside your test plan and selet Export
Then check the options you to export.
Then you will get an export of all the manual test cases included in this test plan with their steps and attachments.
Hope above helps!
Related
Hello everyone I would like to ask for your assistance if there's a way I could create an HTML/GUI Report upon running "dbt test" command. Your response is highly appreciated.
This is the result I got upon running dbt test
There are some 3rd party libraries that will generate reports like the one you're looking for. One you can try is called Elementary Data (https://www.elementary-data.com/). When you run DBT tasks, it creates a schema with all of the stats on model execution timings, test results, etc. It then has a command to generate an HTML page using those results to publish some simple graphs and aggregate test results.
I have been searching for a while now and am surprised that I can't find any solutions out there for test result storage with grouping and searching capabilities.
I'd like a service or self hosted solution that supports:
storing test results in xunit/junit organized by keyword. In other words, I want to keep all my "test process A" test results together and all my "test process B" results together. I want to store failure traces and overall pass/fail at a minimum
get last run results for keyword: get the last "auth" test results with failure details
get run history results by keyword in some format
search of some sort on test results
I happen to be have:
Cypress tests
typescript/mocha tests without cypress
custom test framework tests that will need custom reporters
but I am fine with any test results solution that supports a generic input like xunit.
I am definitely open to suggestions that use any other storage system that can accomplish this even if it isn't strictly a test results tool.
I'm having problems creating and executing tests using TFS Test Manager and I'm hoping someone can help me sort them out.
I can create a Test Plan and associate it to an Iteration, and the add a new Test Case through the Test Plan associated to the same iteration. I've also tried creating a further test case through Parameters and associated it to the same iteration but not the Test Plan. Both the Test Plan Statuses have been set as ready. When I go back to the Test Plan, I have two problems as far as I can see:
The Test Cases created do not list under the Test Plan in the the left hand panel.
In the right hand panel, the menu bar with the buttons to execute tests displays for about half a second and is then replaced with an error message saying "No default test configuration is found". I suspect this is because of 1 above, but can't be sure.
I'm clearly configuring something incorrectly but cannot see what. Is anybody able to help me? I'll be greatly indebted to you if you can.
How do we create a snapshot versions of the test results of a test set in HP ALM - Test LAB
I need this versioning to keep track of the past execution details.
Any suggestions on how to achieve this?
I do not believe you need to copy/archive/version anything to get the results you need. Consider this...
First Testing Effort
Follow these steps for the first time you are executing tests.
Create your test set
Add the tests that will be part of your effort.
All tests will be in the "No Run" status with no run history
Execute your tests (multiple times as necessary if they do not pass initially) until your effort is complete.
Second Testing Effort
The next time you need to run the tests, use the same test set.
Reset the test set. Right-click the test set in Test Lab and select 'Reset Test Set'. This step will reset the 'Status of all your previous test runs to 'No Run'.
Execute your tests (multiple times as necessary if they do not pass initially) until your effort is complete.
Comparison
At this time, each test has maintained a full run history. All the runs from the First Effort are still available along with those from the Second Effort. Open the Test Instance details of one of the tests in your test set, and then select the 'Runs' option. There you will be able to see each and every time that test instance was executed, no matter which testing effort it was.
Alternative
If this doesn't meet your needs, an alternative would be to leave the test set from your first testing effort alone when you are done. When your second effort beings, copy/paste your test set and perform all your new runs from that copy. When you copy/paste, it does not copy run history, so the new test set will be a blank slate. There is a COM-based API for doing all of this through code if necessary.
Currently our Jenkins server only displays a history/graph for the overall number of passed/skipped/failed tests - I'm assuming that's the behavior out of the box.
If you select a single test, you'll get information for how long the test was failing (assuming it did fail).
However, we'd like to see is a history for that single test across the different builds to identify whether the test has been failing in the past (and when) even though it just passed. If you find a build where it failed, you could click on it, and investigate what might have caused the failure; if it passes again, you could check whether something actually fixed the test, or whether it was failing randomly all along.
Is this something that can be done somehow through the config, or do we need an additional plugin for this? If yes, which one?
Not sure if this makes much difference, but we're using Java (Maven) & TestNG (Surefire).
Both the TestNG plugin and the JUnit plugin will actually display history of the test results.
You just need to pick a given result and then:
For JUnit click on "History" on the left side, and
For TestNG click you will see the history in the graph above the result. You can just click on the bars in the bars to see the older results, and also if you click closer to the edge, the scope of the test results will adjust
The Test Results Analyzer plugin does the job for me. There appears to be other suitable plugins out there as well.
https://wiki.jenkins-ci.org/display/JENKINS/Test+Results+Analyzer+Plugin
Does the Static Code Analysis plugin help?