I'm noticing the asterisk next to Test Plans in my view on Visual Studio Team Services. It shows next to the the text in the same manner it shows for items without saved changes. I want to remove the asterisk.
I made sure existing Test Plans were Saved and Closed. Even closed and reopened VSTS. I made sure to all switches were off from the Preview feature menu as suggested by the similar question below.
"What are the asterisks on some Team Services menu items?"
Asterisk next to Test Plans in Team Services.
That’s because we use new Test Plans hub to manage VSTS test plans now.
Introducing the new hub for Test Plans! Not only can you add, edit,
and delete plans from this page but also find and favorite plans
across your team and the project. We pioneered this type of experience
in Dashboards, Queries, and Plans. This is another step to bring this
consistency across the product.
More details, you can refer the release note for Manage test plans using the new Test Plans hub.
Related
I am trying to formalize the development workflow and here's the first draft. Welcome suggestions on the process and any tweaks for optimization. I am pretty new when it comes to setting up the processes and would be great to have feedback on it. P.S: We are working on an AWS Serverless application.
Create an issue link in JIRA - is tested by. The link 'is tested by' has no relevance apart from correcting displaying the relation while viewing the story.
Create a new issue type in JIRA - Testcase. This issue type should have some custom fields to fully describe the test case.
For every user story, there will be a set of test cases that are linked to the user story using the Jira linking function. The test cases will be defined by the QA.
The integration/e2e test cases will be written in the same branch as the developer. E2E test cases will be written in a separate branch as it's a separate repository (Open for discussion).
The Test case issue type should also be associated with a workflow that moves from states New => Under Testing => Success/Failure
Additionally, we could consider adding capability in the CI system to automatically move the Test case to Success when the test case passes in the CI. (This should be possible by using JIRA API ). This is completely optional and we will most probably be doing it manually.
When all the Test cases related to a user story to success, The user story can then be moved to Done.
A few points to note:
We will also be using https://marketplace.atlassian.com/apps/1222843/aio-tests-test-management-for-jira for test management and linking.
The QA should be working on the feature branch from day 1 for adding the test cases. Working in the same branch will enable the QA and developer to be always in Sync. This should ensure that the developer is not blocked waiting for the test cases to be completed for the branch to be merged into development.
The feature branch will be reviewed when the pull request is created by the developer. This is to ensure that the review is not pending until the test cases have been developed/passed. This should help with quick feedback.
The focus here is on the "feature-oriented QA" process to ensure the develop branch is always release-ready and that only well-tested code is merged into the develop branch.
A couple of suggestions:
For your final status consider using Closed rather than Success/Failure. Success/Failures are outcomes rather than states. You may have other outcomes like cancelled or duplicate. You can use the Resolved field for the outcomes. You could also create a custom field for Success/Failure and decouple it from both the outcome and status. You ideally do not want your issue jumping back in forth in your workflow. If Failure is a status then you set yourself up for a lot of back and forth
You may also want to consider a status after New Test Creation for the writing of the test case and a status after that such as Ready for Testing. This would allow you to more specifically where the work is in the flow and also capture the amount of time that is spent writing tests, how long test cases wait, and how much time is spent actually executing tests and defect remediation
Consider adding a verification rule to your Story workflow that prevents a story from being closed until all the linked test cases are closed
AIO Tests for Jira, unlike other test management systems, does not clutter Jira, by creating tests as issues. So you need not create an issue type at all.
With it's zero setup time, you can simply start creating tests against stories. It has a workflow from Draft to Published (essentially equaling Ready for Testing).
The AIO Tests jira panel shows the cases associated with the stories and their last execution status to get a glimpse of the testing progress of the story as shown below. It allows everyone from the Product to the Developer to get a glimpse of the testing status.
You can also create testing tasks and get a glimpse of the entire execution cycle in the AIO Tests panel.
It also has a Jenkins plugin + REST APIs to make it part of your CI/CD process.
We have a main CRM application, that used to be housed externally but has been brought in-house. We have all our stand-alone websites/apps/etc that update/read from this main application. Each of those individual projects have their own selenium tests associated with them. What I assume those projects due, is drum up the resources, builds the tests, and runs against those existing websites, and gets rid of the resources, because no longer needed.
It seems like we should be able to reference those build pipelines from one project into another. Is there a way to do this? Seems like it would be easier to manage this way.
From what I have read, copy the test dll's into the main crm project to run. Is that what is needed to be done? The test piece on the CRM application will find all those test dlls? And then all the reporting are under the main crm sometimes and sometimes under the stand alone projects in other times? I much rather would like the test results to be associated with the repos/projects associated with the functional area.
So, the solution is that you can add a deployment group, with the Deployment Group you could test the applications in different machines.
In your scenario, you could see all projects included in that deployment group under your main CRM, then you could kick off each build pipeline has your tests in them.
Please refer to Automating Selenium Tests in Azure Pipelines for details
For the past day, the 'New Test Plan' button has been missing from our VSTS UI inside the Test-Test Plans area. I also can't find anywhere else in VSTS giving the option to create a new test plan. (I'm sure there used to be a few places, but they were obvious, and now missing)
Could it be anything to do with star symbol showing in the UI after ' Test Plans* ', like some changes need saving somewhere before new test plans can be added? In which case, what changes should I be looking for to save?
Generally Test plan creation is limited to users subscribed to Test Manager extension or VS/MSDN subscribers.
Basic license users can only run tests. You need a valid Visual Studio
subscription (Enterprise, Test Professional or MSDN Platforms) or Test
Manager license ($52 monthly):
https://marketplace.visualstudio.com/items?itemName=ms.vss-testmanager-web
to create test plans.
See this thread : Can't add a new test plan
So please check if you have the license to create the test plan, and for an paid extension you must assign that extension to users who need access, so they can start using that extension's capabilities. Please see Assign paid extension access to users
Whatever, I can reproduce the issue on myside, seems it's an issue with the preview feature.
As a workaround you can create the Test Plan with MTM (Microsoft Test Manager) or REST API (Test Plans - Create). Both work for me, but the test plan will not display immediately after creating it, you may need to wait for several minutes to sync it.
Just wondering if there is a way to integrate TFS with TestRail to replace(get rid of entirely) the Test Hub within TFS to use TestRail to record Test Plans?
My concern with removing the Test Hub, would be if Test Rail can still reference IDs in Bug and Stories within TFS and vice versa?
You currently cannot remove the standard Hubs in Visual Studio Team Services / TFS or replace them with something else. You can enable an extension that either adds its own functionality under said hub as a separate tab or adds another Test Rail hub to the top level (if there is any) or write your own. Extensions currently cannot leave their sandbox to overwrite standard functionality.
There is nothing preventing anything from keeping track of work item numbers anywhere, so the second part of your question, whether it would have broken any form of integrations, that's unlikely.
If you are on TFS, you could try creating a custom process template that doesn't have the Test Case, Shared Step, Test Suite and Test Plan work item types, this will likely at least completely cripple the existing functionality. In the on-prem version you can also customize the files on disk, I've never tried, but it's likely that you could probably hack the test hub away. That would be a totally unsupported scenario through.
I have a process for running automated functional tests which is external to Microsoft Team Foundation Server (TFS) 2010. Test cases are tracked as Test Case work items within TFS, however. After running these tests, how can I publish the results to TFS using the TFS API? Can someone point me to sample code that demonstrates this?
Please note that I expressly want to avoid a solution that requires transformation of my test results into the .trx file format. Searches have turned up dead links, or solutions that rely on this method.
It sounds like the following blogs may almost be what you're after
http://blogs.msdn.com/b/jpricket/archive/2010/02/23/creating-fake-builds-in-tfs-build-2010.aspx
http://msmvps.com/blogs/vstsblog/archive/2011/04/26/creating-fake-builds-in-tfs-build-2010-using-the-command-line.aspx
It doesn't actually have code for adding test results, however it does say this:
"In order to associate test results and the like, you have to create build project nodes with the fake build."
You should be able to create a Microsoft.TeamFoundation.Build.Client.TestSummary with a summary of your test results.
There's a couple of internal classes that look interesting, specifically Microsoft.TeamFoundation.Build.Controls.TestRunDetails, which could potentially be useful if you don't mind using some reflection.
However what I would recommend, is using the API to look at the nodes in a standard TFS build to see how they are built up.