We are using TFS 2015 vNext Build to manage our CI build.
It is quite easy to enable the test coverage in the visual studio test task.
But in the build summary, it only give a overall block overage percentage.
For each assembly, the summary only indicate the covered block and line.
For example:
It is quite time consuming to download and open the .coverage file to get the detail block covered percentage for each assembly when assembly count is huge.
Is there any way that we can configure the summary to show the block covered percentage for each assembly?
Currently, these customizations are not supported. You can't edit the displays in vNext build result summary.
You can submit a feature request to User Voice site at:
https://visualstudio.uservoice.com/forums/330519-team-services The TFS
Administrator and PM will kindly review the suggestion.
Related
I'm new to DevOps work and I've been tasked to setup a build for one of our .net Standard Library projects. I used the build template for this, which works great! But I want to add a code coverage into build output (there's a test project in the solution).
In the "VsTest - testAssemblies" task, I've checked the "Enable Code Coverage" option, as shown below:
I added a Publish code coverage task and it's asking for something called a "Summary File" - see below:
I'm not sure how to set this option? Has anyone done this?
I seem to get test results published in the build results without config, see below:
And I'd like the same for the Code Coverage tab in the build results, which is currently blank:
Also (may be a completely separate question), I was wondering - can I add a "Quality Gate" to say if Coverage is below 80% then fail the build?
Thanks for any advice in advance - it's a case of knowing what I want, but not sure how to achieve it!
Hurrah!!!
There is a solution for it. :)
You have to install a an Add-on to your organization in VSTS (Azure DevOps).
The name of the addon is Build Quality Checks
Here is the link:
https://marketplace.visualstudio.com/items?itemName=mspremier.BuildQualityChecks
It works perfectly. And this is what you want
By default, the VsTest task will publish the Code Coverage result to Build Summary page directly. So we don't need Publish Code Coverage result task to publish the coverage result. Just make sure you have selected "Code Coverage enable" section in VsTest task.
>> Also (may be a completely separate question), I was wondering - can I add a "Quality Gate" to say if Coverage is below 80% then fail the build?
In current VSTS, we could not set "Quality Gate" to set the build failed if it doesn't match the configured % value. There has other communities also have this requirement and have submit a user voice. Please feel free to add your comments or vote it from below link:
https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/3817520-fail-build-on-insufficient-code-coverage
We are in a specific requirement regarding FxCop integration with TFS2010. The requirement is as follows.
- Execute the build.in specific intervals (There is already a method)
- Run the FxCop after each build. (This is too simple and known)
- If anything fails,need to create a TFS bug item and assign to the person who checked in the file last time.
We know that 'gated checkin' the the best way. But due to some reasons we cannot adopt that. The challenge we are facing is on the creation of the bugs against the last checked in person of each file.
Does anybody have done this type of solution before? Are there any code available public which does this?
Thanks in advance.
It was completed by coding the whole part. The basic idea is as follows
Take latest and run the exsting build script () which produce pdb as well
At the end of the build script start FxCops using FxCopCmd and get the output to xml file
Parse the xml and find out the xml message nodes which contains failed reviews
Extract the code file path from the above xml node
Map the file path to TFS path (ie c:\code to tfs path starting with $\code)
Find the last checked in person's details
Create and assign a bug to that person.
This was specific to our project where we cannot implement the gated checkin due to the large code base and high frequency code check ins. But had to implement automated reviews.
This can be closed
We are using TFS 2010 (Visual Studio) for our deployments and have client code projects (.csproj files) and database projects (.dbproj files) We understand that when our Developers add files to our application there is a corresponding reference to these files in the Project file. If I push a changeset from Dev to QA that includes the project file, and the project file contains a reference to a file that's been added that is not in the changeset, I will receive a build error.
Once we started pushing just changesets (as opposed to performing full builds) this quickly became our number one bottleneck in doing TFS builds. I would deploy the database project and there would be 20 errors. The only way I could correct them was to navigate down the entire solution explorer tree and exclude each orphaned reference individually. This has proved far too time consuming and on the advice of our lead programmer we have returned to doing full builds of QA and UAT.
We are in the early stages of this product, and therefore we will be adding many files for some time. We need a better solution for this problem. Neither the manual exclusions nor asking developers to not check in code until it is ready for qa will suffice for us. Has anybody out there had any experience with this problem and if so how did you deal with it? Thanks!
Jon
Pushing changesets to QA selectively is known as cherry picking and causes the sorts of issues that you are experiencing. This is not the recommended practice, instead setup the Qa build so that successful build is part of checkin. This way that if a part of a fix is missed ( as it may be in multiple change sets ) the build will fail and the checkin cannot be performed.
Second have the developers do the second checkin to QA or merge the dev change sets to Qa and have the team lead coordinate changes to project files by watching for changes by turning on "notify changes made by others " or setting a policy for the dev team. Full builds should always be done as partials do not always pick up the complete pick up the dependency graph.
I have a process for running automated functional tests which is external to Microsoft Team Foundation Server (TFS) 2010. Test cases are tracked as Test Case work items within TFS, however. After running these tests, how can I publish the results to TFS using the TFS API? Can someone point me to sample code that demonstrates this?
Please note that I expressly want to avoid a solution that requires transformation of my test results into the .trx file format. Searches have turned up dead links, or solutions that rely on this method.
It sounds like the following blogs may almost be what you're after
http://blogs.msdn.com/b/jpricket/archive/2010/02/23/creating-fake-builds-in-tfs-build-2010.aspx
http://msmvps.com/blogs/vstsblog/archive/2011/04/26/creating-fake-builds-in-tfs-build-2010-using-the-command-line.aspx
It doesn't actually have code for adding test results, however it does say this:
"In order to associate test results and the like, you have to create build project nodes with the fake build."
You should be able to create a Microsoft.TeamFoundation.Build.Client.TestSummary with a summary of your test results.
There's a couple of internal classes that look interesting, specifically Microsoft.TeamFoundation.Build.Controls.TestRunDetails, which could potentially be useful if you don't mind using some reflection.
However what I would recommend, is using the API to look at the nodes in a standard TFS build to see how they are built up.
Is it possible to set number of test runs variable on TFS 2010 build so that is displayed in the build summary? We are running NUnit tests and can get the number of tests run. At the moment we see "No Test Results" in build summary.
and congratulations on your promotion from shoe-salesman to TFS technician... :)
You can add the NUnit test run results to the TFS build report in one of two ways:
1. You can write a simple XSLT to transform the nunit report to a TRX file (which TFS can read).
2. You can use NUnit for Team Build.
Either way, the NUnit results will be added to the report, including the count of successful, failed, inconclusive and (IIRC) total count of tests.
Hope this helps,
Assaf.