Code coverage with .NET Core 2.0 - asp.net-core

I am trying to setup code coverage on our ASP.NET Core 2.0 web application. I use dotcover (from JetBrains) to provide code coverage on all my other builds (by running dotcover from the command-line during the build process).
When I run dotcover from our build server for our .NET Core 2.0 web app I see this.
It states that it has run successfully but then just hangs there and no code coverage files are created.
dotcover analyse /TargetExecutable:"C:\Program Files\dotnet\dotnet.exe" /TargetArguments:"test MyUnitTests.csproj" /Output:report.html /ReportType:HTML /LogFile=dotcover.log
If I try and add code coverage collection I see this.
dotcover analyse /TargetExecutable:"C:\Program Files\dotnet\dotnet.exe" /TargetArguments:"test MyUnitTests.csproj --collect:coverage" /Output:report.html /ReportType:HTML /LogFile=dotcover.log
And finally if I run dotnet test on its own (without dotcover) it seems to have worked, but again no coverage output is created.
dotnet test "MyUnitTests.csproj" -- collect:coverage
I'm unsure how to generate code coverage for a .NET Core 2.0 app, and not sure how / what data collectors are and how they should be used. Is dotcover the data collector in this example?
Basically, I just want to generate code coverage for a .NET Core 2.0 application.
UPDATE:
As per the suggestion I have installed coverlet as an alternative to dotcover. I have got it working but am getting inconsistent behaviour. When I run my batch file from the server it's all fine.
But when run from TFS I get an error.
System.IO.FileNotFoundException: No test is available in . Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate and try again.
My project does have a reference to that assembly (it's installed by default by VS itself when you create a unit test project).
Why is TFS complaining about that assembly when it's definitely there and can be run manually from the command-line without an error?

I don't know if dotcover can works with net core, but you can take a look for coverlet, also you can find an example here: Cross platform code coverage arrives for .NET Core

I have just migrated a project from net461 to dotnet core and I wanted to maintain code coverage. Using the command line suggested by the OP was what I needed. I am migrating fake scripts where I was previously using dot cover to generate the snapshot file to be pushed to teamcity and an html report. I can confirm that the following works for me:
DotNet SDK 2.1.2
JetBrains.dotCover.CommandLineTools 2018.1.0
Also tested positive with 2017.1.20170403.131707
Microsoft.NET.Test.Sdk 15.7.0
NUnit 3.10.1
NUnit3TestAdapter 3.10.0
DotCover cover
<dotcoverpath>\dotCover.exe cover /TargetExecutable:"C:\Program Files\dotnet\dotnet.exe" /TargetArguments:"test --no-build --no-restore ./src/<test-project-root>" /Filters=+:<namespace-to-test> /Output:./bin/DotCover.snapshot
DotCover analyze
I am not doing the following as I use the DotCover report command on the generated snapshot file but I can also confirm that the following is working for me as well.
<dotcoverpath>\dotCover.exe analyze /TargetExecutable:"C:\Program Files\dotnet\dotnet.exe" /TargetArguments:"test --no-build --no-restore ./src/<test-project-root>" /Filters=+:<namespace-to-test> /ReportType=HTML /Output:./bin/coverage.html
Notes
In my test run I am not restoring packages (--no-restore) or compiling (--no-build) the project as I have already done this in a previous step. These can be removed to suit your needs.
The OP did say that dot cover appeared to hang after the tests had completed. It is worth pointing out that the effort in turning the snapshot at the end of the test coverage into a report can take a while if there is a large amount of code and or assemblies. You do in fact see dot cover pausing at the point the tests have completed whilst it generates the html report. Use the filter (/Filters=) parameter to cut down on the size of things being covered, especially if you don't want to do any coverage on third party components.
/Filters=ARG (Optional) Specifies coverage filters. Syntax:
+:module=*;class=*;function=*; Use -:myassembly to exclude an assembly from code coverage. Asterisk wildcard (*) is supported here

Related

Run Xamarin android unit tests for result

I had a question regarding the Test APK topic.
Is there a way to use the tests/RunApkTests.targets and
build-tools/scripts/TestApks.targets
files. so I can reuse the targets to run my own android unit tests and get results as part as our continuous integration?
I tried using the NUnit Tutorial with no success. I also created an issue here but I kept researching and found some potentially useful targets there.
I'd rather use these targets through a nuget package as opposed to manually downloading the target files and their associated files.
Any suggestions?
Thank you for your time!

dnx test on Travis with mono hangs

I'm trying to run my ASP.NET 5.0 (beta 8) tests which use xunit on Travis CI using language: CSharp as the build environment.
The project builds fine, and after installing dnvm I run the tests with dnx test.
However, whether the tests pass or fail, I get the following message after a while:
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated
I think this is happening because the dnx test command doesn't pass control back to the Travis console when the tests have completed. Travis thinks it is still running, but is not producing any output.
I think this is related to https://github.com/xunit/xunit/issues/489 and this bug in Mono: https://bugzilla.xamarin.com/show_bug.cgi?id=28793 but that thread says it was fixed in May 2015, and I'm running on the latest version of Mono (4.0.4).
It might be caused by a known bug. Try running the tests without parallelization and see if that helps:
dnx test -parallel none
You should be able to reproduce the issue locally too.

How to setup a TeamCity build for a ASP.NET 5 project

I'm trying to setup a CI server for a website that I'm developing, but I can't find any info regarding how to do it with the new ASP.NET 5.
I got you brother. This took me a few days to figure out. This configuration is on TeamCity v10 for a ASP.NET Core 1.0 RC2/preview2 project. As a bonus, I am including the step where it pushes to Octopus Deploy. You will need to install the dotnet teamcity plugin and the newest Octopus Deploy plugin with Push functionality. Here's an overview of the build steps:
First off, don't try to use dotnet restore to restore the packages. It won't work if you have internal nuget packages that are not compiled as .Net Core. This took forever to figure out. I would ignore trying to use dotnet restore until people have converted everything over to .Net Core or Microsoft fixes dotnet.exe to be more flexible.
Some of the stuff I read said to use the newest beta version of NuGet, 3.5. When I tried this, I would get the following error.
[14:30:09][restore] Starting NuGet.exe 3.5.0.1737 from D:\buildAgent\tools\NuGet.CommandLine.3.5.0-rc1\tools\NuGet.exe
[14:30:10][restore] Could not load type 'NuGet.CommandAttribute' from assembly 'NuGet, Version=3.5.0.1737, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
I don't know what that means, and I don't care. Use 3.4.4 for now. Fill in the rest as appropriate.
The dotnet publish step is pretty straightforward. Make sure you provide the output directory because you want to use it in the final step. Also, be sure to specify an absolute path by using the %teamcity.build.workingDir% variable because of this bug. Otherwise it will fail to find your web.config file and not finish publishing the entire site. You'll be missing things like web.config and wwwroot!
Finally we Push to Octopus. This was very tricky for me. Note the part that says
%teamcity.build.workingDir%/published-app/**/* => OrderReviewBoard.1.0.0.zip
IF ANY PART OF THIS IS INVALID, YOUR STEP WILL FAIL WITHOUT EXPLAINING ITSELF!!! By invalid, I mean maybe you put a teamcity environment variable (like the %build.number% they show in all the examples) in that zip name that doesn't properly resolve. Or you specify a non-existent path. Or any number of things, you will see an error that says "[Octopus Deploy] Please specify a package to push". That means that one was never generated because that statement failed. I realize you want to have an auto-incrementing build number there. I'll leave it up to you to figure out how to do that.
Don't get all confused by what is running here. Octopus tries to explain it on their site, but it is hidden here. There is octo pack and octo push. The new version of octo pack is running out of sight, based on whatever statement you put in that "Package paths" box. Don't get sidetracked trying to create a nuspec package, or trying to use dotnet pack. These are dead ends for our purposes. Create a .zip file and move on with your life. Finally, notice the additional command line arguments I added. These help you out a tiny bit. They aren't required. Good luck.
We (the ASP.NET team) use TeamCity as the build server. Each repo has a build.cmd file, similar to this one. TeamCity simply invokes that file.
For Mac/Linux builds, there is a build.sh file.
At the moment you can try to use TeamCity plugin for .NET Core projects:
https://github.com/JetBrains/teamcity-dotnet-plugin
Please check these blog posts;
http://blog.coderinserepeat.com/2015/01/25/building-asp-net-5-projects-in-teamcity/
http://blog.maartenballiauw.be/post/2014/12/19/Building-future-NET-projects-is-quite-pleasant.aspx
Since there has been many changes to the ASP.NET Core world and I got asked about it a few times, I wrote down a step-by-step guide on how to setup a CI/CD environment using TeamCity for .NET Core. I think it is especially helpful for beginners.

Once a build fails in TFS 2010, all builds after it on the same agent fail until the service is restarted

This is an extension of an earlier issue I had, which I posted about here: VB.Net Method appears to be called multiple times when it actually isn't, and returns inconsistent results
At that time I had thought the issue was fully resolved, but I've since learned that there were multiple issues. I have worked out all the kinks with the code and logic, and now I've narrowed down the last remaining issue.
Builds will run perfectly fine until one legitimately fails. After that point, all the builds on that specific agent will fail, even if there were no actual errors with the subsequent builds. When they fail, I see the exact same symptoms as I did in the other thread, which I will summarize here:
A method that is only called once appears to be called multiple times as evidenced by the build messages
The build messages will show there is 1 error, but there are no errors in the logs generated by that method (The line to write the error to the log comes immediately after the line where the error is added to the result log, in the same if statement)
Restarting the build service on the build server "fixes" the problem until the next time a build legitimately fails, at which point the cycle starts again
Build server setup:
Windows Server 2008 R2
VS 2010 Premium
Build uses a custom library
There are 4 build agents, one for each environment (Dev, QA, UAT, Prod)
The build and deploy custom library is written in VB using .Net Framework 4.5, and it calls on another library that is written using .Net Framework 4.5 as well.
The build template doesn't use msbuild. It's strictly WF that's been stripped down to the bare minimum and only calls commandline programs (Like robocopy) or custom .net classes.
I have verified that if a build fails for one agent (such as Dev), then all builds using that agent will fail. When it happens, ONLY builds that run on that particular agent will fail, other agents run just fine. I have not had a chance to test yet if just restarting the agent will "fix" it, or if I have to restart the entire service.
I have tried clearing the Visual Studio cache for the user account running the builds, but that didn't help at all. I also have a GUI interface that I designed that does all the same things as the build, and uses all the same code, and I am not able to get it to repeat the behavior.
I've enabled logs on the build controller per these steps: How to enable build agent or build controller’s logs? however they were no help when this problem occurred (There were no errors logged, nothing was different in the logs for builds that were fine vs. ones that bugged out)
I'm not sure what my next troubleshooting steps should be, I know the problem isn't with my code, and I'm having no luck searching the internet via Google, MSDN, StackOverflow, or picking the brains of my peers at other organizations.
I'm wondering if this could be a bug due to running code that requires .Net Framework 4.5 in TFS 2010 build, since it only officially supports up to 4.0.
My advice would be to change the build logger to diagnostic and look at the output, there must be some rubbish being added to the environment.
You can do this with a switch like
/l:FileLogger,Microsoft.Build.Engine;logfile=c:\build.log;Verbosity=Diagnostic
you might want to take a look here

Using MSTest with CruiseControl.NET

We have been using CruiseControl for quite a while with NUnit and NAnt. For a recent project we decided to use the testing framework that comes with Visual Studio, which so far has been adequate.
I'm attempting to get the solution running in CruiseControl. I've finally got the build itself to work; however, I have been unable to get any tests to show up in the CruiseControl interface despite adding custom build tasks and components designed to do just that. Does anyone have a definitive link out there to instructions on getting this set up?
Not sure if that helps (i found the ccnet Documentation somewhat unhelpful at times):
Using CruiseControl.NET with MSTest
The CC.Net interface is generated via an XSL transform on your XML files put together as specified in the ccnet.config file for your projects. The XSL is already written for things like FxCop - check your server's CC xsl directory for examples - shouldn't be too hard to write your own to add in the info - just remember to add the XML output from your tests into the main log.