MSTest not being executed when run by Cruisecontrol schedule - msbuild

Some background: Im trying to implement the integration of Cruisecontrol (2.8.4), MSBuild and MSTest on my current project (.Net 4.0 and VS 2010). I'm using MSBuild to build my solution and added a MSTest as AfterBuild Target into my .csproj files to run my unit tests:
<Target Name="AfterBuild" DependsOnTargets="GetTestAssemblies" >
<Message Text="Normal Test" />
<Exec Command="del $(MSTestResultsFile)" ContinueOnError="true" />
<Exec WorkingDirectory="$(WorkingDir)" Command="MsTest #(TestAssemblies->'%(TestContainerPrefix)%(FullPath)',' ') /noisolation /resultsfile:$(MSTestResultsFile)" />
<Message Text="Normal Test Done" />
I've configured cruisecontrol.config to use MSBuild, ie:
<schedule interval="300">
<exec command="cmd.exe"
workingdir="C:\CruiseControl\projects\Framework"
args="msbuild Solution.sln" />
</schedule>
When I run the build from the command prompt ie: cmd.exe:
msbuild <projectname>.sln
, the solution and projects builds correctly, the unit tests runs and outputs the results to the /resultsfile:$(MSTestResultsFile) specified.
My problem is when the command in the cruisecontrol schedule is executed, the build (MSBuild) actually succeededs but my MSTest: /resultsfile:$(MSTestResultsFile) is empty. It seems that the unit tests wasn't run at all.
I have no clue why this is happening?
Any help would be appreciated!

By change the cruisecontrol schedule config to use the msbuild.exe in stead of cmd.exe resulted in the unit tests being executed
<schedule interval="300">
<exec command="msbuild.exe"
workingdir="C:\Windows\Microsoft.NET\Framework\v4.0.30319"
args="C:\CruiseControl\projects\DTSFramework\DTS.Solution.sln" />
</schedule>

Related

Can I run test project using NAnt without NUnit

I have created test project in MVC for unit testing without using NUnit.
Can I use NAnt to run this project?
If no what is the suitable method to automatically run test cases (if possible sequentially)?
You don't have to use NUnit. If you already created a Unit test project in MVC you can use use command line to run this through nant.
From my understanding, you rather just execute the unit test poject you have in your solution which provide you with your test results. If that's the case then all you need to do it run that project using the msbuild command line. If you can run anything on command line chances are you can do the same in NAnt.
Here is how that translate to NAnt.
Example:
<target name="run.unittest.project" >
<property name="msbuild" value="C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe"/>
<property name="configuration" value="Debug" />
<exec program="${msbuild}" commandline="YourSolution.sln /p:Configuration:${configuration};Platform=OurPlatform /t:Project_Name"/>
</target>
But....If you wanted to run a NUint test on a specific assembly in NANT you can try this...
<target name="nunit.target" description="" >
<property name="UnitTests" value="Path\project\bin\${configuration}" />
<echo message="Starting the UnitTests ....." />
<nunit2>
<formatter type="Plain" />
<test assemblyname="UnitTests.dll" appconfig="UnitTests.dll.config" />
</nunit2>
</target>

How do I run a jar file with msbuild?

I'm trying to make Visual Studio do my entire build, currently I've got my extra build steps written in nant. But it's not ideal having to run ant separately.
I'm trying to run a jar file named plovr as part of my node application, although at the moment publish keeps failing on the line I've added to my build with exit code 1. This is the code I'm trying at the end of my build file within the <Project></Projet> tags.
<Target Name="Build">
<Exec Command="java -jar $(Plovr) build $(PlovrConfig)" />
</Target>
I've got these properties setup earlier in the file
<Plovr>dependencies\plovr.jar</Plovr>
<PlovrConfig>dependencies\plovr-config.js</PlovrConfig>
How can I get msbuild to run the plovr.jar?
If you put in the full path to java.exe.....the EXEC command should work.
It just does a command line call....at the end of the day.
<Target Name="Build">
<Exec Command=""C:\Program Files (x86)\Java\jre7\bin\java.exe" -jar $(Plovr) build $(PlovrConfig)" />
</Target>
Also note the use of " .. to delimit a quote...
You can also put in some Message's to make sure you have what you think you have:
<Message Text="Plovr: $(Plovr)"/>
<Message Text="PlovrConfig: $(PlovrConfig)"/>

MSBuild hangs after NUnit is finished

I'm trying to set up a MSBuild with NUnit as unit test driver but the script keeps hanging after NUnit is done. It doesn't seem to finalize its work and let MSBuild get on with its job.
I'm working in .NET 4.0 and using NUnit 2.5.8.
If I run the test manually or using the gui (either VS2010 or NUnit) it works fine but not when called by MSBuild.
I'd appreciate any help with error finding or just a heads up on where to looks for answers.
The manual command looks like this:
C:\....>nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.IntegrationTest.dll /xml=nunit.xml
and the abbreviated MSBuild:
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<!-- define folders for build output and reports -->
<PropertyGroup>
<BuildPath>buildbinaries\</BuildPath>
<ReportPath>buildreports\</ReportPath>
<ReleaseFolder>release_artefacts\</ReleaseFolder>
<PublishFolder>c:\ZZZ Applications\published builds\</PublishFolder>
<DeploymentFolder>\\seldclq99\ZZZ_Costanza_Dev$\</DeploymentFolder>
</PropertyGroup>
<PropertyGroup>
<!-- specify assemblies that should be included in coverage report -->
<NCoverAssemblyList>YYYY.XXXX.Extractor.Business.dll; YYYY.XXXX.Extractor.Common.dll YYYY.XXXX.Extractor.Configuration.dll YYYY.XXXX.Extractor.DAL.Access.dll YYYY.XXXX.Extractor.DAL.Facade.dll YYYY.XXXX.Extractor.Service.Contracts.dll YYYY.XXXX.Extractor.Service.dll YYYY.XXXX.Extractor.Service.Host.WebHost.dll YYYY.XXXX.Extractor.ServiceGateway.dll</NCoverAssemblyList>
</PropertyGroup>
<!-- define item group for deliverables -->
<ItemGroup>
<Binaries Include="$(BuildPath)/**/*.*" Exclude="$(BuildPath)nunit*" />
</ItemGroup>
<!--
This is the default target that will be executed if MSBuild is not started
with a specific target (this is decided by the DefaultTargets attribute in
the root element of this XML document)
-->
<Target Name="BuildAndTest">
<CallTarget Targets="SetupDirs" />
<CallTarget Targets="Build" />
<CallTarget Targets="UnitAndIntegrationTest" />
<CallTarget Targets="FxCop" />
<CallTarget Targets="CopyToReleaseFolder" />
</Target>
<!-- Setup folders used during the build -->
<Target Name="SetupDirs">
<RemoveDir Directories="$(ReportPath);$(BuildPath);$(ReleaseFolder)" ContinueOnError="true"/>
<MakeDir Directories="$(ReportPath);$(BuildPath);$(ReleaseFolder);$(AssemblyVersionFolder)" ContinueOnError="true"/>
</Target>
<Target Name="Build">
<!-- build the software using msbuild -->
<!-- Build error in the Install build-->
<MSBuild ContinueOnError="true" RebaseOutputs="false" Targets="Clean;Rebuild" Projects="YYYYXXXXExtractor.sln" Properties="Configuration=Release;OutDir=..\$(BuildPath)" />
</Target>
<!--Run the coverage stats-->
<Target Name="UnitAndIntegrationTest">
<Exec Command="nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.IntegrationTest.dll /xml=$(ReportPath)nunit.xml "/>
<CallTarget Targets="UnitTest" />
</Target>
<Target Name="UnitTest">
<Exec Command="nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.UnitTest.dll /xml=$(ReportPath)nunit.xml"/>
</Target>
<!-- Run FxCop -->
<!-- The ForceError.bat fires if the xml file is not found... aka an error was found -->
<!-- The quiet command forces an Xml file ONLY if warnings or Errors are found -->
<Target Name="FxCop">
<Exec Command="..\tools\fxcop\FxCopCmd.exe /p:..\FxCopSettings.FxCop /o:$(ReportPath)fxcop.xml" />
<Exec Condition="Exists('$(ReportPath)fxcop.xml')" Command="..\tools\fxcop\FX_Cop_Failed_Rule_Checks.bat" />
<!--STATS: Run again but don't fail and this time run for all rules.-->
<Exec Command="..\tools\fxcop\FxCopCmd.exe /p:..\FxCopSettingsALLRULES.FxCop /o:$(ReportPath)fxCopAllRules.xml" />
</Target >
I had the same problem with NUnit 2.5.8. There is some discussion of this at the nunit site about the test process hanging. I switched to NUnit 2.5.7 and the problem went away.
It looks like this was fixed a couple of weeks ago in 2.5.9.
I have noticed similar behaviour on out build server since upgrading to .NET 4. MsBuild seems to intermittently hang on either NUnit, FxCop or Dotcover EXEC commands. If you check task manager the process for externally executed command (e.g. Nunit.exe) is still hanging around. If you manually kill the process MsBuild continues on it's merry way - which is far from ideal!
Could this be a bug in the latest version of MsBuild? Our build server was running quite happily until the .NET 4 upgrade.
If you run ProcessExplorer on your server you will notice that an out of band process called nunit-agent is spawned which ends up blocking the nunit runner.
I have not validated that this is fixed in 2.5.9, but it might be some info that could be helpful.

How to call the a unit test target in a project from a 'solution' project

I'm trying to get Team City to build my .NET solution and run my nUnit tests.
I know I can modify the individual projects and tell them always run the unit tests. I don't want the unit tests to run when I click "build" in visual studio, but I do want the unit tests to run when Team City kicks off a msbuild task.
I tried "msbuild solutionname.sln" and gave team city the targets of "BUILD" and my custom build tag of "TEST". However, msbuild can't find any specified target when invoked against a sln solution. So, I ran msbuild to convert my solution into a project which has a target like this:
<Target Name="Build">
<MSBuild Projects="#(BuildLevel0)" >
</Target>
I naively thought I could write a new task like this:
<Target Name="BuildAndTest">
<CallTarget Targets="Build"/> <!-- This builds everything in solution -->
<CallTarget Targets="Test"/> <!-- DOES NOT WORK. This target exists in project that gets built by this solution -->
</Target>
The nunit target looks like this:
<Target Name="Test" DependsOnTargets="Build" Condition=" '$(Configuration)' == 'Release'">
<NUnit Assemblies="$(OutputPath)\Tsa.BaseTest.dll" ContinueOnError="false" ToolPath="C:\Program Files\NUnit 2.5.2\bin\net-2.0\" DisableShadowCopy="true" OutputXmlFile="$(OutputPath)\nunit-results.xml" />
</Target>
As you can see, it references OutputPath, which only the project knows--the solution doesn't have reference to $OutputPath, else I'd just put all the test targets into the "solution project".
Any suggestions on how I can get this to work?
I think you're making this a lot harder than it needs to be. TeamCity has built-in support for running NUnit unit tests after the build - you don't need to modify the MSBuild file at all. Just set up your Build Configuration (I think it's under Runner) to specify the version of NUnit and which assemblies are test assemblies.
NOTE: I checked and we have this under Runner: sln2008 (section NUnit Test Settings) in TeamCity Enterprise Version 4.5.4, but I don't see anything on the JetBrains site that states that it's specific to Enterprise. It may require a version upgrade, though. See TeamCity Testing Frameworks.
This is what finally worked. It is ignored by visual studio, msbuild will run this section correctly, and team city will as well, although it replaces the Target with its own an runtime (according to the build log).
TeamCity will "automatically" run nunit tests and display the results, only in the sense that it will do so after manually editing the msbuild file, doing numerous manual teaks and telling TeamCity where each assembly is and where each output file is.
<Project (snip) DefaultTargets="BuildAndTest" (snip)>
<Target Name="BuildAndTest">
<CallTarget Targets="Build" />
<CallTarget Targets="TestBase" />
</Target>
<Target Name="TestBase" DependsOnTargets="Build">
<NUnit Assemblies="Tsa.BaseTest\bin\RELEASE\Tsa.BaseTest.dll" ContinueOnError="false" ToolPath="C:\Program Files\NUnit 2.5.2\bin\net-2.0\" DisableShadowCopy="true" OutputXmlFile="$(SolutionDir)\Tsa.BaseTest\bin\RELEASE\nunit-results.xml" />
</Target>
</Target>
</Project>

Filter on Category for NUnit MSBuild tasks?

I'm setting up a MSBuild project to run some NUnit test, using the MSBuild Community Tasks Project.
With these settings I'll be able to run the NUnit tests:
<Target Name="Test" DependsOnTargets="Build">
<CreateItem Include="$(ProjectTestDir)\$(ClassLibraryOutputDirectory)\*.Tests.dll">
<Output TaskParameter="Include" ItemName="TestAssembly" />
</CreateItem>
<NUnit Assemblies="#(TestAssembly)" />
</Target>
... but how can i run only certain tests - say the ones with a specific Category? We've added different categories to our tests - some to run all the time, some to only run at a nightly build etc.
I don't see any settings allowing me to filter on this, or did I miss something? Surely some of you have had this same problem and solved it somehow? It doesn't have to be using this msbuild community tasks project.
Looking in the CHM, there's an IncludeCategory property... sounds like what you're looking for? (The CHM file is installed in C:\Program Files (x86)\MSBuild\MSBuildCommunityTasks)