I am using MSTest to execute my unit tests against a web service on a remote server.
Is there way to change the path to where the results file(.trx) is generated. Currently my test results is generated in the directory which the Exec command is invoked from:
<Target Name="ExecuteTheTests" AfterTargets="StartService" Condition="'$(ServiceStarted)' == 0 And '$(Configuration)' == 'Release'">
<Message Text="Executing the Unit Tests" Importance="high" />
<PropertyGroup>
<TestSuccessOrNot>0</TestSuccessOrNot>
</PropertyGroup>
<Exec Command=""C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\Common7\\IDE\\MSTest.exe" /testcontainer:..\..\..\..\\_MyOutput\\UnitTests.dll /detail:testname ">
<Output TaskParameter="ExitCode" PropertyName="TestSuccessOrNot" />
</Exec>
<Error Condition="$(TestSuccessOrNot) == 1" Text="Unit tests fail!" />
Thanks,
Many Bothans died* to bring you this message:
mstest.exe /resultsfile:c:\BadPlaceForTestResults.trx
j/k, many Bothans didn't die, they just typed "MsTest.exe /?"
Related
Currently, I have this msbuild script that scans Test DLLs and run once using msbuild /test
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="RunMSTest">
<ItemGroup>
<TestAssemblies Include="**\bin\*Test*.dll"/>
</ItemGroup>
<Target Name="RunMSTest">
<Exec Condition=" '#(TestAssemblies)' != ''"
Command="Mstest.exe /nologo /resultsfile:TestResults.trx #
(TestAssemblies ->'/testcontainer:"%(RecursiveDir)%(Filename)%(Extension)"', ' ')"
/>
</Target>
</Project>
Ideally, I want each Test DLL a separate mstest.exe and the file name saved to TestResults%(FileName).trx
Separation of Microsoft Unit Tests to Reduce the Load of CI Server
https://helloacm.com/separation-of-microsoft-unit-tests-to-reduce-the-load-of-ci-server/
I'm trying to set up a CI environment at a new client site using Team City and MSbuild and the MS build community extensions. Compiling the code seems to work fine. However, when I run my unit tests I get the following error coming from the NUnit task:
log4net : error XmlConfigurator: Failed to find configuration section 'log4net' in the application's .config file.
I've identified the two test projects that are causing this issue. However, I've ran the tests directly from nunit-console, and the resharper nunit test runner and though I see the warning the tests don't fail. I don't want to do anything with the Log4net configuration file or the assembly.cs in any project. All I want to do is make the MSBuild script behave like Visual Studio which doesn't consider the log4net error as a failure.
Here's the build file
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0" DefaultTargets="Compile">
<Import Project=".\MSBuild.Community.Tasks.Targets"/>
<PropertyGroup>
<Configuration Condition="'$(Configuration)' == ''"> Debug</Configuration>
</PropertyGroup>
<ItemGroup>
<BuildArtifacts Include=".\build_artifacts\"/>
<SolutionFile Include ="..\Core.Services.sln"/>
<NUnitPath Include="..\Packages\NUnit.2.5.10.11092\tools"/>
</ItemGroup>
<Target Name="Clean">
<RemoveDir Directories="#(BuildArtifacts)"/>
</Target>
<Target Name="Init" DependsOnTargets="Clean">
<MakeDir Directories="#(BuildArtifacts)"/>
</Target>
<Target Name="Compile" DependsOnTargets="Init">
<MSBuild
Projects="#(SolutionFile)"
Targets="Rebuild"
Properties="OutDir=%(BuildArtifacts.FullPath)">
</MSBuild>
</Target>
<Target Name="DevelopmentBuild" DependsOnTargets="Compile">
<Message Text="Running Unit Tests from %(BuildArtifacts.FullPath)...." ContinueOnError="true"></Message>
<CreateItem Include="%(BuildArtifacts.FullPath)*.Tests.dll">
<Output TaskParameter="Include" ItemName="TestAssembly" />
</CreateItem>
<NUnit Assemblies="#(TestAssembly)"
ToolPath="#(NUnitPath)\"
ContinueOnError="false"
OutputXmlFile="%(BuildArtifacts.FullPath)test-results.xml"
DisableShadowCopy="true"/>
</Target>
</Project>
I'm trying to set up a MSBuild with NUnit as unit test driver but the script keeps hanging after NUnit is done. It doesn't seem to finalize its work and let MSBuild get on with its job.
I'm working in .NET 4.0 and using NUnit 2.5.8.
If I run the test manually or using the gui (either VS2010 or NUnit) it works fine but not when called by MSBuild.
I'd appreciate any help with error finding or just a heads up on where to looks for answers.
The manual command looks like this:
C:\....>nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.IntegrationTest.dll /xml=nunit.xml
and the abbreviated MSBuild:
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<!-- define folders for build output and reports -->
<PropertyGroup>
<BuildPath>buildbinaries\</BuildPath>
<ReportPath>buildreports\</ReportPath>
<ReleaseFolder>release_artefacts\</ReleaseFolder>
<PublishFolder>c:\ZZZ Applications\published builds\</PublishFolder>
<DeploymentFolder>\\seldclq99\ZZZ_Costanza_Dev$\</DeploymentFolder>
</PropertyGroup>
<PropertyGroup>
<!-- specify assemblies that should be included in coverage report -->
<NCoverAssemblyList>YYYY.XXXX.Extractor.Business.dll; YYYY.XXXX.Extractor.Common.dll YYYY.XXXX.Extractor.Configuration.dll YYYY.XXXX.Extractor.DAL.Access.dll YYYY.XXXX.Extractor.DAL.Facade.dll YYYY.XXXX.Extractor.Service.Contracts.dll YYYY.XXXX.Extractor.Service.dll YYYY.XXXX.Extractor.Service.Host.WebHost.dll YYYY.XXXX.Extractor.ServiceGateway.dll</NCoverAssemblyList>
</PropertyGroup>
<!-- define item group for deliverables -->
<ItemGroup>
<Binaries Include="$(BuildPath)/**/*.*" Exclude="$(BuildPath)nunit*" />
</ItemGroup>
<!--
This is the default target that will be executed if MSBuild is not started
with a specific target (this is decided by the DefaultTargets attribute in
the root element of this XML document)
-->
<Target Name="BuildAndTest">
<CallTarget Targets="SetupDirs" />
<CallTarget Targets="Build" />
<CallTarget Targets="UnitAndIntegrationTest" />
<CallTarget Targets="FxCop" />
<CallTarget Targets="CopyToReleaseFolder" />
</Target>
<!-- Setup folders used during the build -->
<Target Name="SetupDirs">
<RemoveDir Directories="$(ReportPath);$(BuildPath);$(ReleaseFolder)" ContinueOnError="true"/>
<MakeDir Directories="$(ReportPath);$(BuildPath);$(ReleaseFolder);$(AssemblyVersionFolder)" ContinueOnError="true"/>
</Target>
<Target Name="Build">
<!-- build the software using msbuild -->
<!-- Build error in the Install build-->
<MSBuild ContinueOnError="true" RebaseOutputs="false" Targets="Clean;Rebuild" Projects="YYYYXXXXExtractor.sln" Properties="Configuration=Release;OutDir=..\$(BuildPath)" />
</Target>
<!--Run the coverage stats-->
<Target Name="UnitAndIntegrationTest">
<Exec Command="nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.IntegrationTest.dll /xml=$(ReportPath)nunit.xml "/>
<CallTarget Targets="UnitTest" />
</Target>
<Target Name="UnitTest">
<Exec Command="nunit\nunit-console.exe buildbinaries\YYYY.XXXX.Extractor.Test.UnitTest.dll /xml=$(ReportPath)nunit.xml"/>
</Target>
<!-- Run FxCop -->
<!-- The ForceError.bat fires if the xml file is not found... aka an error was found -->
<!-- The quiet command forces an Xml file ONLY if warnings or Errors are found -->
<Target Name="FxCop">
<Exec Command="..\tools\fxcop\FxCopCmd.exe /p:..\FxCopSettings.FxCop /o:$(ReportPath)fxcop.xml" />
<Exec Condition="Exists('$(ReportPath)fxcop.xml')" Command="..\tools\fxcop\FX_Cop_Failed_Rule_Checks.bat" />
<!--STATS: Run again but don't fail and this time run for all rules.-->
<Exec Command="..\tools\fxcop\FxCopCmd.exe /p:..\FxCopSettingsALLRULES.FxCop /o:$(ReportPath)fxCopAllRules.xml" />
</Target >
I had the same problem with NUnit 2.5.8. There is some discussion of this at the nunit site about the test process hanging. I switched to NUnit 2.5.7 and the problem went away.
It looks like this was fixed a couple of weeks ago in 2.5.9.
I have noticed similar behaviour on out build server since upgrading to .NET 4. MsBuild seems to intermittently hang on either NUnit, FxCop or Dotcover EXEC commands. If you check task manager the process for externally executed command (e.g. Nunit.exe) is still hanging around. If you manually kill the process MsBuild continues on it's merry way - which is far from ideal!
Could this be a bug in the latest version of MsBuild? Our build server was running quite happily until the .NET 4 upgrade.
If you run ProcessExplorer on your server you will notice that an out of band process called nunit-agent is spawned which ends up blocking the nunit runner.
I have not validated that this is fixed in 2.5.9, but it might be some info that could be helpful.
We are migrating from MSTests to NUnit. The first step was to migrate all our UnitTests projects which was accomplished using the following msbuild task:
<Target Name="RunTests">
<!-- The location of the necessary tools to run nunit tests -->
<PropertyGroup>
<NUnitToolPath>C:\Program Files\NUnit 2.5.2\bin\net-2.0</NUnitToolPath>
<NUnitResultTool>C:\Program Files\NUnit For Team Build Version 1.2</NUnitResultTool>
</PropertyGroup>
<!-- Create a build step representing running nunit tests -->
<BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Name="NUnitTestStep" Message="Running Nunit Tests">
<Output TaskParameter="Id" PropertyName="NUnitStepId" />
</BuildStep>
<!-- Specify which dll's to include when running tests -->
<CreateItem Include="$(OutDir)\Profdoc.UnitTests*.dll">
<Output TaskParameter="Include" ItemName="TestAssembly" />
</CreateItem>
<NUnit
Assemblies="#(TestAssembly)"
ToolPath="$(NUnitToolPath)"
OutputXmlFile="$(OutDir)\NUnit_TestResults.xml"
ContinueOnError="true">
<Output TaskParameter="ExitCode" PropertyName="NUnitResult" />
</NUnit>
<!-- Update the build step result based on the output from the NUnit task -->
<BuildStep Condition="'$(NUnitResult)'=='0'" TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Succeeded" />
<BuildStep Condition="'$(NUnitResult)'!='0'" TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Failed" />
<!-- Upload the results to TFS. -->
<Exec Command=""$(NUnitResultTool)\NUnitTFS.exe" -n "$(OutDir)\NUnit_TestResults.xml" -t "$(TeamProject)" -b "$(BuildNumber)" -f "%(ConfigurationToBuild.FlavorToBuild)" -p "%(ConfigurationToBuild.PlatformToBuild)" -x "$(NUnitResultTool)\NUnitToMSTest.xslt"" />
<!-- Indicate build failure if any tests failed -->
<Error Condition="'$(NUnitResult)'!='0'" Text="Unit Tests Failed" />
</Target>
But i'm at a loss as to how we are going to accomplish the same with out integration tests, because we need to deploy settings and licence files to the binary folder before running the tests. So, how can i deploy files to the binary folder, preferably as a part of the NUnit task (because i want to run the IntegrationTests against different configuration setups)?
I would suggest to create new one target which will copy all required files and make target RunTests dependent on new one, basically:
<PropertyGroup>
<LicenseFiles>$(PathToLicenseFiles)\**\*.lcx</LicenseFiles>
<SettingsFiles>$(PathToConfigFiles)\**\*.config</SettingsFiles>
</PropertyGroup>
<ItemGroup>
<Files Include="$(LicenseFiles);$(SettingsFiles)"
Exclude="*.tmp"/>
</ItemGroup>
<Target Name="CopyDependencyFiles">
<CopyFiles Inputs="#(Files)" Outputs="..." />
</Target>
<!-- Run Integration tests after all files were copied -->
<Target Name="RunIntegrationTests" DependsOnTargets="CopyDependencyFiles">
<NUnit .. />
</Target>
I have a batch script that I want to call from an MSBuild project, and the documentation says I can't use output from the batch (either console / environment variables) in the MSBuild project.
Is there a workaround?
You can redirect the output of the command to a file using "> output.txt" and read that into a variable.
<PropertyGroup>
<OutputFile>$(DropLocation)\$(BuildNumber)\Output.txt</OutputFile>
</PropertyGroup>
<Exec Command="dir > "$(OutputFile)"" />
<ReadLinesFromFile File="$(OutputFile)">
<Output TaskParameter="Lines" ItemName="OutputLines"/>
</ReadLinesFromFile>
<Message Text="#(OutputLines->'%(Identity)', '%0a%0d')" />