VSTS Build fails running NUnit tests - msbuild

When building a visual studio solution using visual studio team services it runs the unit tests, written in nunit 3 using the nunit 3 NUnit3TestAdapter it starts running the tests.
The error it gives is: Error: Illegal characters in path.
The complete output as follows
2016-02-11T17:02:40.8514411Z Executing the powershell script: C:\BuildAgent\tasks\VSTest\1.0.29\VSTest.ps1
2016-02-11T17:02:41.1483135Z ##[debug]Calling Invoke-VSTest for all test assemblies
2016-02-11T17:02:41.2108131Z Working folder: C:\BuildAgent\_work\5
2016-02-11T17:02:41.2108131Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll" /logger:trx /TestAdapterPath:"C:\BuildAgent\_work\5\s"
2016-02-11T17:02:41.5240692Z Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
2016-02-11T17:02:41.5240692Z Copyright (c) Microsoft Corporation. All rights reserved.
2016-02-11T17:02:42.1326930Z Starting test execution, please wait...
2016-02-11T17:02:46.4921857Z Information: NUnit Adapter 3.0.8.0 executing tests is started
2016-02-11T17:02:46.5077303Z Information: Running all tests in C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll
2016-02-11T17:02:46.6327336Z Information: NUnit Adapter 3.0.8.0 executing tests is finished
2016-02-11T17:02:46.6327336Z ##[error]Error: Exception System.ArgumentException, Exception thrown executing tests
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.6327336Z ##[error]Error: Illegal characters in path.
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.7108474Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2016-02-11T17:02:46.8358887Z ##[error]VSTest Test Run failed with exit code: 1
2016-02-11T17:02:46.8514965Z ##[warning]No results found to publish.
We have an on premise build agent but I don't think this should make an difference.
Any advice you can give it appreciated.

FWIW, I switched on verbose logging by following the Tips and Tricks and got this:
2016-02-23T14:45:49.5620657Z Information: Running all tests in C:\TFS2015\_work\3\s\BlaBlaTests\bin\Debug\BlaBlaTests.dll
2016-02-23T14:45:49.6088663Z ##[error]Error: Exception thrown executing tests
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z ##[error]Error: System.ArgumentException: Illegal characters in path.
2016-02-23T14:45:49.6088663Z ##[error] at System.IO.Path.CheckInvalidPathChars(String path)
2016-02-23T14:45:49.6088663Z ##[error] at System.IO.Path.Combine(String path1, String path2)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.ProcessAddinsFile(DirectoryInfo baseDir, String fileName)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.FindExtensionsInDirectory(DirectoryInfo startDir)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.StartService()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ServiceManager.StartServices()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.TestEngine.Initialize()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.TestEngine.GetRunner(TestPackage package)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnitTestAdapter.GetRunnerFor(String assemblyName)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyName, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z Information: NUnit Adapter 3.0.8.0 executing tests is finished
which correlates with %LOCALAPPDATA%\NUnit\Nunit30Settings.xml being accessed but might not be related. ExtensionService is here.
Issues #117 and #118 are related to this with a fix as provided by RobertK66:
If you install the Version 3.0.8.0 as a Nuget package then there is a
corrupted (not empty!) “ignore.addins” file installed in the
packages\NUnit3TestAdapter.3.0.8-ctp-8\lib folder. Earlier Versions
did install a 0byte empty file!
After removing the content of this file in the
/packages/NUnit3TestAdapter.3.0.8-ctp-8\lib folder and
deleting the cached version of the adapter under
\AppData\Local\Temp\VisualStudioTestExplorerExtensions...
everything works as expected!

I met this issue when I use NUnit3 Test Adapter Nuget Package during the build. And I resolved it by installing NUnit3 Test Adapter and upload all the installed files to Version Control and use it as the Test Adapter.
Since you are using on premise build agent. You can easily install the NUnit3 Test Adapter on the build agent and use "/UseVsixExtensions:true" argument as the error message indicates.

Related

Unable to Publish NUnit test result report getting error message stating, Could not read the XSL XML file

While publishing NUnit report in Jenkin getting an exception "ERROR: Step ‘Publish NUnit test result report’ failed: Could not read the XSL XML file. Please report this issue to the plugin author". I am using
.Net Core MVC App in C#
NUnit project
Jenkin
Using command "dotnet test ".\WebApp2nUnit\WebApp2nUnit.csproj" --logger:"trx;logFileName=Report.xml""
in freestyle jenkin project.
Although build success along with I could see test result report is forming but at the time of publishing this report getting below exception. I am using the NUnit plugin to publish this. Provided is the compete error message:
Test Run Successful.
Total tests: 3
Passed: 3
Total time: 5.3879 Seconds
C:\Program Files (x86)\Jenkins\workspace\WebApp2>exit 0
Recording NUnit tests results
Error in NUnit processing: Could not transform the NUnit report. Please report this issue to the plugin author
ERROR: Step ‘Publish NUnit test result report’ failed: Could not read the XSL XML file. Please report this issue to the plugin author
Finished: FAILURE
Please suggest.
Nishant
I added this package to my solution: https://www.nuget.org/packages/NunitXml.TestLogger/
Then the command used is:
dotnet test folder/project.csproj --filter "TestCategory=category" --results-directory ./folder/ --test-adapter-path:. --logger:"nunit;LogFilePath=TestResult.xml"
and to publish:
nunit testResultsPattern: 'folder/TestResult.xml'

Failed to locate the code coverage command line tool , MSBuild with SonarQube

Getting the following error's once build is triggered from the build definition.
3:46:16 AM Failed to locate the code coverage command line tool Post-processing failed. Exit code: 1
Exception Message: TF270015: 'MSBuild.SonarQube.Runner.exe' returned an unexpected exit code. Expected '0'; actual '1'. See the build logs for more details. (type UnexpectedExitCodeException)
Exception Stack Trace:
at System.Activities.Statements.Throw.Execute(CodeActivityContext context)
at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)
I am using SonarQube 5.3, installed in the build server. I have .net framework installed in build server not the visual studio .
BuildDefinition where I am referencing SonarQube.
Please help me here !!!!
The code coverage tools are installed with Visual Studio not the .Net Framework, so if you want code coverage you'll need to install VS on the build agent.
Depending on the version of Visual Studio you are using, you may not need an additional license for build agent. See the discussion Is a separate Visual Studio license required for a build machine? get more detail.

Visual Studio Online test runner can't load SpecFlow dll

I've added SpecFlow using Nuget and have a single spec (so it is a simple project). I've created a build in VS online 2015 using the web build editor.
Maybe it's a Nuget new build system issue? I'm not sure if it just a SpecFlow thing and haven't gotten a chance try straight MS unit tests yet.
An exception occurred while invoking executor
'executor://mstestadapter/v1': Could not load file or assembly
'TechTalk.SpecFlow, Version=1.9.0.77, Culture=neutral,
PublicKeyToken=0778194805d6db41' or one of its dependencies. The
system cannot find the file specified.
Here's a larger stack trace:
1. 2015-08-03T21:26:27.6325511Z Executing the powershell script: C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\tasks\VSTest\1.0.16\VSTest.ps1
2. 2015-08-03T21:26:28.4231183Z ##[debug]Calling Invoke-VSTest for all test assemblies
3. 2015-08-03T21:26:28.4481177Z Working folder: C:\a\14076a40
4. 2015-08-03T21:26:28.4481177Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\a\14076a40\SpecByExample\SpecByExample.Specs\bin\Debug\SpecByExample.Specs.dll" "C:\a\14076a40\SpecByExample\SpecByExample.Specs\bin\Debug\SpecByExample.Specs.dll.config" "C:\a\14076a40\SpecByExample\SpecByExample.Specs\obj\Debug\SpecByExample.Specs.dll" /logger:trx
5. 2015-08-03T21:26:29.4654019Z Microsoft (R) Test Execution Command Line Tool Version 14.0.23107.0
6. 2015-08-03T21:26:29.4664025Z Copyright (c) Microsoft Corporation. All rights reserved.
7. 2015-08-03T21:26:29.4664025Z Starting test execution, please wait...
8. 2015-08-03T21:26:29.7564024Z Warning: Unable to load types from the test source 'C:\a\14076a40\SpecByExample\SpecByExample.Specs\obj\Debug\SpecByExample.Specs.dll'. Some or all of the tests in this source may not be discovered. If you are running unit tests for Windows Store apps and referencing custom WinMD type then please visit http://go.microsoft.com/fwlink/?LinkId=238340 for more info. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'SpecResults, Version=0.1.0.7, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
9. 2015-08-03T21:26:29.7574029Z .
10. 2015-08-03T21:26:40.6922424Z Passed TheUserVisitsTheWebsite
11. 2015-08-03T21:26:41.3214684Z ##[error]Error: An exception occurred while invoking executor 'executor://mstestadapter/v1': Could not load file or assembly 'TechTalk.SpecFlow, Version=1.9.0.77, Culture=neutral, PublicKeyToken=0778194805d6db41' or one of its dependencies. The system cannot find the file specified.
12. 2015-08-03T21:26:41.3224680Z ##[error]
13. 2015-08-03T21:26:41.4936886Z Results File: C:\a\14076a40\TestResults\buildguest_TASKAGENT-0007 2015-08-03 21_26_40.trx
14. 2015-08-03T21:26:41.4956876Z Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
15. 2015-08-03T21:26:41.4966881Z ##[error]Test Run Failed.
16. 2015-08-03T21:26:41.4966881Z Test execution time: 11.8787 Seconds
17. 2015-08-03T21:26:41.7581094Z ##[error]VSTest Test Run failed with exit code: 1
18. 2015-08-03T21:26:41.7741099Z Publishing Test Results...
2015-08-03T21:26:43.0372165Z Test results remaining: 1
UPDATE: I disabled the SpecFlow test and added the simplest AboutController test possible (from file new project MVC in Visual Studio). I still get a 'could not load assembly problem. SpecByExample.Web is my MVC project in my solution.
2015-08-07T16:16:56.7251975Z ##[error]Error Message:
2015-08-07T16:16:56.7261978Z ##[error] Test method
SpecByExample.Web.Tests.Controllers.HomeControllerTests.AboutTest
threw exception: 2015-08-07T16:16:56.7261978Z
[error]System.IO.FileNotFoundException: Could not load file or assembly 'SpecByExample.Web, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=null' or one of its dependencies. The system cannot
find the file specified.WRN: Assembly binding logging is turned OFF.
2015-08-07T16:16:56.7271982Z ##[error]To enable assembly bind failure
logging, set the registry value
[HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
2015-08-07T16:16:56.7271982Z ##[error]Note: There is some performance
penalty associated with assembly bind failure logging.
2015-08-07T16:16:56.7281979Z ##[error]To turn this feature off, remove
the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
2015-08-07T16:16:56.7281979Z ##[error] 2015-08-07T16:16:56.7291971Z
[error]Stack Trace: 2015-08-07T16:16:56.7291971Z ##[error] at SpecByExample.Web.Tests.Controllers.HomeControllerTests.AboutTest()
2015-08-07T16:16:56.7831986Z Results File:
C:\a\14076a40\TestResults\buildguest_TASKAGENT-0007 2015-08-07
16_16_56.trx 2015-08-07T16:16:56.7861977Z Attachments:
2015-08-07T16:16:56.7861977Z ##[error]Test Run Failed.
Here's a screenshot of my build definition:
In Run unit test step. click advance and give path to your packages folder. (if you are using nuget it will download all dependencies there) give it a try and let me know how it goes.

BuildException when executing Ant task

I have installed Apache Ant and set the correctly the Path, CLASS_PATH and ANT_PATH variables. When I run a jar task in a command line within my project folder where the build.xml file is, I get the following error:
BUILD FAILED
C:\Documents and Settings\TonyS\My Documents\My Workspace\code\build.xml: 32:
The following error occurred while executing this line:
jar:file:/C:/Documents%20and%20Settings/TonyS/My%20Documents/Ant/apache-ant-1.8.1/lib/ant.jar!org/apache/tools/ant/antlib.xml:37: Could not create task or type: componentdef.
Ant could not find the task or a class this task relies upon.
I had this error and found it was caused by having an older version of ant.jar in the classpath.
One possibility is you may have another version of ant installed on your system and your PATH pointing to it.
Check what is ANT_HOME set to.
Run ant -version and see if it shows up 1.8.1
Add the following config in ~/.bashrc
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
Don't forget source ~/.bashrc
Hope useful for you!
you have duplicate ant in your environment.
My case was:
I am using 1.9.7 ant, but ant -version gave me 1.7.1
I installed weblogic and it has 1.7.1 ant, after rename MW_HOME, the issue was gone.
I found this while I was using 1.7.1 to test junit. it shows a warning.
junit:
[junit] WARNING: multiple versions of ant detected in path for junit
[junit] jar:file:/D:/wls12120/oracle_common/modules/org.apache.ant_1.7.1/lib/ant.jar!/org/apache/tools/ant/
Project.class
[junit] and jar:file:/D:/apache-ant-1.7.1/lib/ant.jar!/org/apache/tools/ant/Project.class
[junit] Running HelloWorldTest

Issues with running nunit-console.exe with Mono

I could make a dll for NUnit NUnit test under mono, but when I tried to run in with nunit-console.exe I get the following error message.
Runtime Environment -
OS Version: Unix 10.3.0.0
CLR Version: 2.0.50727.1433 ( 2.6.4 (tarball Thu Apr 22 13:24:33 MDT 2010) )
ProcessModel: Default DomainUsage: Single
Execution Runtime: mono-2.0
** (/Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/nunit-console.exe:36800): WARNING **: The following assembly referenced from /private/var/folders/m4/m4u1hmP+FHOQaiZbHj1UCk+++TI/-Tmp-/nunit20/ShadowCopyCache/36800_634111616836311880/Tests_-22323139/assembly/shadow/54274fc2/118e035c_45a94c9e_00000001/mut.dll could not be loaded:
Assembly: nunit.framework (assemblyref_index=1)
Version: 2.5.5.10112
Public Key: 96d09a1eb7f44a77
System error: Invalid argument
** (/Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/nunit-console.exe:36800): WARNING **: Could not load file or assembly 'nunit.framework, Version=2.5.5.10112, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77' or one of its dependencies.
What might be wrong? I have nunit.framework.dll under /Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/framework, but it doesn't seem to know about this.
This is another frustrating point about using NUnit in Mono. As far as I've been able to tell, you need to have all of those missing files (there will be more than just nunit.framework.dll) in the same directory as nunit-console.exe.
There might be a command line option either in mono or nunit-console.exe to specify another directory to search for missing DLLs -- I'm not sure. Barring that, you'll have to copy the missing DLLs into the same folder as nunit-console.exe, or you'll have to cd into the framework folder and use mono ../nunit-console.exe /path/to/my/tests.dll
Once nuget is installed, you can use it to install nunit.console like so:
(This is from my .circlci/config.yml file)
- run: mono --runtime=v4.0 nuget.exe install nunit.console
- run: msbuild MySolution.sln
- run: mono --runtime=v4.0 NUnit.ConsoleRunner.3.12.0/tools/nunit3-console.exe FirstModule.Tests/bin/Debug/FirstModule.Tests.dll
- run: mono --runtime=v4.0 NUnit.ConsoleRunner.3.12.0/tools/nunit3-console.exe SecondModule.Tests/bin/Debug/SecondModule.Tests.dll
The trick is to specify the entire path for the NUnit.ConsoleRunner and to watchout for the - changing into a . by version 3