Visual Studio Online test runner can't load SpecFlow dll - msbuild

I've added SpecFlow using Nuget and have a single spec (so it is a simple project). I've created a build in VS online 2015 using the web build editor.
Maybe it's a Nuget new build system issue? I'm not sure if it just a SpecFlow thing and haven't gotten a chance try straight MS unit tests yet.
An exception occurred while invoking executor
'executor://mstestadapter/v1': Could not load file or assembly
'TechTalk.SpecFlow, Version=1.9.0.77, Culture=neutral,
PublicKeyToken=0778194805d6db41' or one of its dependencies. The
system cannot find the file specified.
Here's a larger stack trace:
1. 2015-08-03T21:26:27.6325511Z Executing the powershell script: C:\LR\MMS\Services\Mms\TaskAgentProvisioner\Tools\tasks\VSTest\1.0.16\VSTest.ps1
2. 2015-08-03T21:26:28.4231183Z ##[debug]Calling Invoke-VSTest for all test assemblies
3. 2015-08-03T21:26:28.4481177Z Working folder: C:\a\14076a40
4. 2015-08-03T21:26:28.4481177Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\a\14076a40\SpecByExample\SpecByExample.Specs\bin\Debug\SpecByExample.Specs.dll" "C:\a\14076a40\SpecByExample\SpecByExample.Specs\bin\Debug\SpecByExample.Specs.dll.config" "C:\a\14076a40\SpecByExample\SpecByExample.Specs\obj\Debug\SpecByExample.Specs.dll" /logger:trx
5. 2015-08-03T21:26:29.4654019Z Microsoft (R) Test Execution Command Line Tool Version 14.0.23107.0
6. 2015-08-03T21:26:29.4664025Z Copyright (c) Microsoft Corporation. All rights reserved.
7. 2015-08-03T21:26:29.4664025Z Starting test execution, please wait...
8. 2015-08-03T21:26:29.7564024Z Warning: Unable to load types from the test source 'C:\a\14076a40\SpecByExample\SpecByExample.Specs\obj\Debug\SpecByExample.Specs.dll'. Some or all of the tests in this source may not be discovered. If you are running unit tests for Windows Store apps and referencing custom WinMD type then please visit http://go.microsoft.com/fwlink/?LinkId=238340 for more info. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'SpecResults, Version=0.1.0.7, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
9. 2015-08-03T21:26:29.7574029Z .
10. 2015-08-03T21:26:40.6922424Z Passed TheUserVisitsTheWebsite
11. 2015-08-03T21:26:41.3214684Z ##[error]Error: An exception occurred while invoking executor 'executor://mstestadapter/v1': Could not load file or assembly 'TechTalk.SpecFlow, Version=1.9.0.77, Culture=neutral, PublicKeyToken=0778194805d6db41' or one of its dependencies. The system cannot find the file specified.
12. 2015-08-03T21:26:41.3224680Z ##[error]
13. 2015-08-03T21:26:41.4936886Z Results File: C:\a\14076a40\TestResults\buildguest_TASKAGENT-0007 2015-08-03 21_26_40.trx
14. 2015-08-03T21:26:41.4956876Z Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
15. 2015-08-03T21:26:41.4966881Z ##[error]Test Run Failed.
16. 2015-08-03T21:26:41.4966881Z Test execution time: 11.8787 Seconds
17. 2015-08-03T21:26:41.7581094Z ##[error]VSTest Test Run failed with exit code: 1
18. 2015-08-03T21:26:41.7741099Z Publishing Test Results...
2015-08-03T21:26:43.0372165Z Test results remaining: 1
UPDATE: I disabled the SpecFlow test and added the simplest AboutController test possible (from file new project MVC in Visual Studio). I still get a 'could not load assembly problem. SpecByExample.Web is my MVC project in my solution.
2015-08-07T16:16:56.7251975Z ##[error]Error Message:
2015-08-07T16:16:56.7261978Z ##[error] Test method
SpecByExample.Web.Tests.Controllers.HomeControllerTests.AboutTest
threw exception: 2015-08-07T16:16:56.7261978Z
[error]System.IO.FileNotFoundException: Could not load file or assembly 'SpecByExample.Web, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=null' or one of its dependencies. The system cannot
find the file specified.WRN: Assembly binding logging is turned OFF.
2015-08-07T16:16:56.7271982Z ##[error]To enable assembly bind failure
logging, set the registry value
[HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
2015-08-07T16:16:56.7271982Z ##[error]Note: There is some performance
penalty associated with assembly bind failure logging.
2015-08-07T16:16:56.7281979Z ##[error]To turn this feature off, remove
the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
2015-08-07T16:16:56.7281979Z ##[error] 2015-08-07T16:16:56.7291971Z
[error]Stack Trace: 2015-08-07T16:16:56.7291971Z ##[error] at SpecByExample.Web.Tests.Controllers.HomeControllerTests.AboutTest()
2015-08-07T16:16:56.7831986Z Results File:
C:\a\14076a40\TestResults\buildguest_TASKAGENT-0007 2015-08-07
16_16_56.trx 2015-08-07T16:16:56.7861977Z Attachments:
2015-08-07T16:16:56.7861977Z ##[error]Test Run Failed.
Here's a screenshot of my build definition:

In Run unit test step. click advance and give path to your packages folder. (if you are using nuget it will download all dependencies there) give it a try and let me know how it goes.

Related

Failed to locate the code coverage command line tool , MSBuild with SonarQube

Getting the following error's once build is triggered from the build definition.
3:46:16 AM Failed to locate the code coverage command line tool Post-processing failed. Exit code: 1
Exception Message: TF270015: 'MSBuild.SonarQube.Runner.exe' returned an unexpected exit code. Expected '0'; actual '1'. See the build logs for more details. (type UnexpectedExitCodeException)
Exception Stack Trace:
at System.Activities.Statements.Throw.Execute(CodeActivityContext context)
at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)
I am using SonarQube 5.3, installed in the build server. I have .net framework installed in build server not the visual studio .
BuildDefinition where I am referencing SonarQube.
Please help me here !!!!
The code coverage tools are installed with Visual Studio not the .Net Framework, so if you want code coverage you'll need to install VS on the build agent.
Depending on the version of Visual Studio you are using, you may not need an additional license for build agent. See the discussion Is a separate Visual Studio license required for a build machine? get more detail.

Unit tests pass with "dotnet test" but fail when run from Visual Studio 2017 Test Explorer

My XUnit 2.2 tests all pass when run from the command line with dotnet test. When running them from Visual Studio 2017's Test Explorer some of the tests fail with Assembly Binding errors.
My test project targets .Net 4.6.2 and references an ASP.Net Core 1.1 application also targeting .Net 4.6.2. The unit tests were working fine in VS 2015 prior to upgrading to the .NET Core 1.1 release tooling and Visual Studio 2017.
The workaround was to create an app.config file for the test project and add all of the needed binding redirects to it via trial and error. I'm not sure why I have to do this, though.
It appears that when using dotnet test the runtime uses the newer assembly versions without complaint. When running from Visual Studio, the redirects are necessary.
Why is there a difference in the assembly binding behavior between these two methods of running the tests?
Sample assembly binding error:
The operation failed.
Bind result: hr = 0x80131040. No description available.
Assembly manager loaded from: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Running under executable C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\PROFESSIONAL\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\TESTWINDOW\vstest.executionengine.x86.exe
--- A detailed error log follows.
=== Pre-bind state information ===
LOG: DisplayName = Microsoft.Extensions.Configuration.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
(Fully-specified)
LOG: Appbase = file:///C:/xx/xx/xx.Tests/bin/Debug/net462
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = C:\Users\xxxxx\AppData\Local\Temp\a1ec4d3c-04ff-4fa0-9e56-129e799dd870
LOG: AppName = a1ec4d3c-04ff-4fa0-9e56-129e799dd870
Calling assembly : Serilog.Settings.Configuration, Version=2.2.0.0, Culture=neutral, PublicKeyToken=24c2f752a8e58a10.
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\PROFESSIONAL\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\TESTWINDOW\vstest.executionengine.x86.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Microsoft.Extensions.Configuration.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/xx/xx/xx.Tests/bin/Debug/net462/Microsoft.Extensions.Configuration.Abstractions.DLL.
LOG: Assembly download was successful. Attempting setup of file: C:\xx\xx\xx.Tests\bin\Debug\net462\Microsoft.Extensions.Configuration.Abstractions.dll
LOG: Entering download cache setup phase.
LOG: Assembly Name is: Microsoft.Extensions.Configuration.Abstractions, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60
WRN: Comparing the assembly name resulted in the mismatch: Minor Version
ERR: The assembly reference did not match the assembly definition found.
ERR: Setup failed with hr = 0x80131040.
ERR: Failed to complete setup of assembly (hr = 0x80131040). Probing terminated.
And the redirect needed to solve it:
<dependentAssembly>
<assemblyIdentity name="Microsoft.Extensions.Configuration.Abstractions" culture="neutral" publicKeyToken="adb9793829ddae60" />
<bindingRedirect oldVersion="0.0.0.0-1.0.0.0" newVersion="1.1.1.0" />"
</dependentAssembly>
It turns out this issue is being tracked at https://github.com/Microsoft/vstest/issues/428
The workaround is to add the following two items to the test project's csproj file:
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
This eliminates the need for manually adding binding redirects.

VSTS Build fails running NUnit tests

When building a visual studio solution using visual studio team services it runs the unit tests, written in nunit 3 using the nunit 3 NUnit3TestAdapter it starts running the tests.
The error it gives is: Error: Illegal characters in path.
The complete output as follows
2016-02-11T17:02:40.8514411Z Executing the powershell script: C:\BuildAgent\tasks\VSTest\1.0.29\VSTest.ps1
2016-02-11T17:02:41.1483135Z ##[debug]Calling Invoke-VSTest for all test assemblies
2016-02-11T17:02:41.2108131Z Working folder: C:\BuildAgent\_work\5
2016-02-11T17:02:41.2108131Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll" /logger:trx /TestAdapterPath:"C:\BuildAgent\_work\5\s"
2016-02-11T17:02:41.5240692Z Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
2016-02-11T17:02:41.5240692Z Copyright (c) Microsoft Corporation. All rights reserved.
2016-02-11T17:02:42.1326930Z Starting test execution, please wait...
2016-02-11T17:02:46.4921857Z Information: NUnit Adapter 3.0.8.0 executing tests is started
2016-02-11T17:02:46.5077303Z Information: Running all tests in C:\BuildAgent\_work\5\s\xxxxxxxxxxTests\bin\Release\xxxxxxxxxxTests.dll
2016-02-11T17:02:46.6327336Z Information: NUnit Adapter 3.0.8.0 executing tests is finished
2016-02-11T17:02:46.6327336Z ##[error]Error: Exception System.ArgumentException, Exception thrown executing tests
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.6327336Z ##[error]Error: Illegal characters in path.
2016-02-11T17:02:46.6327336Z ##[error]
2016-02-11T17:02:46.7108474Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2016-02-11T17:02:46.8358887Z ##[error]VSTest Test Run failed with exit code: 1
2016-02-11T17:02:46.8514965Z ##[warning]No results found to publish.
We have an on premise build agent but I don't think this should make an difference.
Any advice you can give it appreciated.
FWIW, I switched on verbose logging by following the Tips and Tricks and got this:
2016-02-23T14:45:49.5620657Z Information: Running all tests in C:\TFS2015\_work\3\s\BlaBlaTests\bin\Debug\BlaBlaTests.dll
2016-02-23T14:45:49.6088663Z ##[error]Error: Exception thrown executing tests
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z ##[error]Error: System.ArgumentException: Illegal characters in path.
2016-02-23T14:45:49.6088663Z ##[error] at System.IO.Path.CheckInvalidPathChars(String path)
2016-02-23T14:45:49.6088663Z ##[error] at System.IO.Path.Combine(String path1, String path2)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.ProcessAddinsFile(DirectoryInfo baseDir, String fileName)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.FindExtensionsInDirectory(DirectoryInfo startDir)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ExtensionService.StartService()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.Services.ServiceManager.StartServices()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.TestEngine.Initialize()
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.Engine.TestEngine.GetRunner(TestPackage package)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnitTestAdapter.GetRunnerFor(String assemblyName)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyName, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error] at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
2016-02-23T14:45:49.6088663Z ##[error]
2016-02-23T14:45:49.6088663Z Information: NUnit Adapter 3.0.8.0 executing tests is finished
which correlates with %LOCALAPPDATA%\NUnit\Nunit30Settings.xml being accessed but might not be related. ExtensionService is here.
Issues #117 and #118 are related to this with a fix as provided by RobertK66:
If you install the Version 3.0.8.0 as a Nuget package then there is a
corrupted (not empty!) “ignore.addins” file installed in the
packages\NUnit3TestAdapter.3.0.8-ctp-8\lib folder. Earlier Versions
did install a 0byte empty file!
After removing the content of this file in the
/packages/NUnit3TestAdapter.3.0.8-ctp-8\lib folder and
deleting the cached version of the adapter under
\AppData\Local\Temp\VisualStudioTestExplorerExtensions...
everything works as expected!
I met this issue when I use NUnit3 Test Adapter Nuget Package during the build. And I resolved it by installing NUnit3 Test Adapter and upload all the installed files to Version Control and use it as the Test Adapter.
Since you are using on premise build agent. You can easily install the NUnit3 Test Adapter on the build agent and use "/UseVsixExtensions:true" argument as the error message indicates.

Unity Di container not working with 64 bit build

I am using Unity for in WCF service to load component.
I am referring below mention article.
https://msdn.microsoft.com/en-us/library/vstudio/hh323725(v=vs.100).aspx
Service is working fine when i build service in visual studio with build option option any CPU.
As one third party component required 64 bit specific build. So i selected 64 bit build option.
I have downloaded Unity code and build for 64 bit but it is also not working.
I am getting below mention exception.
Could not load file or assembly 'Common.Unity' or one of its dependencies. An attempt was made to load a program with an incorrect format.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.BadImageFormatException: Could not load file or assembly 'Common.Unity' or one of its dependencies. An attempt was made to load a program with an incorrect format.
Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.
Assembly Load Trace: The following information can be helpful to determine why the assembly 'Common.Unity' could not be loaded.
As this is not issue of Unity.
It is issue of IISExprees of VS2012 which is support on 32 bit.
Can't get IIS Express 8 beta to run website as 64-bit process
https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/3254745-allow-for-iis-express-64-bit-to-run-from-visual-st

Issues with running nunit-console.exe with Mono

I could make a dll for NUnit NUnit test under mono, but when I tried to run in with nunit-console.exe I get the following error message.
Runtime Environment -
OS Version: Unix 10.3.0.0
CLR Version: 2.0.50727.1433 ( 2.6.4 (tarball Thu Apr 22 13:24:33 MDT 2010) )
ProcessModel: Default DomainUsage: Single
Execution Runtime: mono-2.0
** (/Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/nunit-console.exe:36800): WARNING **: The following assembly referenced from /private/var/folders/m4/m4u1hmP+FHOQaiZbHj1UCk+++TI/-Tmp-/nunit20/ShadowCopyCache/36800_634111616836311880/Tests_-22323139/assembly/shadow/54274fc2/118e035c_45a94c9e_00000001/mut.dll could not be loaded:
Assembly: nunit.framework (assemblyref_index=1)
Version: 2.5.5.10112
Public Key: 96d09a1eb7f44a77
System error: Invalid argument
** (/Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/nunit-console.exe:36800): WARNING **: Could not load file or assembly 'nunit.framework, Version=2.5.5.10112, Culture=neutral, PublicKeyToken=96d09a1eb7f44a77' or one of its dependencies.
What might be wrong? I have nunit.framework.dll under /Users/smcho/bin/NUnit-2.5.5.10112/bin/net-2.0/framework, but it doesn't seem to know about this.
This is another frustrating point about using NUnit in Mono. As far as I've been able to tell, you need to have all of those missing files (there will be more than just nunit.framework.dll) in the same directory as nunit-console.exe.
There might be a command line option either in mono or nunit-console.exe to specify another directory to search for missing DLLs -- I'm not sure. Barring that, you'll have to copy the missing DLLs into the same folder as nunit-console.exe, or you'll have to cd into the framework folder and use mono ../nunit-console.exe /path/to/my/tests.dll
Once nuget is installed, you can use it to install nunit.console like so:
(This is from my .circlci/config.yml file)
- run: mono --runtime=v4.0 nuget.exe install nunit.console
- run: msbuild MySolution.sln
- run: mono --runtime=v4.0 NUnit.ConsoleRunner.3.12.0/tools/nunit3-console.exe FirstModule.Tests/bin/Debug/FirstModule.Tests.dll
- run: mono --runtime=v4.0 NUnit.ConsoleRunner.3.12.0/tools/nunit3-console.exe SecondModule.Tests/bin/Debug/SecondModule.Tests.dll
The trick is to specify the entire path for the NUnit.ConsoleRunner and to watchout for the - changing into a . by version 3