I am setting up new tfs build definition using Web based build definition. I am getting an issue with the unit test cases.
If I run the build without passing any MS Build Arguments, my build is getting done successfully and it's running all test cases included in solution.
But when I pass MS Build arguments, it's not running any unit test cases - just ignoring it and going to next build step. What can be it?
I am building .NET Solution and passing MS BUILD ARGUMENTS AS
(/p:DeployOnBuild=true /P:PackageTempRootDir=\Release /p:DeployIisAppPath="Default Web Site";DefaultPackageFileName=SampleApp.zip /p:OutputPath="$(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)")
Is there any connection between these two (MS Build arguments and Unit test) cases?
I am not getting any error, the build is also successfully passing in both cases. But unit test are not getting executed.
Here is test step output:
2017-02-14T22:35:06.6488286Z Executing the powershell script: E:\agnt\tasks\VSTest\1.0.32\VSTest.ps1
2017-02-14T22:35:07.4248374Z ##[debug]Calling Invoke-VSTest for all test assemblies
2017-02-14T22:35:07.6348369Z Working folder: E:\agnt\_work\39
2017-02-14T22:35:07.6348369Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "E:\agnt\_work\39\s\CaseAPI\Main\packages\Agero.TestRunner.1.0.0\lib\net45\Agero.TestsRunner.NUnit.dll" /TestCaseFilter:"TestCategory!=Ignore" /EnableCodeCoverage /logger:trx
2017-02-14T22:35:07.8428336Z Microsoft (R) Test Execution Command Line Tool Version 14.0.25420.1
2017-02-14T22:35:07.8438332Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-02-14T22:35:07.9678332Z Starting test execution, please wait...
2017-02-14T22:35:08.0108349Z Warning: Using Isolation mode to run the tests as diagnostic data adapters were enabled in the runsettings. Use the /inIsolation parameter to suppress this warning.
2017-02-14T22:35:16.0528901Z Attachments:
2017-02-14T22:35:16.0538895Z E:\agnt\_work\39\TestResults\6eaffd7e-b2ac-4bc4-b356-41a2fda7bad9\csr1_B-TFSBC001WV 2017-02-14 17_35_10.coverage
2017-02-14T22:35:16.2028915Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2017-02-14T22:35:16.3698948Z ##[warning]No results found to publish.
Any advice/help is appreciated!
As you have changed the OutputPath to $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration), so you also need to specify the test assembly in VS test task to: $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)\*test*.dll:
Related
I'm trying to run google test using vstest.console.exe.
I'm wondering if there is a way to debug why vstest.console.exe is not finding google tests.
Here is some output running locally:
C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug>vstest.console.exe Infrastructure.Native.Tests.exe /TestAdapterPath:c:\packages\GoogleTestAdapter.0.18.0\build\_common -- RunConfiguration.TreatNoTestsAsError=true /Platform:x64 /Configuration:Debug
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220626-01 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug\Infrastructure.Native.Tests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug>Infrastructure.Native.Tests.exe --gtest_list_tests
Running main() from C:\Development\Source\Bentley.Reliability\packages\gmock.1.11.0\lib\native\src\gtest\src\gtest_main.cc
RestoreDatabaseTests.
RestoreDatabaseTest
CreateSnapshotTest
RestoreDatabaseBlankServerNameTest
RestoreDatabaseBlankDatabaseNameTest
RestoreDatabaseBlankUserNameTest
RestoreDatabaseIncorrectPasswordTest
TestExceptions.
InvalidActivityExceptionTest
InvalidConfigurationExceptionTest
InvalidSessionExceptionTest
PlatformNotSupportedExceptionTest
OQGenericObjectTest
TestDynamicCast_IncorrectExceptionType_Returns_Null
TestConfigurationProxy.
TestBasics
TestUnmanagedSettings
TestKernelExecutionContextThreadStorage.
TestKernelExecutionContextThreadLocalStorage
TestSqlUomListLoader.
TestLoadUomsFromDatabase_Success
TestMockAdapter_LoadUomsFromDatabase_Success
TestMockAdapter_LoadUomsFromDatabase_FetchFails
TestMockAdapter_LoadUomsFromDatabase_FetchFails_ValidateMessage
TestUOMListInterop.
CreateUOMList_throws_on_null_loggingProxy
CreateUOMList_throws_on_null_sqlAdapter
oUOM_GetUOMS_asserts_on_null_kernelExecutionContext
oUOM_GetUOMS_asserts_on_null_uomlist
oUOM_GetUOMS_no_asserts_when_list_defined
MockConfigurationTests.
TestGetValueFromFakeCollection
TestGetValueFromEnvironment
TestGetValueOrderPrecidence
TestRemovingFakeValue
I have a TFS Release definition that runs several tests as part of the deployment:
When I check the "Rerun Failed Tests" Option in my "Visual Studio Test" step in:
My unit tests no longer run as part of my release. The release fails with the following error in the "Visual Studio Test" step:
2019-02-22T16:21:33.0983821Z ##[error]Error opening response file
'C:\TFS' 2019-02-22T16:21:33.1413767Z ##[error]The test source file
"C:\TFS Agent\ _work\r6\a\Agent_work_temp\ea14a091-36bd-11e9-99f5-51895cf3942e.txt" provided was not found.
I figured since the error started with the following statement:
Error opening response file 'C:\TFS'
That there is some sort of bug with the TFS test runner where the call to the task to execute the tests is not putting the path in quotes. I changed all my TFS build agents to using the following "type" of paths for all builds(I removed all spaces from the folder the TFS agents lived in):
C:\TFSAgent\ _work
And it seems to work now...
I configured xunit parser task under my project's Bamboo plan, I ran my plan:
Starting task 'Xunit' of type 'com.wwwlicious.xunit.xunit:xunitParser' Failing task since test cases were expected but none were found.
It dawns on my xunit parser only parses test result produced somehow from prev steps?
I tried to add a script task to run xunit tests manually: https://xunit.github.io/docs/getting-started/netcore/cmdline
C:\SomeApplication>dotnet test -l:trx;LogFileName=SomeApplTests.xml
Build completed.
Test run for C:\SomeApplication\bin\Debug\netcoreapp2.0\YI.LS.Test.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Total tests: 20. Passed: 20. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.3114 Seconds
So, okay - I can configure a Bamboo Script task to run "dotnet test", it'd produce SomeApplTests.xml under "TestResult" folder.
Now the challenge is: how to feed it to xunit parser? Also trx logger also no good, prolly need xunit logger https://github.com/spekt/xunit.testlogger
Thanks
References:
https://marketplace.atlassian.com/apps/1216522/xunit-net-parser?hosting=server&tab=support
https://github.com/wwwlicious/wwwlicious.bamboo.xunit/issues/6
How to publish results using dotnet test command
https://github.com/spekt/xunit.testlogger
Answer is found:
You need install in test project "XunitXmlTestLogger" (https://github.com/spekt/xunit.testlogger)
Then configure a Script task:
dotnet test -l:"xunit;LogFileName=Test.xml"
You need set Working Directory to point at your test project, for example: MySoln\MyTestProj
Then configure your xUnit.net Parser task in Bamboo, with xUnit Test Results File/Directory to point at: MySoln\MyTestProj*.xml (Default is: artifacts/**/*.xml)
https://github.com/wwwlicious/wwwlicious.bamboo.xunit/issues/6
I have a Coded UI test program that runs without any error on local Visual Studio (2017). Only one test method is in the program. One feature it has is saving file in Azure Storage container. During build's Test Runs process, Build message says it cannot find any tests. Hoping for any feedback to see if I am making any error in build definition.
The build message says:
No test is available in D:\a\3\s\FullSite1\bin\Debug\FullSite1.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.Edm.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.OData.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.Services.Client.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\FullSite1\bin\Debug\Newtonsoft.Json.dll D:\a\3\s\packages\Microsoft.Azure.KeyVault.Core.1.0.0\lib\net40\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\packages\Microsoft.Azure.KeyVault.Core.1.0.0\lib\portable-net45+wp8+wpa81+win\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net20\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net35\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net40\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net45\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\netstandard1.0\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\netstandard1.3\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\portable-net40+sl5+win8+wpa81+wp8\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\portable-net45+win8+wpa81+wp8\Newtonsoft.Json.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\net45\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\netstandard1.0\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\netstandard1.3\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\win8\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\wp8\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\wpa\Microsoft.WindowsAzure.Storage.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
I am using the following in the build definition:
Process
Agent queue: Hosted VS2017
Get Sources
Select a Source: TFVC
Phase 1
Agent Queue: Hosted VS2017
Demands
Exists: msbuild, visualstudio, vstest
NuGet Restore
Version: 2
Command: restore
Path to solution, packages.config or project.json: **/*.sln
Use packages from NuGet.org
Build solution **.sln
Visual Studio Build
Build solution **\*.sln
Version = 1.*
Solution = **\*.sln
Visual Studio Version = Latest
Test Runs (Visual Studio Test)
Version = 2.*
Select tests using = Test assemblies
Test Assemblies = **\*.dll
!**\*TestAdapter.dll
!**\obj\**
Search Folder = $(System.DefaultWorkingDirectory)
Test mix contains UI Tests
Select test platform using = Version
Update test attachments = Checked
Control Options = Enabled
After comparing the code with another test I found the reason was because CodedUITest.cs file was missing [CodedUITest] after the namespace. I have added it and VSTS now can find the test method.
I have executed my tests using the command:
dotnet test -l trx
But when I saw the my test results are not appearing in the build summary:
Some hint show to make it visible? Within TFS2015, and I can't use the current build step for running tests as they do not work with dotnet core apps
This is the output of the tests in my logs:
2017-06-01T14:56:08.3158317Z Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
2017-06-01T14:56:08.3158317Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-06-01T14:56:08.4228414Z Starting test execution, please wait...
2017-06-01T14:56:11.4560909Z Results File: D:\APPS\NVSDeploymentAgent\_work\12\s\tests\Tests\TestResults\2017-06-01_16_56_11.trx
To make the generated trx files available as test results in VSTS/TFS, use the "Publish Test Results" task: