I have executed my tests using the command:
dotnet test -l trx
But when I saw the my test results are not appearing in the build summary:
Some hint show to make it visible? Within TFS2015, and I can't use the current build step for running tests as they do not work with dotnet core apps
This is the output of the tests in my logs:
2017-06-01T14:56:08.3158317Z Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
2017-06-01T14:56:08.3158317Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-06-01T14:56:08.4228414Z Starting test execution, please wait...
2017-06-01T14:56:11.4560909Z Results File: D:\APPS\NVSDeploymentAgent\_work\12\s\tests\Tests\TestResults\2017-06-01_16_56_11.trx
To make the generated trx files available as test results in VSTS/TFS, use the "Publish Test Results" task:
Related
I'm trying to run google test using vstest.console.exe.
I'm wondering if there is a way to debug why vstest.console.exe is not finding google tests.
Here is some output running locally:
C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug>vstest.console.exe Infrastructure.Native.Tests.exe /TestAdapterPath:c:\packages\GoogleTestAdapter.0.18.0\build\_common -- RunConfiguration.TreatNoTestsAsError=true /Platform:x64 /Configuration:Debug
Microsoft (R) Test Execution Command Line Tool Version 17.3.0-preview-20220626-01 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug\Infrastructure.Native.Tests.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
C:\Development\Source\Bentley.Reliability\oq.framework\Tests\Infrastructure.Native.Tests\bin\x64\Debug>Infrastructure.Native.Tests.exe --gtest_list_tests
Running main() from C:\Development\Source\Bentley.Reliability\packages\gmock.1.11.0\lib\native\src\gtest\src\gtest_main.cc
RestoreDatabaseTests.
RestoreDatabaseTest
CreateSnapshotTest
RestoreDatabaseBlankServerNameTest
RestoreDatabaseBlankDatabaseNameTest
RestoreDatabaseBlankUserNameTest
RestoreDatabaseIncorrectPasswordTest
TestExceptions.
InvalidActivityExceptionTest
InvalidConfigurationExceptionTest
InvalidSessionExceptionTest
PlatformNotSupportedExceptionTest
OQGenericObjectTest
TestDynamicCast_IncorrectExceptionType_Returns_Null
TestConfigurationProxy.
TestBasics
TestUnmanagedSettings
TestKernelExecutionContextThreadStorage.
TestKernelExecutionContextThreadLocalStorage
TestSqlUomListLoader.
TestLoadUomsFromDatabase_Success
TestMockAdapter_LoadUomsFromDatabase_Success
TestMockAdapter_LoadUomsFromDatabase_FetchFails
TestMockAdapter_LoadUomsFromDatabase_FetchFails_ValidateMessage
TestUOMListInterop.
CreateUOMList_throws_on_null_loggingProxy
CreateUOMList_throws_on_null_sqlAdapter
oUOM_GetUOMS_asserts_on_null_kernelExecutionContext
oUOM_GetUOMS_asserts_on_null_uomlist
oUOM_GetUOMS_no_asserts_when_list_defined
MockConfigurationTests.
TestGetValueFromFakeCollection
TestGetValueFromEnvironment
TestGetValueOrderPrecidence
TestRemovingFakeValue
I am having an error in the dotnet Core step
I am trying to integrate my selenium test to Azure devops
This the logs of the dot net step
2020-10-16T03:11:25.7523865Z ##[section]Starting: Run Selenium
2020-10-16T03:11:25.8123424Z ==============================================================================
2020-10-16T03:11:25.8124047Z Task : .NET Core
2020-10-16T03:11:25.8124510Z Description : Build, test, package, or publish a dotnet application, or run a custom dotnet command
2020-10-16T03:11:25.8124816Z Version : 2.175.0
2020-10-16T03:11:25.8125178Z Author : Microsoft Corporation
2020-10-16T03:11:25.8125636Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
2020-10-16T03:11:25.8126003Z ==============================================================================
2020-10-16T03:11:27.2023831Z [command]C:\windows\system32\chcp.com 65001
2020-10-16T03:11:27.2216959Z Active code page: 65001
2020-10-16T03:11:27.2376255Z Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
2020-10-16T03:11:27.3059263Z [command]"C:\Program Files\dotnet\dotnet.exe" vstest "D:\a\r1\a\_SeleniumDemo-ASP.NET Core-CI (5)\drop\Demo2.dll" --logger:trxlogfilename=TEST.xml
2020-10-16T03:11:34.8402885Z
2020-10-16T03:11:34.9823477Z Welcome to .NET Core 3.1!
2020-10-16T03:11:34.9824051Z ---------------------
2020-10-16T03:11:34.9824560Z SDK Version: 3.1.402
2020-10-16T03:11:34.9824775Z
2020-10-16T03:11:34.9836054Z Telemetry
2020-10-16T03:11:34.9836648Z ---------
2020-10-16T03:11:34.9838913Z The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
2020-10-16T03:11:34.9840343Z
2020-10-16T03:11:34.9840938Z Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
2020-10-16T03:11:34.9841242Z
2020-10-16T03:11:34.9841449Z ----------------
2020-10-16T03:11:34.9841756Z Explore documentation: https://aka.ms/dotnet-docs
2020-10-16T03:11:34.9842152Z Report issues and find source on GitHub: https://github.com/dotnet/core
2020-10-16T03:11:34.9842572Z Find out what's new: https://aka.ms/dotnet-whats-new
2020-10-16T03:11:34.9842993Z Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
2020-10-16T03:11:34.9843479Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
2020-10-16T03:11:34.9843894Z Write your first app: https://aka.ms/first-net-core-app
2020-10-16T03:11:34.9844375Z --------------------------------------------------------------------------------------
2020-10-16T03:11:39.1360071Z Microsoft (R) Test Execution Command Line Tool Version 16.7.0
2020-10-16T03:11:39.1368990Z Copyright (c) Microsoft Corporation. All rights reserved.
2020-10-16T03:11:39.1369374Z
2020-10-16T03:11:40.1936906Z The Test Logger URI 'trxlogfilename=TEST.xml' is not valid. The Test Logger will be ignored.
2020-10-16T03:11:40.7284653Z ##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
2020-10-16T03:11:40.8737953Z Info: Azure Pipelines hosted agents have been updated to contain .Net Core 3.x (3.1) SDK/Runtime along with 2.1. Unless you have locked down a SDK version for your project(s), 3.x SDK might be picked up which might have breaking behavior as compared to previous versions.
2020-10-16T03:11:40.8738674Z Some commonly encountered changes are:
2020-10-16T03:11:40.8740232Z If you're using `Publish` command with -o or --Output argument, you will see that the output folder is now being created at root directory rather than Project File's directory. To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
2020-10-16T03:11:40.8747693Z ##[error]Dotnet command failed with non-zero exit code on the following projects : D:\a\r1\a\_SeleniumDemo-ASP.NET Core-CI (5)\drop\Demo2.dll
2020-10-16T03:11:40.8951889Z ##[section]Finishing: Run Selenium
I tried to run my code using my local machine and got not error. Please help me with Azure integration
I can reproduce this issue with the same configuration, and then I change the argument to --logger:"trx;Logfilename=TEST.trx" and it works.
Result:
I configured xunit parser task under my project's Bamboo plan, I ran my plan:
Starting task 'Xunit' of type 'com.wwwlicious.xunit.xunit:xunitParser' Failing task since test cases were expected but none were found.
It dawns on my xunit parser only parses test result produced somehow from prev steps?
I tried to add a script task to run xunit tests manually: https://xunit.github.io/docs/getting-started/netcore/cmdline
C:\SomeApplication>dotnet test -l:trx;LogFileName=SomeApplTests.xml
Build completed.
Test run for C:\SomeApplication\bin\Debug\netcoreapp2.0\YI.LS.Test.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Total tests: 20. Passed: 20. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.3114 Seconds
So, okay - I can configure a Bamboo Script task to run "dotnet test", it'd produce SomeApplTests.xml under "TestResult" folder.
Now the challenge is: how to feed it to xunit parser? Also trx logger also no good, prolly need xunit logger https://github.com/spekt/xunit.testlogger
Thanks
References:
https://marketplace.atlassian.com/apps/1216522/xunit-net-parser?hosting=server&tab=support
https://github.com/wwwlicious/wwwlicious.bamboo.xunit/issues/6
How to publish results using dotnet test command
https://github.com/spekt/xunit.testlogger
Answer is found:
You need install in test project "XunitXmlTestLogger" (https://github.com/spekt/xunit.testlogger)
Then configure a Script task:
dotnet test -l:"xunit;LogFileName=Test.xml"
You need set Working Directory to point at your test project, for example: MySoln\MyTestProj
Then configure your xUnit.net Parser task in Bamboo, with xUnit Test Results File/Directory to point at: MySoln\MyTestProj*.xml (Default is: artifacts/**/*.xml)
https://github.com/wwwlicious/wwwlicious.bamboo.xunit/issues/6
The TFS instance I am working on was recently upgraded from TFS 2017 Update 1 to TFS 2018 Update 2, allowing me to change the dotnet task version used in my build definitions from 0.* to 2.*.
In doing so, the dotnet test step no longer works, returning the following error:
MSBUILD : error MSB1008: Only one project can be specified.
Switch: trx
The command it runs is:
C:\Program Files\dotnet\dotnet.exe" test <Agent_WorkFolder>\1\w\3\s\source\MySolution\MyProject.csproj --configuration release --logger trx --logger trx --results-directory <Agent_WorkFolder>\1\w\_temp
The parameters given to the task are:
Paths to projects = **\*Tests*.csproj
Arguments = --configuration $(BuildConfiguration) --logger trx
Reverting the task version back to 0.*, and it runs again. What is causing this error?
The problem is caused by --logger trx being specified in the Arguments to the task. The newer versions of the task automatically adds this switch when executing the dotnet test command as its the output which TFS supports for reading test results. The extra argument results in the switch being given twice, so whilst MSBUILD error is unhelpful, the Switch: trx part gives a clue as to what is the problem.
Removing the switch from the arguments resolved the problem.
I am setting up new tfs build definition using Web based build definition. I am getting an issue with the unit test cases.
If I run the build without passing any MS Build Arguments, my build is getting done successfully and it's running all test cases included in solution.
But when I pass MS Build arguments, it's not running any unit test cases - just ignoring it and going to next build step. What can be it?
I am building .NET Solution and passing MS BUILD ARGUMENTS AS
(/p:DeployOnBuild=true /P:PackageTempRootDir=\Release /p:DeployIisAppPath="Default Web Site";DefaultPackageFileName=SampleApp.zip /p:OutputPath="$(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)")
Is there any connection between these two (MS Build arguments and Unit test) cases?
I am not getting any error, the build is also successfully passing in both cases. But unit test are not getting executed.
Here is test step output:
2017-02-14T22:35:06.6488286Z Executing the powershell script: E:\agnt\tasks\VSTest\1.0.32\VSTest.ps1
2017-02-14T22:35:07.4248374Z ##[debug]Calling Invoke-VSTest for all test assemblies
2017-02-14T22:35:07.6348369Z Working folder: E:\agnt\_work\39
2017-02-14T22:35:07.6348369Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "E:\agnt\_work\39\s\CaseAPI\Main\packages\Agero.TestRunner.1.0.0\lib\net45\Agero.TestsRunner.NUnit.dll" /TestCaseFilter:"TestCategory!=Ignore" /EnableCodeCoverage /logger:trx
2017-02-14T22:35:07.8428336Z Microsoft (R) Test Execution Command Line Tool Version 14.0.25420.1
2017-02-14T22:35:07.8438332Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-02-14T22:35:07.9678332Z Starting test execution, please wait...
2017-02-14T22:35:08.0108349Z Warning: Using Isolation mode to run the tests as diagnostic data adapters were enabled in the runsettings. Use the /inIsolation parameter to suppress this warning.
2017-02-14T22:35:16.0528901Z Attachments:
2017-02-14T22:35:16.0538895Z E:\agnt\_work\39\TestResults\6eaffd7e-b2ac-4bc4-b356-41a2fda7bad9\csr1_B-TFSBC001WV 2017-02-14 17_35_10.coverage
2017-02-14T22:35:16.2028915Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2017-02-14T22:35:16.3698948Z ##[warning]No results found to publish.
Any advice/help is appreciated!
As you have changed the OutputPath to $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration), so you also need to specify the test assembly in VS test task to: $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)\*test*.dll: