I have a TFS Release definition that runs several tests as part of the deployment:
When I check the "Rerun Failed Tests" Option in my "Visual Studio Test" step in:
My unit tests no longer run as part of my release. The release fails with the following error in the "Visual Studio Test" step:
2019-02-22T16:21:33.0983821Z ##[error]Error opening response file
'C:\TFS' 2019-02-22T16:21:33.1413767Z ##[error]The test source file
"C:\TFS Agent\ _work\r6\a\Agent_work_temp\ea14a091-36bd-11e9-99f5-51895cf3942e.txt" provided was not found.
I figured since the error started with the following statement:
Error opening response file 'C:\TFS'
That there is some sort of bug with the TFS test runner where the call to the task to execute the tests is not putting the path in quotes. I changed all my TFS build agents to using the following "type" of paths for all builds(I removed all spaces from the folder the TFS agents lived in):
C:\TFSAgent\ _work
And it seems to work now...
Related
The TFS instance I am working on was recently upgraded from TFS 2017 Update 1 to TFS 2018 Update 2, allowing me to change the dotnet task version used in my build definitions from 0.* to 2.*.
In doing so, the dotnet test step no longer works, returning the following error:
MSBUILD : error MSB1008: Only one project can be specified.
Switch: trx
The command it runs is:
C:\Program Files\dotnet\dotnet.exe" test <Agent_WorkFolder>\1\w\3\s\source\MySolution\MyProject.csproj --configuration release --logger trx --logger trx --results-directory <Agent_WorkFolder>\1\w\_temp
The parameters given to the task are:
Paths to projects = **\*Tests*.csproj
Arguments = --configuration $(BuildConfiguration) --logger trx
Reverting the task version back to 0.*, and it runs again. What is causing this error?
The problem is caused by --logger trx being specified in the Arguments to the task. The newer versions of the task automatically adds this switch when executing the dotnet test command as its the output which TFS supports for reading test results. The extra argument results in the switch being given twice, so whilst MSBUILD error is unhelpful, the Switch: trx part gives a clue as to what is the problem.
Removing the switch from the arguments resolved the problem.
I have a Coded UI test program that runs without any error on local Visual Studio (2017). Only one test method is in the program. One feature it has is saving file in Azure Storage container. During build's Test Runs process, Build message says it cannot find any tests. Hoping for any feedback to see if I am making any error in build definition.
The build message says:
No test is available in D:\a\3\s\FullSite1\bin\Debug\FullSite1.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.Edm.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.OData.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.Data.Services.Client.dll D:\a\3\s\FullSite1\bin\Debug\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\FullSite1\bin\Debug\Newtonsoft.Json.dll D:\a\3\s\packages\Microsoft.Azure.KeyVault.Core.1.0.0\lib\net40\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\packages\Microsoft.Azure.KeyVault.Core.1.0.0\lib\portable-net45+wp8+wpa81+win\Microsoft.Azure.KeyVault.Core.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net20\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net35\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net40\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\net45\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\netstandard1.0\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\netstandard1.3\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\portable-net40+sl5+win8+wpa81+wp8\Newtonsoft.Json.dll D:\a\3\s\packages\Newtonsoft.Json.10.0.2\lib\portable-net45+win8+wpa81+wp8\Newtonsoft.Json.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\net45\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\netstandard1.0\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\netstandard1.3\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\win8\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\wp8\Microsoft.WindowsAzure.Storage.dll D:\a\3\s\packages\WindowsAzure.Storage.9.1.1\lib\wpa\Microsoft.WindowsAzure.Storage.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
I am using the following in the build definition:
Process
Agent queue: Hosted VS2017
Get Sources
Select a Source: TFVC
Phase 1
Agent Queue: Hosted VS2017
Demands
Exists: msbuild, visualstudio, vstest
NuGet Restore
Version: 2
Command: restore
Path to solution, packages.config or project.json: **/*.sln
Use packages from NuGet.org
Build solution **.sln
Visual Studio Build
Build solution **\*.sln
Version = 1.*
Solution = **\*.sln
Visual Studio Version = Latest
Test Runs (Visual Studio Test)
Version = 2.*
Select tests using = Test assemblies
Test Assemblies = **\*.dll
!**\*TestAdapter.dll
!**\obj\**
Search Folder = $(System.DefaultWorkingDirectory)
Test mix contains UI Tests
Select test platform using = Version
Update test attachments = Checked
Control Options = Enabled
After comparing the code with another test I found the reason was because CodedUITest.cs file was missing [CodedUITest] after the namespace. I have added it and VSTS now can find the test method.
In the build definition in TFS 2015, I've got a Command Line step that runs the following command:
xunit.console.exe \PathToTests\Tests.dll -xml \PathToResultsFolder\Results.xml
During the build, I can see the tests are being discovered and executing and everything's looking good.
But if I don't check "Continue on error" in the Command Line step, after the tests run and the result XML file has been saved, the step fails with the following error:
Task CmdLine failed. This caused the job to fail. Look at the logs for the task for more details.
But there's actually no error or anything I can see. The tests have run and the XML file has saved properly and is able to published to TFS. And I don't see an error like this if I run the command from the build machine.
Any ideas?
I am setting up new tfs build definition using Web based build definition. I am getting an issue with the unit test cases.
If I run the build without passing any MS Build Arguments, my build is getting done successfully and it's running all test cases included in solution.
But when I pass MS Build arguments, it's not running any unit test cases - just ignoring it and going to next build step. What can be it?
I am building .NET Solution and passing MS BUILD ARGUMENTS AS
(/p:DeployOnBuild=true /P:PackageTempRootDir=\Release /p:DeployIisAppPath="Default Web Site";DefaultPackageFileName=SampleApp.zip /p:OutputPath="$(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)")
Is there any connection between these two (MS Build arguments and Unit test) cases?
I am not getting any error, the build is also successfully passing in both cases. But unit test are not getting executed.
Here is test step output:
2017-02-14T22:35:06.6488286Z Executing the powershell script: E:\agnt\tasks\VSTest\1.0.32\VSTest.ps1
2017-02-14T22:35:07.4248374Z ##[debug]Calling Invoke-VSTest for all test assemblies
2017-02-14T22:35:07.6348369Z Working folder: E:\agnt\_work\39
2017-02-14T22:35:07.6348369Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "E:\agnt\_work\39\s\CaseAPI\Main\packages\Agero.TestRunner.1.0.0\lib\net45\Agero.TestsRunner.NUnit.dll" /TestCaseFilter:"TestCategory!=Ignore" /EnableCodeCoverage /logger:trx
2017-02-14T22:35:07.8428336Z Microsoft (R) Test Execution Command Line Tool Version 14.0.25420.1
2017-02-14T22:35:07.8438332Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-02-14T22:35:07.9678332Z Starting test execution, please wait...
2017-02-14T22:35:08.0108349Z Warning: Using Isolation mode to run the tests as diagnostic data adapters were enabled in the runsettings. Use the /inIsolation parameter to suppress this warning.
2017-02-14T22:35:16.0528901Z Attachments:
2017-02-14T22:35:16.0538895Z E:\agnt\_work\39\TestResults\6eaffd7e-b2ac-4bc4-b356-41a2fda7bad9\csr1_B-TFSBC001WV 2017-02-14 17_35_10.coverage
2017-02-14T22:35:16.2028915Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
2017-02-14T22:35:16.3698948Z ##[warning]No results found to publish.
Any advice/help is appreciated!
As you have changed the OutputPath to $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration), so you also need to specify the test assembly in VS test task to: $(Build.BinariesDirectory)\$(BuildPlatform)\$(BuildConfiguration)\*test*.dll:
Is there any way to activate a test related log at our TFS build server?
If a tests fails, I can't find any information about it (besides the name of the test).
I checked:
The drop folder. There is the normal log, but this log ends after the compilation, so before the tests are checked
The compilation folder at the build server (C:\Builds\8...). There is a folder "Test Results" next to the "bin" and the "src" folder, but the "Test Results" folder is always empty.
The build definition. Couldn't find any option to activate a log regarding tests.
When i check the build in the web view, all I see is:
x/y test(s) passed, z failed, 0 inconclusive
No Code Coverage Results
Other Errors and Warnings
z+1 error(s), 0 warning(s)
Failed z1
Failed z2
...
Failed zn
Test Run Failed.
How can I find further information, why the tests are failing?
We are using:
TFS 2012 + a TFS 2012 build server
Rolling builds
Visual Studio 2013 + Resharper at our local machines
NUnit + MOQ
Rolling builds (no more than 30 minutes) build definition
We got rid of the problem after we upgraded to TFS 2015.
Now I just have to check the web interface, if a build is broken because of a unit test I click at the specific build to open the detail view of it. At the right side there is an area named Test result, where I can click at a link to get to the Test Run page. Under Attachments (1) I got a file of the type Visual Studio Test Results File, which shows me exactly what the problem was.