I am using Azure DevOps to run some XUnit tests of an Asp.Net Core application:
- task: DotNetCoreCLI#2
displayName: 'Test'
inputs:
command: test
projects: '**/*[Tt]est/*.csproj'
arguments: '--configuration $(buildConfiguration)'
The task succeeds but I get two warnings:
[warning] Invalid results file. Make sure the result format of the file '/home/vsts/work/_temp/_fv-az592_2019-04-09_21_14_05.trx' matches 'VSTest' test results format.
[warning] Invalid results file. Make sure the result format of the file '/home/vsts/work/_temp/_fv-az592_2019-04-09_21_14_10.trx' matches 'VSTest' test results format.
What am I missing?
From github:
warning is as per our design of task since we wants to warn customer
if he is trying to use PTR task without any results, but i agree with
you about more apt warning message and we will correct it in upcoming
release.
Also try yo use VSTest task instead
Related
This is what I have working in place at the moment on the pipeline.
- task: DotNetCoreCLI#2
displayName: Run API Scenarios
inputs:
command: 'test'
projects: '**/*Myproject.csproj'
arguments: '--filter Category=Portfolio'
testRunTitle: 'API Test Execution'
Although I don't think there's anything specific for tests themselves, there is:
- task: <name of task>
retryCountOnTaskFailure: <max number of retries>
Link to similar question and answer
Although, you may want to add some retry mechanism in your tests if it's a flaky API test.
I am trying to generate code coverage for one of my solutions in an Azure DevOps build pipeline. Whilst I have the results of my tests appearing upon build completion along with the Cobertura code coverage report, there are a lot of files and namespaces I would like to exclude from the code coverage report as they are not testable units of code (e.g. models or database migrations).
I understand that I can utilise the coverlet.runsettings.xml file to do namespace exclusions but this does not appear to be working.
My runsettings file is setup inside one of the test projects that is run and is setup as follows:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="XPlat code coverage">
<Configuration>
<Exclude>[MyDatabase.Migrations.*]*,[*]MyDatabase.Migrations*</Exclude>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I'm trying to exclude all files that sit under the MyDatabase.Migrations namespace but the syntax I have used above doesn't seem to have any effect and I still see the files beneath this namespace shown in final code coverage report in Devops.
My pipeline is setup as follows:
steps:
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '$(solution)'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'Tests'
inputs:
command: 'test'
projects: |
**\MyDatabase.Tests
**\MySearch.Common.Tests
**\MySearchService.Tests
**\MyConfiguration.Tests
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage"'
publishTestResults: true
# ReportGenerator extension to combine code coverage outputs into one
- task: reportgenerator#5
inputs:
reports: '$(Agent.WorkFolder)/**/coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)/CoverageResults'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)/CoverageResults/Cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)/CoverageResults'
I am aware I can use the [ExcludeFromCodeCoverage] tag but with things that are auto-generated like migrations this could get pretty messy having to go in and modify them every time a new one is generated.
I would like the migrations folder completely omitted from even appearing in the generated report.
Any help with this issue would be greatly appreciated.
The VSTestIntegration documentation indicates you will need to include the custom runsettings file as an agrument:
This runsettings file can easily be provided using command line option as given : dotnet test --collect:"XPlat Code Coverage" --settings coverlet.runsettings
Try the following:
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" --settings [path to tests project]\coverlet.runsettings.xml'
Unknown lifecycle phase "mvn". You must specify a valid lifecycle phase or a goal in the format : or :[:]:. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy.
The Maven command in Jenkins is set incorrectly.
Use something like mvn clean deploy as Maven command. You may have to skip the mvn if it is already implicit.
The TFS instance I am working on was recently upgraded from TFS 2017 Update 1 to TFS 2018 Update 2, allowing me to change the dotnet task version used in my build definitions from 0.* to 2.*.
In doing so, the dotnet test step no longer works, returning the following error:
MSBUILD : error MSB1008: Only one project can be specified.
Switch: trx
The command it runs is:
C:\Program Files\dotnet\dotnet.exe" test <Agent_WorkFolder>\1\w\3\s\source\MySolution\MyProject.csproj --configuration release --logger trx --logger trx --results-directory <Agent_WorkFolder>\1\w\_temp
The parameters given to the task are:
Paths to projects = **\*Tests*.csproj
Arguments = --configuration $(BuildConfiguration) --logger trx
Reverting the task version back to 0.*, and it runs again. What is causing this error?
The problem is caused by --logger trx being specified in the Arguments to the task. The newer versions of the task automatically adds this switch when executing the dotnet test command as its the output which TFS supports for reading test results. The extra argument results in the switch being given twice, so whilst MSBUILD error is unhelpful, the Switch: trx part gives a clue as to what is the problem.
Removing the switch from the arguments resolved the problem.
I have a step in my .gitlab-ci.yml to run a script that generates some artifacts. Under normal circumstances, the directory contains artifacts and they are are picked up as such by gitlab-ci. But, I'm trying to set things up so that the task fails if there are no artifacts. All I get now is a warning in the log telling me there are no artifacts. I want to treat this warning as an error and fail the task. Is there a way to do this?
I suppose I could just update my bash script to exit non-zero if the artifacts aren't present, but I'd like to do it in the gitlab task definition if possible.
rpm_build:
stage: build
script: ./scripts/build_rpms.sh
artifacts:
paths:
- my/RPMS/
expire_in: 3 days
I've looked at the documentation on the artifacts section, but couldn't find anything.
https://docs.gitlab.com/ce/ci/yaml/#artifacts
At the moment, this is still an open issue in GitLab: https://gitlab.com/gitlab-org/gitlab-ce/issues/35641
Therefore you will have to resort to updating your script in a way that it returns a non-zero exit status.