Azure DevOps build pipeline fails for solution with .NET Framework and .NET 5 projects - msbuild

I have a .NET solution that used to contain just .NET 5 applications. This compiled just fine using a YAML file on Azure DevOps.
However, since adding some .NET Framework 4.8 projects this now fails. I have read some articles that seem to explain the problem (to some extent) but the solution isn't clear to me. Perhaps someone could spell it out for me here?
In more detail...
Projects
I have the following mixture of projects:
5 x SDK projects, net5.0 (Test Suite)
1 x SDK project, netstandard 2.0 (library)
1 x SDK project, net5.0 (Web API)
1 x SDK project, net4.8 (Test)
1 x non-SDK project, net4.8 (Web API)
The library is common to all. The .NET 4.8 WebAPI is referenced by the .NET 4.8 test suite, and the other .NET 5 test suites reference the .NET 5 WebAPI.
I have the following YAML file:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildConfiguration: 'Release'
buildPlatform: 'Any CPU'
platform: x64
steps:
- task: NuGetCommand#2
displayName: 'Restore for all'
inputs:
restoreSolution: '$(solution)'
feedsToUse: 'select'
vstsFeed: '{Guid # 1}/{Guid # 2}'
- task: VSBuild#1
displayName: 'Build all'
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
maximumCpuCount: true
- task: DotNetCoreCLI#2
displayName: 'Run Tests'
inputs:
command: 'test'
projects: '**/*Test.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
We have a private NuGet gallery in the Azure Artifacts and the two Guids ({Guid # 1}/{Guid # 2}) allow us to access that.
When this now executes, the following steps appear to succeed: Restore for all and Build all. However, the step that appears to fail is the Run Tests.
The error message is:
##[error]Dotnet command failed with non-zero exit code on the following projects : D:\a\1\s\path\MySdkNet48.Test.csproj
This is the Test project that is .NET 4.8 and uses a SDK project file format.
Digging into the more verbose details in the Run Tests step I see:
D:\a\1\s\path\MyNonSdkNet48WebApi.csproj(328,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\5.0.401\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the expression in the Import declaration "C:\Program Files\dotnet\sdk\5.0.401\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" is correct, and that the file exists on disk.
##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
and
##[error]Dotnet command failed with non-zero exit code on the following projects : D:\a\1\s\path\MySdkNet48Test.csproj
As mentioned above, I'm not clear on how to fix this...

Since the 4.8 project isn't a Dotnet Core/5.0 project, it'll need to run using the Visual Studio Test task.
Split the test step in 2 separate steps, one configured to run the 4.8 projects, one to run the dotnet core/5.0 projects.
- task: DotNetCoreCLI#2
displayName: 'Run Tests'
inputs:
command: 'test'
projects: '**/*Test.csproj;!**/*48.Test.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
- task: VSTest#2
displayName: 'VsTest - testAssemblies'
inputs:
testAssemblyVer2: |
**\*48.Test.dll

Related

Cobertura - Coverlet.runsettings.xml values ignored by dotnetcore CLI test command

I am trying to generate code coverage for one of my solutions in an Azure DevOps build pipeline. Whilst I have the results of my tests appearing upon build completion along with the Cobertura code coverage report, there are a lot of files and namespaces I would like to exclude from the code coverage report as they are not testable units of code (e.g. models or database migrations).
I understand that I can utilise the coverlet.runsettings.xml file to do namespace exclusions but this does not appear to be working.
My runsettings file is setup inside one of the test projects that is run and is setup as follows:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="XPlat code coverage">
<Configuration>
<Exclude>[MyDatabase.Migrations.*]*,[*]MyDatabase.Migrations*</Exclude>
</Configuration>
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
</RunSettings>
I'm trying to exclude all files that sit under the MyDatabase.Migrations namespace but the syntax I have used above doesn't seem to have any effect and I still see the files beneath this namespace shown in final code coverage report in Devops.
My pipeline is setup as follows:
steps:
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '$(solution)'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'Tests'
inputs:
command: 'test'
projects: |
**\MyDatabase.Tests
**\MySearch.Common.Tests
**\MySearchService.Tests
**\MyConfiguration.Tests
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage"'
publishTestResults: true
# ReportGenerator extension to combine code coverage outputs into one
- task: reportgenerator#5
inputs:
reports: '$(Agent.WorkFolder)/**/coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)/CoverageResults'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Build.SourcesDirectory)/CoverageResults/Cobertura.xml'
reportDirectory: '$(Build.SourcesDirectory)/CoverageResults'
I am aware I can use the [ExcludeFromCodeCoverage] tag but with things that are auto-generated like migrations this could get pretty messy having to go in and modify them every time a new one is generated.
I would like the migrations folder completely omitted from even appearing in the generated report.
Any help with this issue would be greatly appreciated.
The VSTestIntegration documentation indicates you will need to include the custom runsettings file as an agrument:
This runsettings file can easily be provided using command line option as given : dotnet test --collect:"XPlat Code Coverage" --settings coverlet.runsettings
Try the following:
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" --settings [path to tests project]\coverlet.runsettings.xml'

AutoIncrementing Build with Core and DevOps Pipeline

I'm trying to setup an auto-incrementing version number for my ASP.NET Core 3.1 web app built using Azure Pipelines. I've tried various snippets and pipeline tasks and got as far as generating the version number but my build fails with the message
The specified version string does not conform to the required format - major[.minor[.build[.revision]
I'm using the below snippet of yml to generate the version number:
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
version.MajorMinor: '1' # Manually adjust the version number as needed for semantic versioning. Revision is auto-incremented.
version.Revision: $[counter(variables['version.MajorMinor'], 0)]
versionNumber: '$(version.MajorMinor).$(version.Revision)'
steps:
- task: PowerShell#2
displayName: Set the name of the build (i.e. the Build.BuildNumber)
inputs:
targetType: 'inline'
script: |
$doy = (Get-Date).DayofYear
Write-Host "##vso[task.setvariable variable=DayOfYear]$doy"
$rev = $env:BUILD_BUILDNUMBER.Split('.')[-1]
Write-Host "##vso[task.setvariable variable=Revision]$rev
Write-Host "set Revision to $rev"
[string] $buildName = "$(versionNumber).$doy.$rev"
Write-Host "Setting the name of the build to '$buildName'."
Write-Host "##vso[build.updatebuildnumber]$buildName"
This does generate a version number (1.26.199.50), however when I get the the MSBuild task:
task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\WebApp.zip" /p:DeployIisAppPath="Default Web Site" /p:Version=$(buildName)'
I get the version error. I've even tried replacing the version with 1.0.199.50 but I get the same error. I understand that each segment has to be less than 65534 as that's the max value for a UInt so I don't understand why it's failing.
After much frustration, I did this with a Powershell script in the release pipeline instead. I've got a Powershell task containing the below which is before the File Transform task. Steps to get it working.
Add a release pipeline variable called "MajorVersion". This is the first part of the version string
Add a property to appsettings.json called AppConfig.Version, set it to whetever for local testing.
Add a powershell task. The order should be Powershell Task->File Transform->App Service Deploy:
$doy = $(Get-Date).DayofYear
[string] $buildName = "$Env:MajorVersion.$doy.$Env:BUILD_BUILDNUMBER"
Write-Host "##vso[task.setvariable variable=AppConfig.Version;]$buildName"
Access the variable from your app settings using your preferred method. In core, probably via Configuration.GetSection("blah")
The assembly isn't stamped with the version, but you can get to it to display on the UI. I feel like this needs to be simplified by MS as auto-incrementing a version shouldn't be so difficult.

"__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping" after V2-V3 pipeline converstion

UPDATE: Here's the contents of azure_pipelines.yml:
resources:
repositories:
- repository: pf
type: git
name: _
ref: refs/tags/3.6.6
trigger: none
stages:
- template: __
parameters:
project:
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "__" # Project Path
file: "__.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "___.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "_______"
fortifyVersion: "____"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "___" # Project Path
file: "___.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "_.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "______"
fortifyVersion: "__"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
I'm attempting to run the ADO pipeline for a .NET Framework 4.7 app that has undergone the V2 to V3 pipeline conversion. I get the following error message, and the build doesn't even try to run:
__built-in-schema.yml: Maximum object depth exceeded
__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping
__built-in-schema.yml: Maximum object depth exceeded
That's using the msbuild toolset. I have an API project and a web app project in the solution, so if I removed either of them and then run the pipeline, it progresses past this stage, but gets hung on another error. (Not really necessary to go into detail about that error though, because it's directly related to me having removed the other project).
With the dotnet toolset, it runs, but then complains about a missing reference:
> D:\a\1\s\___\___.csproj(350,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the expression in the Import declaration "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" is correct, and that the file exists on disk.
0 Warning(s)
I've tried commenting out the reference to Microsoft.WebApplication.targets in the .csproj files for the app, but it just adds them back in when I run the build locally or via the pipeline.
I've also tried changing the build pool to VS 2017 and Hosted Windows 2019 with VS2019, thinking the VS targets would be available in those pools, but no luck.
I also tried installing MSBuild.Microsoft.VisualStudio.Web.targets via NuGet, but end up with the same results.
What am I missing here?
“__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping” after V2-V3 pipeline converstion
According to the error log:
error MSB4019: The imported project "C:\Program
Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets"
was not found
We could to know the Microsoft.WebApplication.targets invoked from dotnet SDK instead of MSBuild. The correct path for Microsoft.WebApplication.targets should be:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets
So, we need use MSBuild toolset instead of the dotnet toolset. And since it use the MSBuild 16.0, we need use the build agent pool Hosted Windows 2019 with VS2019 to build the project.
For the error Maximum object depth exceeded, it seems your yaml file has too deep file path, please try to shortening the file path to see if this resolves the issue.
If above info not help you, please share more info about you issue, like what is your project type, is python project? And share your build pipeline (yaml).
Hope this helps.

When deploying my build to azure (i use devops pipeline but i also tried to deploy via visual studio directly) i keep getting resource not found 404

When I try to publish my .net core 2.2 webapi to my azure app service (via azure devops using Azure App Service deploy or via visual studio publish method) I keep getting:
"The resource you are looking for has been removed, had its name changed, or is temporarily unavailable."
But when I publish it directly with visual studio ( without package or zip configuration) it works. (but we can't keep doing this because we need it working on the azure devops pipeline).
I have tried to build the project differently using dotnet build/publish and vsbuild. I Also tried using different methods of publish (zip,package, web deploy).
I went thru the generated xml files (deploy.cmd,deploy-readme,parameters,setparametersa and sourcemanifest) to check if there is some wrong naming issue or if the folder structure is incorrect but everything matches correctly. The zip file where my project is in, is also in the same location as all the generated xml files(root).
The strange thing is that it was working before and we are using terraform to generate the azure resources. So we threw away the resources which we made manually and remade them using terraform and then we get the error. We did not change anything on the pipeline.
yaml build code from the pipeline:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:CreatePackageOnPublish=true /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:DesktopBuildPackageLocation="$(build.artifactStagingDirectory)\{redactedprojectname}.zip" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
Terraform script for the app service part :
# Create App Service Plan
resource "azurerm_app_service_plan" "asp" {
name = "${local.aspName}"
location = "${azurerm_resource_group.rg.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
tags = "${var.tags}"
sku {
tier = "${var.aspSku.tier}"
size = "${var.aspSku.size}"
}
}
# Create App Service
resource "azurerm_app_service" "as" {
name = "${local.apiName}"
location = "${azurerm_resource_group.rg.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
app_service_plan_id = "${azurerm_app_service_plan.asp.id}"
tags = "${var.tags}"
https_only = true
site_config {
dotnet_framework_version = "v4.0"
always_on = true
}
app_settings = {
"APPINSIGHTS_INSTRUMENTATIONKEY" = "${azurerm_application_insights.ai.instrumentation_key}"
"ASPNETCORE_ENVIRONMENT" = "${var.environment}"
}
}
# --- Output section --
output "appServiceName" {
value = azurerm_app_service.as.name
}
Terraform steps in azure devops:
use terraform 0.12.11
Terraform init
Terraform plan
Terraform apply
Terraform output get appservice name
after this we do Azure App Service deploy
yaml code app service deploy:
variables:
environment: 'prd'
steps:
- task: AzureRmWebAppDeployment#4
displayName: 'Azure App Service Deploy: $(appServiceName)'
inputs:
azureSubscription: '{redactedprojectname}'
WebAppName: '$(appServiceName)'
packageForLinux: '$(System.DefaultWorkingDirectory)/{redactedprojectname}/drop'
AppSettings: ASPNETCORE_ENVIRONMENT "$(environment)"'
enableCustomDeployment: true
DeploymentType: zipDeploy
TakeAppOfflineFlag: false
I would expect that after having a working solution before, that deleting the resources and redeploying to let terraform recreate the resources from scratch without modifying the pipeline, would just provide a working solution
We ended up solving the build issue by using msbuild with filesystem instead of package
Here is the yaml code for anyone who encounters the same issue:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:Configuration=Release /p:WebPublishMethod=FileSystem /p:SkipInvalidConfigurations=true /p:DeployDefaultTarget=WebPublish /p:DeleteExistingFiles=True /p:publishUrl="$(build.artifactStagingDirectory)\YourProjectNameFolder" /p:DeployIisAppPath="Default Web Site"'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
All you need to change is the msbuildArgs property.
The msbuild arguments for the filesystem are based on https://www.joe-stevens.com/2018/12/14/publish-to-file-system-with-msbuild/

Invalid results file Warning when running Azure DevOps Test Task

I am using Azure DevOps to run some XUnit tests of an Asp.Net Core application:
- task: DotNetCoreCLI#2
displayName: 'Test'
inputs:
command: test
projects: '**/*[Tt]est/*.csproj'
arguments: '--configuration $(buildConfiguration)'
The task succeeds but I get two warnings:
[warning] Invalid results file. Make sure the result format of the file '/home/vsts/work/_temp/_fv-az592_2019-04-09_21_14_05.trx' matches 'VSTest' test results format.
[warning] Invalid results file. Make sure the result format of the file '/home/vsts/work/_temp/_fv-az592_2019-04-09_21_14_10.trx' matches 'VSTest' test results format.
What am I missing?
From github:
warning is as per our design of task since we wants to warn customer
if he is trying to use PTR task without any results, but i agree with
you about more apt warning message and we will correct it in upcoming
release.
Also try yo use VSTest task instead