"__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping" after V2-V3 pipeline converstion - msbuild

UPDATE: Here's the contents of azure_pipelines.yml:
resources:
repositories:
- repository: pf
type: git
name: _
ref: refs/tags/3.6.6
trigger: none
stages:
- template: __
parameters:
project:
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "__" # Project Path
file: "__.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "___.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "_______"
fortifyVersion: "____"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "___" # Project Path
file: "___.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "_.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "______"
fortifyVersion: "__"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
I'm attempting to run the ADO pipeline for a .NET Framework 4.7 app that has undergone the V2 to V3 pipeline conversion. I get the following error message, and the build doesn't even try to run:
__built-in-schema.yml: Maximum object depth exceeded
__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping
__built-in-schema.yml: Maximum object depth exceeded
That's using the msbuild toolset. I have an API project and a web app project in the solution, so if I removed either of them and then run the pipeline, it progresses past this stage, but gets hung on another error. (Not really necessary to go into detail about that error though, because it's directly related to me having removed the other project).
With the dotnet toolset, it runs, but then complains about a missing reference:
> D:\a\1\s\___\___.csproj(350,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the expression in the Import declaration "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" is correct, and that the file exists on disk.
0 Warning(s)
I've tried commenting out the reference to Microsoft.WebApplication.targets in the .csproj files for the app, but it just adds them back in when I run the build locally or via the pipeline.
I've also tried changing the build pool to VS 2017 and Hosted Windows 2019 with VS2019, thinking the VS targets would be available in those pools, but no luck.
I also tried installing MSBuild.Microsoft.VisualStudio.Web.targets via NuGet, but end up with the same results.
What am I missing here?

“__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping” after V2-V3 pipeline converstion
According to the error log:
error MSB4019: The imported project "C:\Program
Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets"
was not found
We could to know the Microsoft.WebApplication.targets invoked from dotnet SDK instead of MSBuild. The correct path for Microsoft.WebApplication.targets should be:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets
So, we need use MSBuild toolset instead of the dotnet toolset. And since it use the MSBuild 16.0, we need use the build agent pool Hosted Windows 2019 with VS2019 to build the project.
For the error Maximum object depth exceeded, it seems your yaml file has too deep file path, please try to shortening the file path to see if this resolves the issue.
If above info not help you, please share more info about you issue, like what is your project type, is python project? And share your build pipeline (yaml).
Hope this helps.

Related

Bamboo Spec YAML and location of shared artifacts

in the context of using Gradle to drive build, testing, and further jobs/stages on Bamboo server (version 7.2.1) I've configured env. variable GRADLE_USER_HOME to save downloaded Gradle binary to project-local path with the intent to share it with further downstream jobs/stages.
But unfortunately Bamboo ignores "source" or location folder of the artifact. Excerpt from our bamboo.yaml:
Build Java application artifact:
tasks:
- script:
scripts:
- "export GRADLE_USER_HOME=${bamboo.build.working.directory}/GradleUserHome"
- ./gradlew --no-daemon assemble
- "echo GRADLE USER HOME content; ls -al $GRADLE_USER_HOME/; echo '---'" # DEBUG
artifacts:
- name: "Gradle Wrapper installation"
location: GradleUserHome
pattern: '**/*.*'
required: true
shared: true
Debugging output of the echo command shows expected content.
But next downstream job shows that content of artifact "Gradle Wrapper installation" is installed relative to project's workspace, but not in sub-folder ./GradleUserHome as denoted by location key (just as if mentioned location config item is simply ignored with downstream jobs/stages).
Any ideas how to fix this?
Thanks
PS: Next downstream job exhibits in its log messages something like the following:
Preparing artifact 'Gradle Wrapper installation' for use at /var/atlassian/bamboo-agent02-home/xml-data/build-dir/[...] (location: )
Take notice of empty location!

CodePipeline ElasticBeanstalk [ERROR] An error occurred during execution of command [app-deploy] - [CheckProcfileForDotNetCoreApplication]

I've built a Code Pipeline (Source > Build > Deploy) and it's failing on the deploy step.
It's a Net Core 3.1 Api project.
I check the elastic beanstalk logs and I see:
2020/07/02 14:14:00.600060 [ERROR] An error occurred during execution of command [app-deploy] - [CheckProcfileForDotNetCoreApplication]. Stop running the command. Error: error stat /var/app/staging/MyApi/MyApi.dll: no such file or directory with file /var/app/staging/MyApi/MyApi.dll
As far as I know I have no control over /var/app/staging/ and this is built in AWS stuff?
The build step is working so I am unsure on this error.
My buildspec.yml is:
version: 0.2
phases:
build:
commands:
- dotnet publish -c release -o ./build_output ./MyApi/MyApi.csproj
artifacts:
files:
- '**/*'
base-directory: 'build_output'
This is the "zipfile/build_output" folder:
This is the zip file root folder:
These are the files in the build artifacts zip file that pipeline is using. The error says it cannot find MyAppName.dll (renamed to MyApi in the pic). It's there so I wonder why the problem.
Perhaps it doesnt like the folder structure in the zip file - see pic.
I had the same problem.
In my case, if the solution name and project name were different, I got the same error when I deployed the code from Visual Studio to Beanstalk, but when I added a project with the same name as the solution name and built it, I didn't get an error.
I suspect that there will be behavior during deployment that assumes the .dll file has the same name as the solution name.
Warning: This is a workaround, not a solution!
On the project that's failing to deploy, change the "Assembly name" in Project Properties / Application tab, to the name of the DLL it's missing (typically the solution name or the first period-separated part of the namespace).
i.e. "SLNNAME"
Then, redeploy your beanstalk app and it should work.
As Marcin correctly noticed, the indentation was incorrect for the "base-directory"
base-directory: 'build_output'
Should be
base-directory: 'build_output'
As others have noted, it is looking for only the first part of your project name .dll. In my case, my project and Assembly name were both UC.Web which yielded the error during deployment:
Error: error stat /var/app/staging/UC.dll: no such file or directory with file /var/app/staging/UC.dll
My solution that worked was I renamed my Assembly name from UC.Web to simply UC and it deployed successfully. While a solution for everyone, it is a workaround for the time being until Amazon fixes this.

SonarQube scanner for MSBuild: end MSBuild integration failed

I'm having issues trying to execute the SonarQube Scanner. The begin part seems to run without issues, and also the MSBuild command, but the end part fails.
1st step: I execute the following SonarQube Scanner command:
D:\workspace\MasterData>SonarQube.Scanner.MSBuild.exe begin /k:"masterdata" /n:"MasterData" /v:"1.0" /d:sonar.host.url=http: //xyz.com.br:9000/sonarqube /d:sonar.login=5773c2ca935fde42c72494a96de5a68a5b6899d1 /d:sonar.verbose=true
SonarQube Scanner for MSBuild 2.2
Default properties file was found at D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Loading analysis properties from D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
sonar.verbose=true was specified - setting the log verbosity to 'Debug'
Pre-processing started.
Preparing working directories...
Using environment variables to determine the download directory...
Removing the existing directory: D:\workspace\MasterData\.sonarqube
Creating directory: D:\workspace\MasterData\.sonarqube
SonarQube server URL: http: //xyz.com.br:9000/sonarqube
SonarQube Scanner for MSBuild 2.2
15:11:10.032 Loading analysis properties from D:\sonar-scanner-msbuild-2.2
.0.24\SonarQube.Analysis.xml
15:11:10.04 sonar.verbose=true was specified - setting the log verbosity to 'Debug'
15:11:10.042 Updating build integration targets...
15:11:10.045 Installed SonarQube.Integration.ImportBefore.targets to C:\Users\AppData\Local\Microsoft\MSBuild\14.0\Microsoft.Common.targets\ImportBefore
15:11:10.046 Installed SonarQube.Integration.ImportBefore.targets to C:\Users\f
g009adm\AppData\Local\Microsoft\MSBuild\12.0\Microsoft.Common.targets\ImportBefore
15:11:10.048 Installed SonarQube.Integration.targets to D:\workspace\MasterData\.sonarqube\bin\targets
15:11:10.049 Creating config and output folders...
15:11:10.049 Creating directory: D:\workspace\MasterData\.sonarqube\conf
15:11:10.049 Creating directory: D:\workspace\MasterData\.sonarqube\out
15:11:10.055 Fetching analysis configuration settings...
15:11:10.062 Fetching properties for project 'masterdata' from http: //xyz.com.br:9000/sonarqube/api/properties?resource=masterdata...
15:11:10.062 Downloading from http: //xyz.com.br:9000/sonarqube/api/properties?resource=masterdata...
15:11:10.214 Downloading from http: //xyz.com.br:9000/sonarqube/api/updatecenter/installed_plugins...
15:11:10.239 Fetching quality profile for project 'masterdata' from http: //xyz.com.br:9000/sonarqube/api/qualityprofiles/search?projectKey=masterdata...
15:11:10.24 Downloading from http: //xyz.com.br:9000/sonarqube/api/qu
alityprofiles/search?projectKey=masterdata...
15:11:10.272 Downloading from http: //xyz.com.br:9000/sonarqube/api/rules/search?f=repo,name,severity,lang,internalKey,templateKey,params,actives&ps=
500&activation=true&qprofile=cs-sbs-cs-default-201410-86215&p=1...
15:11:10.451 Downloading from http: //xyz.com.br:9000/sonarqube/api/rules/search?f=internalKey&ps=500&activation=false&qprofile=cs-sbs-cs-default-201410-86215&p=1&languages=cs...
15:11:10.506 Generating rulesets...
15:11:10.506 Generating the FxCop ruleset: D:\workspace\MasterData\.sonarq
ube\conf\SonarQubeFxCop-cs.ruleset
15:11:10.518 Writing Roslyn generated ruleset to D:\workspace\MasterData\.sonarqube\conf\SonarQubeRoslyn-cs.ruleset...
15:11:10.534 Writing Roslyn analyzer additional file to D:\workspace\MasterData\.sonarqube\conf\cs\SonarLint.xml...
15:11:10.536 Provisioning analyzer assemblies for cs...
15:11:10.538 Installing required Roslyn analyzers...
15:11:10.538 Local analyzer cache: C:\Users\fg009adm\AppData\Local\Temp\7\.sonarqube\.static
15:11:10.538 Processing plugin: csharp version 1.23.0.1828
15:11:10.54 Cache hit: using plugin files from C:\Users\AppData\Local\Temp\7\.sonarqube\.static\csharp_1.23.0.1828\SonarAnalyzer-1.23.0.1828.zip
15:11:10.566 Pre-processing succeeded.
2nd step: I execute the following MSBuild command:
MSBuild.exe /t:Rebuild MasterDataWebServices.sln
Since the MSBuild log is quite big, I will not provide it here. There is a Build succeeded message, with 5 warnings and 0 errors.
3rd step: I execute the following SonarQube Scanner end command:
D:\workspace\MasterData>SonarQube.Scanner.MSBuild.exe end
SonarQube Scanner for MSBuild 2.2
Default properties file was found at D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Loading analysis properties from D:\sbs1\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Post-processing started.
SonarQube Scanner for MSBuild 2.2
14:25:22.961 Loading the SonarQube analysis config from D:\workspace\MasterData\.sonarqube\conf\SonarQubeAnalysisConfig.xml
14:25:22.964 Not running under TeamBuild
14:25:22.964 Analysis base directory: D:\workspace\MasterData\.sonarqube
Build directory:
Bin directory: D:\workspace\MasterData\.sonarqube\bin
Config directory: D:\workspace\MasterData\.sonarqube\conf
Output directory: D:\workspace\MasterData\.sonarqube\out
Config file: D:\workspace\MasterData\.sonarqube\conf\SonarQubeAnalysisConfig.xml
Generating SonarQube project properties file to D:\workspace\MasterData\.sonarqube\out\sonar-project.properties
The SonarQube MSBuild integration failed: SonarQube was unable to collect the required information about your projects.
Possible causes:
1. The project has not been built - the project must be built in between the begin and end steps
2. An unsupported version of MSBuild has been used to build the project. Currently MSBuild 12.0 upwards are supported
3. The begin, build or end steps have not all been launched from the same folder
Writing processing summary to D:\workspace\MasterData\.sonarqube\out\ProjectInfo.log
Generation of the sonar-properties file failed. Unable to complete SonarQube ana
lysis.
14:25:22.988 Creating a summary markdown file...
14:25:22.989 Post-processing failed. Exit code: 1
I was facing the same problem. In my case I installed corresponding MSBuild version required by my version of Sonar and it worked just fine.
My SonarScanner required version 14.0 or upwards of MSBuild, then I downloaded Microsoft Build Tools 2015.
In your case I believe it this version will work too and if not you can try this version more specifically.
Hope it helps.

Index Sources & Publish Symbols step fails in TFS 2015

By default TFS Build creates a "Index Sources & Publish Symbols" step in a Visual Studio Build definition. When the step's property "Path to publish symbols" is empty, everything works fine. But when setting the path to a local directory, the step generates the following error:
2015-09-25T11:00:09.7991491Z Executing the powershell script: C:\NewAgent\tasks\PublishSymbols\1.0.4\PublishSymbols.ps1
2015-09-25T11:00:10.0022755Z Find-Files -SearchPattern **\bin\**\*.pdb -RootFolder C:\NewAgent\_work\990dcb3f\Projects
2015-09-25T11:00:10.0491548Z Found 15 files to index...
2015-09-25T11:00:10.0491548Z Invoke-IndexSources -RepositoryEndpoint <repositoryEndpoint> -SourceFolder C:\NewAgent\_work\990dcb3f\Projects -PdbFiles <pdbFiles>
2015-09-25T11:00:10.6272794Z Invoke-PublishSymbols -PdbFiles <pdbFiles> -Share C:\DebugSymbols -Product EntityMapper -Version 20150925.14 -MaximumWaitTime 7200000 -MaximumSemaphoreAge 1440 -ArtifactName
2015-09-25T11:00:10.7348541Z ##[error]Can't infer artifact type from artifact location C:\DebugSymbols.
2015-09-25T11:00:10.7504779Z
2015-09-25T11:00:10.7504779Z Start: AssociateArtifact
2015-09-25T11:00:10.7504779Z ##[error]Artifact Type is required.
2015-09-25T11:00:10.7504779Z End: AssociateArtifact
The pdb files are published to the specified directory, so i don't really understand what is causing the error (which then causes the whole build job to fail).
When the step's property "Path to publish symbols" is empty, everything works fine.
Yes, because if symbols path is not set, the script only index sources, not publishing symbols.
But when setting the path to a local directory, the step generates the error
You need prepare your symbol store. In your case setup folder sharing and permissions.
In my case changing the "Path to publish symbols" from
//myserver/symbols
to
\\myserver\symbols
solved the issue. Don't know why this happend to me but ok... :)

Building .NET project with xbuild on Linux environment

I have very simple project that created with Visual Studio 2013, using Nancy Asp.Net Web Application template.
In my development machine, it is building by Visual Studio 2013 successfully.
But when I try to build same sources on Linux Mono environment with xbuild, it is failing.
Here is the output:
root#jannes:/home/hitly/hitly/src/Hitly.Service# xbuild
XBuild Engine Version 12.0
Mono, Version 3.2.8.0
Copyright (C) 2005-2013 Various Mono authors
Build started 3/31/2014 2:11:49 PM.
__________________________________________________
Project "/home/hitly/hitly/src/Hitly.Service/Hitly.Service.csproj" (default target(s)):
Target RestorePackages:
Executing: mono --runtime=v4.0.30319 ../.nuget/NuGet.exe install "" -source "" -RequireConsent -solutionDir "../"
The specified path is not of a legal form (empty). /home/hitly/hitly/src/.nuget/NuGet.targets: error : Command 'mono --runtime=v4.0.30319 ../.nuget/NuGet.exe install "" -source "" -RequireConsent -solutionDir "../"' exited with code: 1.
Task "Exec" execution -- FAILED
Done building target "RestorePackages" in project "/home/hitly/hitly/src/Hitly.Service/Hitly.Service.csproj".-- FAILED
Done building project
"/home/hitly/hitly/src/Hitly.Service/Hitly.Service.csproj".-- FAILED
Build FAILED. Errors:
/home/hitly/hitly/src/Hitly.Service/Hitly.Service.csproj (default targets) -> /home/hitly/hitly/src/.nuget/NuGet.targets
(RestorePackages target) ->
/home/hitly/hitly/src/.nuget/NuGet.targets: error : Command 'mono --runtime=v4.0.30319 ../.nuget/NuGet.exe install "" -source "" -RequireConsent -solutionDir "../"' exited with code: 1.
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:00.7254640
I couldn't understand reason of the error.
How can I build these sources on Linux?
Fixed!
I simply added PackagesConfig element to .nuget/NuGet.targets:
<PropertyGroup Condition=" '$(OS)' != 'Windows_NT'">
<!-- We need to launch nuget.exe with the mono command if we're not on windows -->
<NuGetToolsPath>$(SolutionDir).nuget</NuGetToolsPath>
++++<PackagesConfig>packages.config</PackagesConfig>
</PropertyGroup>
NuGet.targets is a key file for NuGet package restore, but its versions are not well managed by NuGet guys.
The issue you met simply indicates that the version you use is not Mono friendly. Your edit might work, but I recommend you keep an eye on NuGet Git repository to grab the latest, such as http://nuget.codeplex.com/SourceControl/latest#src/Build/NuGet.targets.
Hope NuGet guys can do more testing on Mono as that can prevent similar issues in the long run.
(Please forget about this answer. NuGet latest console can run on latest Mono and the restore mechanism is also changed to get rid of this targets file.)