SonarQube scanner for MSBuild: end MSBuild integration failed - msbuild

I'm having issues trying to execute the SonarQube Scanner. The begin part seems to run without issues, and also the MSBuild command, but the end part fails.
1st step: I execute the following SonarQube Scanner command:
D:\workspace\MasterData>SonarQube.Scanner.MSBuild.exe begin /k:"masterdata" /n:"MasterData" /v:"1.0" /d:sonar.host.url=http: //xyz.com.br:9000/sonarqube /d:sonar.login=5773c2ca935fde42c72494a96de5a68a5b6899d1 /d:sonar.verbose=true
SonarQube Scanner for MSBuild 2.2
Default properties file was found at D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Loading analysis properties from D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
sonar.verbose=true was specified - setting the log verbosity to 'Debug'
Pre-processing started.
Preparing working directories...
Using environment variables to determine the download directory...
Removing the existing directory: D:\workspace\MasterData\.sonarqube
Creating directory: D:\workspace\MasterData\.sonarqube
SonarQube server URL: http: //xyz.com.br:9000/sonarqube
SonarQube Scanner for MSBuild 2.2
15:11:10.032 Loading analysis properties from D:\sonar-scanner-msbuild-2.2
.0.24\SonarQube.Analysis.xml
15:11:10.04 sonar.verbose=true was specified - setting the log verbosity to 'Debug'
15:11:10.042 Updating build integration targets...
15:11:10.045 Installed SonarQube.Integration.ImportBefore.targets to C:\Users\AppData\Local\Microsoft\MSBuild\14.0\Microsoft.Common.targets\ImportBefore
15:11:10.046 Installed SonarQube.Integration.ImportBefore.targets to C:\Users\f
g009adm\AppData\Local\Microsoft\MSBuild\12.0\Microsoft.Common.targets\ImportBefore
15:11:10.048 Installed SonarQube.Integration.targets to D:\workspace\MasterData\.sonarqube\bin\targets
15:11:10.049 Creating config and output folders...
15:11:10.049 Creating directory: D:\workspace\MasterData\.sonarqube\conf
15:11:10.049 Creating directory: D:\workspace\MasterData\.sonarqube\out
15:11:10.055 Fetching analysis configuration settings...
15:11:10.062 Fetching properties for project 'masterdata' from http: //xyz.com.br:9000/sonarqube/api/properties?resource=masterdata...
15:11:10.062 Downloading from http: //xyz.com.br:9000/sonarqube/api/properties?resource=masterdata...
15:11:10.214 Downloading from http: //xyz.com.br:9000/sonarqube/api/updatecenter/installed_plugins...
15:11:10.239 Fetching quality profile for project 'masterdata' from http: //xyz.com.br:9000/sonarqube/api/qualityprofiles/search?projectKey=masterdata...
15:11:10.24 Downloading from http: //xyz.com.br:9000/sonarqube/api/qu
alityprofiles/search?projectKey=masterdata...
15:11:10.272 Downloading from http: //xyz.com.br:9000/sonarqube/api/rules/search?f=repo,name,severity,lang,internalKey,templateKey,params,actives&ps=
500&activation=true&qprofile=cs-sbs-cs-default-201410-86215&p=1...
15:11:10.451 Downloading from http: //xyz.com.br:9000/sonarqube/api/rules/search?f=internalKey&ps=500&activation=false&qprofile=cs-sbs-cs-default-201410-86215&p=1&languages=cs...
15:11:10.506 Generating rulesets...
15:11:10.506 Generating the FxCop ruleset: D:\workspace\MasterData\.sonarq
ube\conf\SonarQubeFxCop-cs.ruleset
15:11:10.518 Writing Roslyn generated ruleset to D:\workspace\MasterData\.sonarqube\conf\SonarQubeRoslyn-cs.ruleset...
15:11:10.534 Writing Roslyn analyzer additional file to D:\workspace\MasterData\.sonarqube\conf\cs\SonarLint.xml...
15:11:10.536 Provisioning analyzer assemblies for cs...
15:11:10.538 Installing required Roslyn analyzers...
15:11:10.538 Local analyzer cache: C:\Users\fg009adm\AppData\Local\Temp\7\.sonarqube\.static
15:11:10.538 Processing plugin: csharp version 1.23.0.1828
15:11:10.54 Cache hit: using plugin files from C:\Users\AppData\Local\Temp\7\.sonarqube\.static\csharp_1.23.0.1828\SonarAnalyzer-1.23.0.1828.zip
15:11:10.566 Pre-processing succeeded.
2nd step: I execute the following MSBuild command:
MSBuild.exe /t:Rebuild MasterDataWebServices.sln
Since the MSBuild log is quite big, I will not provide it here. There is a Build succeeded message, with 5 warnings and 0 errors.
3rd step: I execute the following SonarQube Scanner end command:
D:\workspace\MasterData>SonarQube.Scanner.MSBuild.exe end
SonarQube Scanner for MSBuild 2.2
Default properties file was found at D:\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Loading analysis properties from D:\sbs1\sonar-scanner-msbuild-2.2.0.24\SonarQube.Analysis.xml
Post-processing started.
SonarQube Scanner for MSBuild 2.2
14:25:22.961 Loading the SonarQube analysis config from D:\workspace\MasterData\.sonarqube\conf\SonarQubeAnalysisConfig.xml
14:25:22.964 Not running under TeamBuild
14:25:22.964 Analysis base directory: D:\workspace\MasterData\.sonarqube
Build directory:
Bin directory: D:\workspace\MasterData\.sonarqube\bin
Config directory: D:\workspace\MasterData\.sonarqube\conf
Output directory: D:\workspace\MasterData\.sonarqube\out
Config file: D:\workspace\MasterData\.sonarqube\conf\SonarQubeAnalysisConfig.xml
Generating SonarQube project properties file to D:\workspace\MasterData\.sonarqube\out\sonar-project.properties
The SonarQube MSBuild integration failed: SonarQube was unable to collect the required information about your projects.
Possible causes:
1. The project has not been built - the project must be built in between the begin and end steps
2. An unsupported version of MSBuild has been used to build the project. Currently MSBuild 12.0 upwards are supported
3. The begin, build or end steps have not all been launched from the same folder
Writing processing summary to D:\workspace\MasterData\.sonarqube\out\ProjectInfo.log
Generation of the sonar-properties file failed. Unable to complete SonarQube ana
lysis.
14:25:22.988 Creating a summary markdown file...
14:25:22.989 Post-processing failed. Exit code: 1

I was facing the same problem. In my case I installed corresponding MSBuild version required by my version of Sonar and it worked just fine.
My SonarScanner required version 14.0 or upwards of MSBuild, then I downloaded Microsoft Build Tools 2015.
In your case I believe it this version will work too and if not you can try this version more specifically.
Hope it helps.

Related

Azure Devops VSTEST Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1

I am having an error in the dotnet Core step
I am trying to integrate my selenium test to Azure devops
This the logs of the dot net step
2020-10-16T03:11:25.7523865Z ##[section]Starting: Run Selenium
2020-10-16T03:11:25.8123424Z ==============================================================================
2020-10-16T03:11:25.8124047Z Task : .NET Core
2020-10-16T03:11:25.8124510Z Description : Build, test, package, or publish a dotnet application, or run a custom dotnet command
2020-10-16T03:11:25.8124816Z Version : 2.175.0
2020-10-16T03:11:25.8125178Z Author : Microsoft Corporation
2020-10-16T03:11:25.8125636Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
2020-10-16T03:11:25.8126003Z ==============================================================================
2020-10-16T03:11:27.2023831Z [command]C:\windows\system32\chcp.com 65001
2020-10-16T03:11:27.2216959Z Active code page: 65001
2020-10-16T03:11:27.2376255Z Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
2020-10-16T03:11:27.3059263Z [command]"C:\Program Files\dotnet\dotnet.exe" vstest "D:\a\r1\a\_SeleniumDemo-ASP.NET Core-CI (5)\drop\Demo2.dll" --logger:trxlogfilename=TEST.xml
2020-10-16T03:11:34.8402885Z
2020-10-16T03:11:34.9823477Z Welcome to .NET Core 3.1!
2020-10-16T03:11:34.9824051Z ---------------------
2020-10-16T03:11:34.9824560Z SDK Version: 3.1.402
2020-10-16T03:11:34.9824775Z
2020-10-16T03:11:34.9836054Z Telemetry
2020-10-16T03:11:34.9836648Z ---------
2020-10-16T03:11:34.9838913Z The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
2020-10-16T03:11:34.9840343Z
2020-10-16T03:11:34.9840938Z Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
2020-10-16T03:11:34.9841242Z
2020-10-16T03:11:34.9841449Z ----------------
2020-10-16T03:11:34.9841756Z Explore documentation: https://aka.ms/dotnet-docs
2020-10-16T03:11:34.9842152Z Report issues and find source on GitHub: https://github.com/dotnet/core
2020-10-16T03:11:34.9842572Z Find out what's new: https://aka.ms/dotnet-whats-new
2020-10-16T03:11:34.9842993Z Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
2020-10-16T03:11:34.9843479Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
2020-10-16T03:11:34.9843894Z Write your first app: https://aka.ms/first-net-core-app
2020-10-16T03:11:34.9844375Z --------------------------------------------------------------------------------------
2020-10-16T03:11:39.1360071Z Microsoft (R) Test Execution Command Line Tool Version 16.7.0
2020-10-16T03:11:39.1368990Z Copyright (c) Microsoft Corporation. All rights reserved.
2020-10-16T03:11:39.1369374Z
2020-10-16T03:11:40.1936906Z The Test Logger URI 'trxlogfilename=TEST.xml' is not valid. The Test Logger will be ignored.
2020-10-16T03:11:40.7284653Z ##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
2020-10-16T03:11:40.8737953Z Info: Azure Pipelines hosted agents have been updated to contain .Net Core 3.x (3.1) SDK/Runtime along with 2.1. Unless you have locked down a SDK version for your project(s), 3.x SDK might be picked up which might have breaking behavior as compared to previous versions.
2020-10-16T03:11:40.8738674Z Some commonly encountered changes are:
2020-10-16T03:11:40.8740232Z If you're using `Publish` command with -o or --Output argument, you will see that the output folder is now being created at root directory rather than Project File's directory. To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
2020-10-16T03:11:40.8747693Z ##[error]Dotnet command failed with non-zero exit code on the following projects : D:\a\r1\a\_SeleniumDemo-ASP.NET Core-CI (5)\drop\Demo2.dll
2020-10-16T03:11:40.8951889Z ##[section]Finishing: Run Selenium
I tried to run my code using my local machine and got not error. Please help me with Azure integration
I can reproduce this issue with the same configuration, and then I change the argument to --logger:"trx;Logfilename=TEST.trx" and it works.
Result:

"__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping" after V2-V3 pipeline converstion

UPDATE: Here's the contents of azure_pipelines.yml:
resources:
repositories:
- repository: pf
type: git
name: _
ref: refs/tags/3.6.6
trigger: none
stages:
- template: __
parameters:
project:
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "__" # Project Path
file: "__.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "___.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "_______"
fortifyVersion: "____"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
- name: "__" # Must be unique within the list of projects
type: "msbuild" # Used with Publish/Deploy. Options: adla, dacpac, dotnet, egg, maven, msbuild, node, nuget, python, sap, ssis
path: "___" # Project Path
file: "___.csproj" # Project file
toolset: "msbuild" # Used with Build/Package. Options: adla, dotnet, maven, msbuild, node, python, sap, ssis
playbook: "_.yml"
sonarqube:
name: scan
scan: true
sqExclusions: ""
additionalProperties: "" #|
# sonar.branch.name=master
# sonar.branch.target=master
fortify:
fortifyApp: "______"
fortifyVersion: "__"
sast: true
dast: false
buildConfiguration: $(BuildConfiguration) #release, debug
buildPlatform: $(BuildPlatform) #any cpu, x86, x64
I'm attempting to run the ADO pipeline for a .NET Framework 4.7 app that has undergone the V2 to V3 pipeline conversion. I get the following error message, and the build doesn't even try to run:
__built-in-schema.yml: Maximum object depth exceeded
__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping
__built-in-schema.yml: Maximum object depth exceeded
That's using the msbuild toolset. I have an API project and a web app project in the solution, so if I removed either of them and then run the pipeline, it progresses past this stage, but gets hung on another error. (Not really necessary to go into detail about that error though, because it's directly related to me having removed the other project).
With the dotnet toolset, it runs, but then complains about a missing reference:
> D:\a\1\s\___\___.csproj(350,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the expression in the Import declaration "C:\Program Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" is correct, and that the file exists on disk.
0 Warning(s)
I've tried commenting out the reference to Microsoft.WebApplication.targets in the .csproj files for the app, but it just adds them back in when I run the build locally or via the pipeline.
I've also tried changing the build pool to VS 2017 and Hosted Windows 2019 with VS2019, thinking the VS targets would be available in those pools, but no luck.
I also tried installing MSBuild.Microsoft.VisualStudio.Web.targets via NuGet, but end up with the same results.
What am I missing here?
“__built-in-schema.yml (Line: 2012, Col: 24): Expected a mapping” after V2-V3 pipeline converstion
According to the error log:
error MSB4019: The imported project "C:\Program
Files\dotnet\sdk\3.1.101\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets"
was not found
We could to know the Microsoft.WebApplication.targets invoked from dotnet SDK instead of MSBuild. The correct path for Microsoft.WebApplication.targets should be:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets
So, we need use MSBuild toolset instead of the dotnet toolset. And since it use the MSBuild 16.0, we need use the build agent pool Hosted Windows 2019 with VS2019 to build the project.
For the error Maximum object depth exceeded, it seems your yaml file has too deep file path, please try to shortening the file path to see if this resolves the issue.
If above info not help you, please share more info about you issue, like what is your project type, is python project? And share your build pipeline (yaml).
Hope this helps.

Sonar Buildbreaker Skip Failing the Build

I am using Jenkins job to build .Net project. I have added sonar analysis.
D:\Temp\tools\Scanner_MSBuild-2.2\MSBuild.SonarQube.Runner.exe begin /k:myproj /n:myproj-full /v:1.0.0 /s:D:\Temp\SonarQube.Analysis.xml
I have updated "SonarQube.Analysis.xml" with required sonar url and login credentials. But it fails with following error:
Pre-processing started.
Preparing working directories...
SonarQube Scanner for MSBuild 2.2
09:07:00.437 Updating build integration targets...
09:07:00.451 Fetching analysis configuration settings...
09:07:01.239 Could not find a file on the SonarQube server. URL: https:///api/qualityprofiles/search?defaults=true
09:07:01.239 Pre-processing failed. Exit code: 1
I tried following:
D:\Temp\tools\Scanner_MSBuild-2.2\MSBuild.SonarQube.Runner.exe begin /k:myproj /n:myproj-full /v:1.0.0 /s:D:\Temp\SonarQube.Analysis.xml -Dsonar.buildbreaker.skip=true
But it threw error that -Dsonar.buildbreaker.skip=true is not recognised command. By default buildbreaker is set to false and I am not able to make it true via commandline. SonarQube version I am using is below 5.2.
How do I fix this?
Thank you
If you're using the 'Begin Analysis' and 'End Analysis' build steps, then configuring your server URL in SonarQube.Analysis.xml won't get you far (as you've learned).
Instead, you need to configure the location of your server at the global level as described in the docs. Briefly:
Manage Jenkins > Configure System > SonarQube configuration
Add SonarQube
fill in your server info.
Then double check that that server instance is selected in your build steps (if you have only one server configured, build steps will default to it). And analyze.

SonarQube integration in TeamCity for .NET C# projecs

I'm facig some issues with SonarQube integration in TeamCity with MSBuild.
Basically our nightly build using SonarQube is split into 5 steps:
1 - SonarQube Initialization
MSBuild.SonarQube.Runner.exe begin
/k:"tsmm-%sonar.project%"
/n:"%sonar.project%"
/v:"%sonar.project.version%"
/d:sonar.host.url="%sonar.host.url%"
/d:sonar.cs.dotcover.reportsPaths="coverage-report.html"
/d:sonar.cs.nunit.reportsPaths="nunit-results.xml"
/d:sonar.verbose=true
/d:sonar.jdbc.username="%sonar.jdbc.username%"
/d:sonar.jdbc.password="%sonar.jdbc.password%"
/d:sonar.jdbc.url="%sonar.jdbc.url%"
2 - Muggets restauration
3 - MSBuild
4 - Tests via NUnit
5 - Result publication to SonarQube
MSBuild.SonarQube.Runner.exe end
/d:sonar.scm.disabled=true
/d:sonar.jdbc.username="%sonar.jdbc.username%"
/d:sonar.jdbc.password="%sonar.jdbc.password%"
When the TeamCity is running the build definition above, everything is going fine until it reach the step no. 5 as stated in the build log:
Step 5/5: Publish to SonarQube (Command Line)
ERROR: Error during Sonar runner execution
org.sonar.runner.impl.RunnerException: Unable to execute Sonar
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:91)
...
Caused by: org.sonar.api.utils.SonarException: SCM URL should be provided. Please review "sonar.scm.url" parameter (or the <scm> section of pom.xml if you are using Maven).
I went through different resources including the official one there: http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+MSBuild
But could not manage to find out any proper explanation, I also tried to disable the scm support as mentioned on that page: http://docs.sonarqube.org/display/SONAR/SCM+support
by adding /d:sonar.scm.disabled=true to the step 1 but it does not change anything.
The sonar.jdbc.* properties are only for older versions of SonarQube and if you are on a newer than 5.4, you should not set them.
To authenticate to SonarQube you need to use the sonar.login and sonar.password properties, as described in the article below:
http://docs.sonarqube.org/display/SCAN/From+the+Command+Line#FromtheCommandLine-Analyzeaproject
In all cases, if you have older SonarQube, I would strongly recommend upgrading to the LTS (long term support) version:
https://www.sonarqube.org/downloads/

No ProjectInfo.xml files were found error from command prompt using MSBuild SonarQube Runner v1.0

I'm trying out SonarQube using the new MSBuild SonarQube Runner v1.0.
The Pre-processing works fine
E:\sonarQube\MSBuild.SonarQube.Runner-1.0>MSBuild.SonarQube.Runner.exe begin /k:"MyKey" /n:"MyProject" /v:"1.0"
.......
Process returned exit code 0
Pre-processing succeeded.
Then MSBuild works
E:\sonarQube\WebApplication2013>msbuild
Microsoft (R) Build Engine version 4.0.30319.34209
[Microsoft .NET Framework, version 4.0.30319.34209]
Copyright (C) Microsoft Corporation. All rights reserved.
........
Done Building Project "E:\sonarQube\WebApplication2013\WebApplication2013.sln" (default targets).
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:00.51
While end command following error gets generated
E:\sonarQube\MSBuild.SonarQube.Runner-1.0>MSBuild.SonarQube.Runner.exe end
.......
6:35:10 PM Generating SonarQube project properties file to E:\sonarQube\MSBuild.SonarQube.Runner-1.0\.sonarqube\out\sonar-project.properties
6:35:10 PM No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
6:35:10 PM Writing processing summary to E:\sonarQube\MSBuild.SonarQube.Runner-1.0\.sonarqube\out\ProjectInfo.log
6:35:10 PM Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
6:35:10 PM Creating a summary markdown file...
Process returned exit code 1
Post-processing failed. Exit code: 1
Required your assistance
As pointed out by duncanpMS's comment, you need to run all the 3 commands from the folder of the project you want to analyze.
Here is how:
cd E:\sonarQube\WebApplication2013
E:\sonarQube\MSBuild.SonarQube.Runner-1.0\MSBuild.SonarQube.Runner.exe begin /k:"MyKey" /n:"MyProject" /v:"1.0"
msbuild /t:rebuild
E:\sonarQube\MSBuild.SonarQube.Runner-1.0\MSBuild.SonarQube.Runner.exe end
You can add E:\sonarQube\MSBuild.SonarQube.Runner-1.0 to your %PATH% environment variable so that you can simply type MSBuild.SonarQube.Runner.exe instead of the full path from the command line when you analyze projects.
You need msbuild version 12 or version 14. If you have VS2013 , Msbuild version 12 comes with it. If you have VS2015, Msbuild version 14 comes with it.
If you don't have , download Microsoft Build Tools 2013 or 2015.
Run Msbuild -version to verify the version before running the 3 commands sequentially in the solution folder.