SonarQube 5.1 execution error using Bamboo - msbuild

I have MSBuild.SonarQube.Runner.exe running on my own computer and everything is working fine. I am now trying to automate this job and give it to Bamboo. I am using the same SonarQube configuration (but not the same environment as it is a Windows 2008 R2 Server versus Windows 7). I am using MSBuild.SonarQube.Runner.exe with the simplest configuration file (only the MySQL server).
My error indicates a path to Dinesh's desktop (if I'm not mistaking, he's a Sonar Qube developer)
This error happens after running MSBuild.SonarQube.Runner.exe end. Any idea ?
18:00:06 Writing processing summary to E:\bamboo-home\xml-data\build-dir\SON-SVACN0-TEST\.sonarqube\out\ProjectInfo.log
18:00:06 Creating directory: E:\bamboo-home\xml-data\build-dir\SON-SVACN0-TEST\.sonarqube\bin\sonar-runner
18:00:06 The SONAR_RUNNER_HOME environment variable is not required and will be ignored.
18:00:06 Calling the sonar-runner...
18:00:06 SONAR_RUNNER_OPTS is not configured. Setting it to the default value of -Xmx1024m
Unhandled Exception: System.ArgumentException: Item has already been added. Key in dictionary: 'bamboo_capability_system_builder_snr_Sonar_Runner' Key being added: 'bamboo_capability_system_builder_snr_Sonar_runner'
at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
at System.Collections.Hashtable.Add(Object key, Object value)
at System.Collections.Specialized.StringDictionaryWithComparer.Add(String key, String value)
at System.Diagnostics.ProcessStartInfo.get_EnvironmentVariables()
at SonarQube.Common.ProcessRunner.SetEnvironmentVariables(ProcessStartInfo psi, IDictionary`2 envVariables, ILogger logger) in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarQube.Common\ProcessRunner.cs:line 151
at SonarQube.Common.ProcessRunner.Execute(String exeName, String args, String workingDirectory, Int32 timeoutInMilliseconds, IDictionary`2 envVariables, ILogger logger) in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarQube.Common\ProcessRunner.cs:line 87
at SonarRunner.Shim.SonarRunnerWrapper.ExecuteJavaRunner(ILogger logger, String exeFileName, String propertiesFileName) in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarRunner.Shim\SonarRunner.Wrapper.cs:line 114
at SonarRunner.Shim.SonarRunnerWrapper.Execute(AnalysisConfig config, ILogger logger) in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarRunner.Shim\SonarRunner.Wrapper.cs:line 68
at SonarQube.TeamBuild.PostProcessor.Program.Main() in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarQube.TeamBuild.PostProcessor\Program.cs:line 51

On Windows, environment variables are case-insensitive.
For some reason, you are trying to define both:
bamboo_capability_system_builder_snr_Sonar_Runner
bamboo_capability_system_builder_snr_Sonar_runner
This is not well supported by the .NET framework, see:
https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.Process/src/System/Diagnostics/ProcessStartInfo.cs#L97
Any call to ProcessStartInfo.Environment when the same environment variable exists with multiple casing will result in the failure you see.
So - perhaps this is something to be fixed in CoreFX (the .NET Framework) - but I do not think a change in the MSBuild SonarQube Runner itself is required.
Another fix is to have Bamboo not to define the same environment variable multiple times. For example, the Windows User Interface to set environment variables overwrites the value if you attempt to define FOO when foo is already defined.

Bamboo is creating many environment variables. The problem is that it creates two times the same variable (Runner vs runner).
bamboo_capability_system_builder_snr_Sonar_Runner=E:\sonar-runner-2.3
bamboo_capability_system_builder_snr_Sonar_runner=E:\sonar-runner-2.3
bamboo_capability_system_builder_snr_Sonar_runners=E:\sonar-runner-2.3
I think there is a bug in MSBuild.SonarQube.Runner as it should check that the key is already contained in this function
at SonarQube.Common.ProcessRunner.SetEnvironmentVariables(ProcessStartInfo psi, IDictionary`2 envVariables, ILogger logger) in c:\Users\dinesh\Desktop\sonarsource\sonar-msbuild-runner\SonarQube.Common\ProcessRunner.cs:line 151
As a quick workaround I decided to set one of the two variables to null in my bamboo script:
set bamboo_capability_system_builder_snr_Sonar_runner=

Related

Getting UnauthorizedAccessException when deploying asp.net core app using Kudu build service

I have set up Continuous Deployment of my aspnet core 2.2 app using Kudu build service (directly connected through GitHub). It was fine until the day before yesterday but started getting UnauthorizedAccessException when I try to deploy anything since yesterday. Nothing changed in my application though. Any idea what could be wrong here??
Here is the detail exception:
Command: "D:\home\site\deployments\tools\deploy.cmd"
Handling ASP.NET Core Web Application deployment.
System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.
at Internal.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
at Internal.Win32.RegistryKey.SetValue(String name, String value)
at System.Environment.SetEnvironmentVariableFromRegistry(String variable, String value, Boolean fromMachine)
Failed exitCode=1, command=dotnet restore "D:\home\site\repository\myproject.sln"
at System.Environment.SetEnvironmentVariable(String variable, String value, EnvironmentVariableTarget target)
An error has occurred during web site deployment.
at Microsoft.DotNet.Cli.Utils.EnvironmentProvider.SetEnvironmentVariable(String variable, String value, EnvironmentVariableTarget target)
at Microsoft.DotNet.ShellShim.WindowsEnvironmentPath.AddPackageExecutablePathToUserPath()
at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure()
at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(IFirstTimeUseNoticeSentinel firstTimeUseNoticeSentinel, IAspNetCertificateSentinel aspNetCertificateSentinel, IFileSentinel toolPathSentinel, Boolean hasSuperUserAccess, DotnetFirstRunConfiguration dotnetFirstRunConfiguration, IEnvironmentProvider environmentProvider)
at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
at Microsoft.DotNet.Cli.Program.Main(String[] args)
System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.\r\n at Internal.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)\r\n at Internal.Win32.RegistryKey.SetValue(String name, String value)\r\n at System.Environment.SetEnvironmentVariableFromRegistry(String variable, String value, Boolean fromMachine)\r\n at System.Environment.SetEnvironmentVariable(String variable, String value, EnvironmentVariableTarget target)\r\n at Microsoft.DotNet.Cli.Utils.EnvironmentProvider.SetEnvironmentVariable(String variable, String value, EnvironmentVariableTarget target)\r\n at Microsoft.DotNet.ShellShim.WindowsEnvironmentPath.AddPackageExecutablePathToUserPath()\r\n at Microsoft.DotNet.Configurer.DotnetFirstTimeUseConfigurer.Configure()\r\n at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(IFirstTimeUseNoticeSentinel firstTimeUseNoticeSentinel, IAspNetCertificateSentinel aspNetCertificateSentinel, IFileSentinel toolPathSentinel, Boolean hasSuperUserAccess, DotnetFirstRunConfiguration dotnetFirstRunConfiguration, IEnvironmentProvider environmentProvider)\r\n at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)\r\n at Microsoft.DotNet.Cli.Program.Main(String[] args)\r\nD:\Program Files (x86)\SiteExtensions\Kudu\86.20224.4450\bin\Scripts\starter.cmd "D:\home\site\deployments\tools\deploy.cmd"
After trying "dotnet restore --help" inside kudu and getting the same error I realized that the error was connected to doing "dotnet restore" inside kudu and not with my application.
After looking at the code related to the error message
https://github.com/dotnet/sdk/blob/5d747e7b4f1450a00eff5844bd76b73588531b2c/src/Cli/dotnet/Program.cs#L152
I realized that I could solve the issue by adding a new application setting with the name DOTNET_ADD_GLOBAL_TOOLS_TO_PATH and the value false
Application setting
After that I could deploy my application using kudu as I did before.

FluentAssertions throws ArgumentOutOfRangeException when assertion fails (possibly NCrunch related)

I'm using FluentAssertions 5.3.0 run under NCrunch 3.14.0.1. When a test fails, I often seen this error as FluentAssertions tries to generate a report message...
System.ArgumentOutOfRangeException: startIndex cannot be larger than
length of string. Parameter name: startIndex at
System.String.Substring(Int32 startIndex, Int32 length) at
FluentAssertions.CallerIdentifier.ExtractVariableNameFrom(StackFrame
frame) at
FluentAssertions.CallerIdentifier.DetermineCallerIdentity() at
FluentAssertions.Execution.AssertionScope.GetIdentifier() at
FluentAssertions.Execution.AssertionScope.FailWith(String message,
Object[] args) at FluentAssertions.Numeric.NumericAssertions`1.Be(T
expected, String because, Object[] becauseArgs) at
TableTests.ConsolidatedMovementsTester.AlternativingMovementStatesAreRecognised()
in ...test.cs
The error is sporadic - often it will disappear if the test is rerun (in which case, a proper error report is shown). I've seen this in the last couple of versions of both FluentAssertions and NCrunch and it appears on multiple machines (all running some flavour of Visual Studio 2017 Professional).
Here's an example of the latest call to bite me...
results.Count(c => c.IsStationary).Should().Be(5);
where results is an ImmutableArray of POCOs
In this case, right-clicking on the failing test and selecting "rerun in new test runner" in the ncrunch window, caused the test to re-run and FA to emit the correct report.
Can anyone suggest a workaround for this?

TFS 2015.4 Unable to start or detach a collection

I'm having an issue with a particular team project collection on my TFS2015.4. This collection caused issues when I wanted to upgrade TFS sometime ago as well. I was able to detach it in TFS2013.3 and then upgraded. Now I want to upgrade to TFS2017 and I don't know how to resolve the issue with this collection.
TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790
Here is the current status:
Won't detach
Won't start
Status -> ApplyPatch -> won't execute
Stays Offline
One project inside and stays at 'Deleting' state
If I try to start, I get this error:
TF400783: The host 'MyDAS' cannot be started. The host is in the process of being serviced. The servicing may have failed and needs to be restarted and completed before the host can be started.
I did a pre-production upgrade to TFS2017 and there was a validation error with this collection's state that prevented me from finishing the upgrade.
The detailed log for ApplyPatch Rerun has just one failure point:
[12:29:58.457] +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[12:29:58.457] Executing step: Populate commit changes
[12:29:58.457] Executing step: 'Populate commit changes' Git.M83PopulateCommitChanges (1017 of 1201)
[12:29:58.477] [Error] TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790.
[12:29:58.480] Microsoft.TeamFoundation.Framework.Server.JobDefinitionNotFoundException: TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790.
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationJobService.ResolveJobPriorityClasses(IVssRequestContext requestContext, IEnumerable`1 jobReferences, ITFLogger logger)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationJobService.QueueJobsRaw(IVssRequestContext requestContext, IEnumerable`1 jobReferences, JobPriorityLevel priorityLevel, Int32 maxDelaySeconds, ITFLogger logger, Boolean queueAsDormant)
[12:29:58.480] at Microsoft.TeamFoundation.Server.Deploy.TFCollection.GitStepPerformer.M83PopulateCommitChanges(IVssRequestContext requestContext, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationStepPerformerBase.PerformHostStep(String servicingOperation, ServicingOperationTarget target, IServicingStep servicingStep, String stepData, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationStepPerformerBase.PerformStep(String servicingOperation, ServicingOperationTarget target, String stepType, String stepData, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.ServicingStepDriver.PerformServicingStep(ServicingStep step, ServicingContext servicingContext, ServicingStepGroup group, ServicingOperation servicingOperation, Int32 stepNumber, Int32 totalSteps)
[12:29:58.480] Step failed: Populate commit changes. Execution time: 23 milliseconds.
[12:29:58.480] [StepDuration] 0.0236576
[12:29:58.480] [GroupDuration] 0.2517195
[12:29:58.480] [OperationDuration] 0.2517302
[12:29:58.587] Clearing dictionary, removing all items.
======================================================================================================
Step execution times in descending order
======================================================================================================
Updates all rows in tbl_GitCommit and sets the Status to ... (GitToDev14M83Collection, ToDev14M83Collection) - 227 milliseconds
Populate commit changes (GitToDev14M83Collection, ToDev14M83Collection) - 23 milliseconds
Write service level to stamp (StartInstallUpdates, StartInstallUpdates) - 20 milliseconds
Configure framework servicing tokens (VsspToDev14M71Collection, VsspToDev14M71Collection) - 20 milliseconds
Setup integration environment (TestManagementToDev12M65FinalConfiguration, ToDev12M65FinalConfiguration) - 3 milliseconds
Setup Git environment (GitToDev14M74Collection, ToDev14M74Collection) - 1 millisecond
Setup Git environment (GitToDev14M83Collection, ToDev14M83Collection) - 1 millisecond
Set the collection partition id tokens in servicing context (GitToDev14M83Collection, ToDev14M83Collection) - 1 millisecond
======================================================================================================
Execution times by group in descending order
======================================================================================================
GitToDev14M83Collection (ToDev14M83Collection) - 250 milliseconds
StartInstallUpdates (StartInstallUpdates) - 20 milliseconds
VsspToDev14M71Collection (VsspToDev14M71Collection) - 20 milliseconds
TestManagementToDev12M65FinalConfiguration (ToDev12M65FinalConfiguration) - 3 milliseconds
GitToDev14M74Collection (ToDev14M74Collection) - 1 millisecond
I was pondering the fact that this is a rouge job that needs to be deleted from the databse manually but I mmight be wrong. Any pointer will be greatfully +1-ed.
The main reason for TFS being unable to upgrade team project collections usually is that their data was already either corrupted, incomplete, or stuck between schema versions.
You will need to contact customer support for assistance troubleshooting these data issues and getting your data back to a workable state.
Since there is only one project inside. If you don't care the source control history and have the back up of project. A crude (not recommend) way should be deleting the special project collection, delete the database from SQL Server and create a new collection, restore the project inside.
Also take a look at this similar question with the same error as you: Starting a team project collection after detaching and attaching again
Ultimately, there was no way to recover the collection. So I used the admin command to permanently remove the collection. In my case, n one came back to me asking where the collection go so it's solved for me. I'm running the latest TFS 2017.2 and the rest of the dev are happy.

NServiceBus - Scanning all assemblies

NServiceBus.dll - Version 5.2.9 & NServiceBus.Host - Version 6.0.0
I am developing a workflow application with pluggable addins.
In my solution I have a NServiceBus host assembly which I host using the NServiceBus.Host.exe. To prevent scanning I have defined the EndpointConfigurationType in the NServiceBus.Host.exe.config.
<appSettings>
<add key="EndpointConfigurationType" value="Libra.Workflow.Host.EndpointConfig, Libra.Workflow.Host" />
</appSettings>
I have verified that this config is being used because if I put some unknown type I get an error and also because me EndpointConfig class is instantiated before any scanning happens.
In the Customize method of this class I have added
public void Customize(BusConfiguration cfg)
{
cfg.AssembliesToScan(AllAssemblies.Matching("Libra.Workflow.Messages.dll"));
...
}
Now when I run this project I get an error because NServiceBus is scanning all assemblies and do to the nature of System.AddIn some assemblies cannot be scanned!
This scanning happens right after Libra.Workflow.Host has been instantiated but before the Customize method is called. Here is the call stack for this scan:
at NServiceBus.Hosting.Helpers.AssemblyScanner.ScanAssembly(String assemblyPath, AssemblyScannerResults results) in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Hosting\Helpers\AssemblyScanner.cs:line 153
at NServiceBus.Hosting.Helpers.AssemblyScanner.GetScannableAssemblies() in C:\BuildAgent\work\3206e2123f54fce4\src\NServiceBus.Core\Hosting\Helpers\AssemblyScanner.cs:line 63
at NServiceBus.GenericHost..ctor(IConfigureThisEndpoint specifier, String[] args, List`1 defaultProfiles, String endpointName, IEnumerable`1 scannableAssembliesFullName) in c:\BuildAgent\work\a3de8759ee491634\src\NServiceBus.Hosting.Windows\GenericHost.cs:line 33
at NServiceBus.Hosting.Windows.WindowsHost..ctor(Type endpointType, String[] args, String endpointName, IEnumerable`1 scannableAssembliesFullName) in c:\BuildAgent\work\a3de8759ee491634\src\NServiceBus.Hosting.Windows\WindowsHost.cs:line 21
at NServiceBus.Hosting.Windows.HostServiceLocator.DoGetInstance(Type serviceType, String key) in c:\BuildAgent\work\a3de8759ee491634\src\NServiceBus.Hosting.Windows\HostServiceLocator.cs:line 31
at Microsoft.Practices.ServiceLocation.ServiceLocatorImplBase.GetInstance(Type serviceType, String key) in c:\Home\Chris\Projects\CommonServiceLocator\main\Microsoft.Practices.ServiceLocation\ServiceLocatorImplBase.cs:line 49
The error message I get is:
Could not enumerate all types for
'C:\msc\Trunk\Libra.Workflow\Build\Libra.Workflow.Host\AddIns\Libra.Workflow\Libra.Workflow.Processors.dll'
Why is NServiceBus scanning this DLL and how can I prevent it?
Note: Since this is an AddIn DLL, there isn't even a reference to it in the Libra.Workflow.Host nor any other related assemblies so there should be absolutely no reason for NServiceBus to have to touch it.
One way of limiting assembly scanning done by NServiceBus.Host is to use the /scannedAssemblies switch. One caveat is to pass NServiceBus.Core and NServiceBus.Host assemblies explicitly:
NServiceBus.Host.exe /scannedAssemblies:"NServiceBus.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226c" /scannedAssemblies:"NServiceBus.Host, Version=6.0.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226c"
This command will scan those NServiceBus assemblies and the assembly specified via EndpointConfigurationType app setting. If you want to specify additional assemblies (like your Libra.Workflow.Messages) you may add additional /scannedAssemblies switch.
Please see this documentation page for details: http://docs.particular.net/nservicebus/hosting/nservicebus-host/#configuring-the-endpoint-controlling-assembly-scanning-using-the-command-line.

Obscure NHibernate/Fluent NHibernate error

I've been encountering the following error when trying to build a
session factory:
PersistenceTests.Can_Map_Orders_To_Database : Failed
System.IndexOutOfRangeException: Index was outside the bounds of the
array.
at NHibernate.Mapping.Column.set_Name(String value)
at NHibernate.Cfg.XmlHbmBinding.ClassBinder.BindColumns(XmlNode node,
SimpleValue model, Boolean isNullable, Boolean autoColumn, String
propertyPath)
at NHibernate.Cfg.XmlHbmBinding.ClassBinder.BindColumnsOrFormula
(XmlNode node, SimpleValue simpleValue, String path, Boolean
isNullable)
at NHibernate.Cfg.XmlHbmBinding.ClassBinder.BindSimpleValue(XmlNode
node, SimpleValue model, Boolean isNullable, String path)
at
NHibernate.Cfg.XmlHbmBinding.CollectionBinder.BindCollectionSecondPass
(XmlNode node, Collection model, IDictionary`2 persistentClasses)
at
NHibernate.Cfg.XmlHbmBinding.CollectionBinder.<>c__DisplayClassd.<AddCollec tionSecondPass>b__c
(IDictionary`2 persistentClasses)
at NHibernate.Cfg.Configuration.SecondPassCompile()
at NHibernate.Cfg.Configuration.BuildSessionFactory()
at FluentNHibernate.Cfg.FluentConfiguration.BuildSessionFactory() in c:
\Code Samples\NHibernate\Fluent Nhibernate - Trunk\src\FluentNHibernate
\Cfg\FluentConfiguration.cs: line 94
FluentNHibernate.Cfg.FluentConfigurationException: An invalid or
incomplete configuration was used while creating a SessionFactory.
Check PotentialReasons collection, and InnerException for more detail.
at FluentNHibernate.Cfg.FluentConfiguration.BuildSessionFactory() in c:
\Code Samples\NHibernate\Fluent Nhibernate - Trunk\src\FluentNHibernate
\Cfg\FluentConfiguration.cs: line 99
at FluentNHibernate.SessionSource..ctor(FluentConfiguration config) in
c:\Code Samples\NHibernate\Fluent Nhibernate - Trunk\src
\FluentNHibernate\SessionSource.cs: line 38
at
FluentNHibernate.Testing.SingleConnectionSessionSourceForSQLiteInMemoryTest ing..ctor
(FluentConfiguration config) in c:\Code Samples\NHibernate\Fluent
Nhibernate - Trunk\src\FluentNHibernate\Testing
\SingleConnectionSessionSourceForSQLiteInMemoryTesting.cs: line 15
at Core.Infrastructure.Data.NHibernate.Tests.PersistenceTests.SetUp()
in PersistenceTests.cs: line 26
I'm working against the FNH trunk and NH 2.0.1. Funny thing is I am
able to compile my mappings (via AutoPersistenceModel.CompileMappings)
and write them to the file system successfully - FNH doesn't complain.
It is only when attempting to build the session factory that
everything goes kaboom with the not very helpful error message above.
Anyone got any ideas?
I found that the inner exception gave more details. In my case I had to add the NHibernate.ByteCode.Castle.dll file to the references.
Fluent NHibernate itself rarely complains at you directly. Internally it's just building up your HBM files for you so if you told it to build something wrong then NHibernate proper will get grumpy.
If you're not already, you might want to start with exporting your mapping files like so:
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<MyClass>()
.ExportTo("path")
Then you can dig around in there to see if something is wrong. I do recall getting this error once before and I think it relates to a mismatch in the number of columns mapped. It was one of those easy-to-miss errors in my mapping, so unfortunately all I can suggest is to really scour the output of ExportTo for anything that doesn't make sense.
We'll need to see your Configuration to help much. But this part of the stack trace should give you an idea of where to start.
at FluentNHibernate.Cfg.FluentConfiguration.BuildSessionFactory() in c:
\Code Samples\NHibernate\Fluent Nhibernate - Trunk\src\FluentNHibernate
\Cfg\FluentConfiguration.cs: line 94
FluentNHibernate.Cfg.FluentConfigurationException: An invalid or
incomplete configuration was used while creating a SessionFactory.
Check PotentialReasons collection, and InnerException for more detail.