TFSBuild - Database Deploy - There is a file locking violation - msbuild

I have inherited a system that requires a dbschema to be deployed to multiple databases (one per client) and set up a nightly build to deploy changes to my dev tier (sample below). About 50% of the time it works great, but the other 50% I get an error somewhere in the middle of the build:
"d:\builds{Project}\Dev Nightly Database Update\BuildType\TFSBuild.proj" (EndToEndIteration target) (1) ->
"d:\builds{Project}\Dev Nightly Database Update\Sources\Database\Database.dbproj" (Rebuild;Deploy target) (3:18) ->
(DspDeploy target) ->
C:\Program Files\MSBuild\Microsoft\VisualStudio\v10.0\TeamData\Microsoft.Data.Schema.TSqlTasks.targets(120,5): error MSB4018: The "SqlDeployTask" task failed unexpectedly. [d:\builds{Project}\Dev Nightly Database Update\Sources\Database\Database.dbproj]
C:\Program Files\MSBuild\Microsoft\VisualStudio\v10.0\TeamData\Microsoft.Data.Schema.TSqlTasks.targets(120,5): error MSB4018: There is a file locking violation. Close any external application that uses the database file. [d:\builds{Project}\Dev Nightly Database Update\Sources\Database\Database.dbproj]
I have looked around but not found any real solutions. I assume that I am getting the error because msbuild is still using the dbproj file when the next iteration starts up. I tried putting a Sleep task in as a last ditch effort but that hasn't helped at all.
Any other suggestions?
<Target Name="Client DB Deploy" Outputs="%(Database.Name)">
<MSBuild Projects="$(SolutionRoot)\Database\Database.dbproj"
Properties="Configuration=Default; OutDir=$(BinariesRoot)\$(Configuration)\;
DeployToDatabase=True;TargetDatabase=%(Database.Name);
TargetConnectionString=Data Source=$(DatabaseServer)%3BIntegrated Security=True;
AlwaysCreateNewDatabase=False;BlockIncrementalDeploymentIfDataLoss=False"
Targets="Rebuild;Deploy" />
<Copy SourceFiles="$(OutDir)..\Database.sql" DestinationFolder="$(OutDir)_SetupPackages\_Database\%(Database.Name)" />
<SleepTask Timeout="10000" /> <!--sleep the build to try and avoid file locks we get in the loop-->

Try to add MSBuild task parameters RunEachTargetSeparately="True" or UnloadProjectsOnCompletion="True". See http://msdn.microsoft.com/en-us/library/z7f65y0d.aspx

Related

console application is not building in vs 2019

I am constantly this error in debug mode.
Severity Code Description Project File Line Suppression State Error MSB3027 Could not copy "C:\Users\N3617\Source\Repos\Core\CoreData\ConsoleApp1\obj\Debug\netcoreapp3.1\ConsoleApp1.exe" to "bin\Debug\netcoreapp3.1\ConsoleApp1.exe". Exceeded retry count of 10. Failed. The file is locked by: "ConsoleApp1 (1080)" ConsoleApp1 C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets 4643
I had to restart system to get rid off this error. Can anyone tell why is this happening?
It seems an old problem that your project is locked by some other process due to some reasons. You can see this similar issue.
It is complex to explain that but you can try the following steps if you faced them next time:
Suggestion
1) open Task Manager--> shut down ConsoleApp1.exe process, any dotnet process, NET Core Host process or similar process every time when you faced this issue and then build your project again.
2) close VS, delete .vs hidden folder under solution folder, bin and obj folder and then restart VS
3) enter Tools-->Options-->Projects and Solutions-->Build and Run-->set maximum numbers of parallel project builds to 1.
4) uncheck option Use Managed Compatibility Mode under Tools-->Options-->Debugging-->General

trouble publishing website TFS 2013

I have a TFS 2013 build that I'm trying to get to publish to a folder on the build server. I've installed WebDeploy, but I always get the error:
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.targets (4274): Web deployment task failed. (Could not connect to the remote computer ("localhost"). On the remote computer, make sure that Web Deploy is installed and that the required process ("Web Management Service") is started. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_DESTINATION_NOT_REACHABLE.)
I've set up a website on the build server and that's where I'm trying to publish the website. Actually I don't even care about publishing it per se - I just need my build output to go to a folder locally automatically. Right now we have to manually open the solution and choose Publish... to get the output that subsequent InstallShield builds need for input. Here are my MSBuild arguments. Does anyone have any idea what could be missing?
/p:SrcDir=C:\Builds\TFS\WebApps\Src
/p:RevKeyname=WebAppsRevNr
/p:DeployOnBuild=true
/p:DeployTarget=MsDeployPublish
/p:MSDeployServiceURL=https://127.0.0.1:8172/msdeploy.axd
/p:CreatePackageOnPublish=True
/p:DeployIisAppPath="WebApp"
/p:MsDeployPublishMethod=WMSVC
/p:AllowUntrustedCertificate=True
/p:AutoParameterizationWebConfigConnectionStrings=False
/p:Authtype=NTLM /p:username=""
I've checked both net start wmsvc and net start msdepsvc and both are running. Any ideas?
Thanks!
UPDATE
I've tried everything that Andy suggested and now when I run this from the command line I get this bizarre error message:
"C:\Workspace\VS2013\WebApps\Main\Src\webapps.sln" (default target) (1) ->
"C:\Workspace\VS2013\WebApps\Main\Src\CoreWebApps\CoreWebApps.csproj"
(default
target) (7) ->
(AutoParameterizationWebConfigConnectionStringsCore target) ->
C:\Program Files
(x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Microsoft.Web
.Publishing.targets(2295,5): error : Could not open Source file: Could not
find
a part of the path
'C:\Workspace\VS2013\WebApps\Main\Src\CoreWebApps\Areas\Adm
in\Views\Web.config;Areas\Admin\Views\Web.config'.
[C:\Workspace\VS2013\WebApps
\Main\Src\CoreWebApps\CoreWebApps.csproj]
Any idea why it's looking for Web.config;Areas\Admin\Views\Web.config'.? That makes no sense.
Please try below items to narrow down the issue:
Try to use IP or Machine Name instead of "localhost"
Logon your build agent machine, then manually execute the same
MSBuild command within the same arguments(which you provided in build
definition) to build and deploy your solution, then check result. You
need to ensure you can manually run the same MSBuild command within
deploy argument to build and deploy your solution successfully from
build agent machine. Then use the same deploy arguments in TFS Build
definition.
Double check Web Deploy settings to make sure that the name of the
website is exactly that of what's in IIS.
Install Web Management Tools before Web
Deploy : Install the Web Management Services (Roles -> Web Server >
Management Tools > Management Services). Then uninstall Web Deploy, and then install Web Deploy again.
You can also reference this thread for your troubleshooting.
Update:
For the issue "Could not open Source file: Could not find a part of the path" you can reference below similar articles for the troubleshooting.
https://social.msdn.microsoft.com/Forums/en-US/8f959964-c951-4f9a-8486-8283a925c9f6/build-error-could-not-open-source-file-though-i-know-it-exists?forum=windowsazurewebsitespreview
https://our.umbraco.org/forum/getting-started/installing-umbraco/60222-Umbraco-721-Build-fails-after-deploy-to-Azure-WebSite (See the last two answers)

WebDeploy runcommand issue

I'm trying to deploy a package with WebDeploy V3.
The installation process is to sync between a source folder to a destination folder on the remote computer and run a certain powershell script after the sync is done.
The command being executed is:
'"C:\Program Files\IIS\Microsoft Web Deploy V3\msdeploy.exe" -verb:sync -source:dirPath='C:\source' -dest:dirPath='D:\destination',computerName=XXX -postSync:runcommand='powershell -inputformat none D:\destination\Install.ps1',successReturnCodes=0'
This yields the following error:
Info: Using ID '49edd786-d8a0-4acf-be7b-95dd6e1391cc' for connections to the remote server. Performing '-postSync'... Info:
Using ID '5ef9d005-82fa-4811-9f51-1741c8d622de' for connections to the remote server.
Info: Adding MSDeploy.runCommand (MSDeploy.runCommand).
Error: (11/28/2012 4:34:24 AM) An error occurred when the request was processed on the remote computer. Error: The entry type 'Unknown' was not expected at this time. The serialization stream may be corrupted.
Error count: 1.
Error during '-postSync'. Total changes: 0 (0 added, 0 deleted, 0 updated, 0 parameters changed, 0 bytes copied)
Searching the net for this error, I didn't see anybody who encountered it when using runcommand provider. If anybody encountered a similar issue and has ideas or suggestions I would be most thankful..
From what I've seen, using runCommand to execute an arbitrary command line might be a bit buggy. Try moving the commandline into a bat or cmd file and providing a (full?) path to that. The file will be uploaded and executed, as long as you don't try to pass in any arguments to it.
For future viewers of this post: I encountered this same specific error (Error: The entry type 'Unknown' was not expected at this time. The serialization stream may be corrupted) after adding runCommand provider usage to my MyProject.wpp.targets file for the Web Publishing Pipeline MSBuild process. The path was direct cmd shell input used in order to clear readonly flags with attrib -R.
In my case, my build server was configured with WebDeploy 3.0, while the server targeted by the deployment package was configured with Webdeploy 2.0. After upgrading the target server to Webdeploy 3.0 this particular problem was resolved.
However due to other errors surrounding runCommand (providing the correct path to the destination executable at package runtime) my solution still doesn't work entirely so take this all with a grain of salt.

The mystery of stuck inactive msbuild.exe processes, locked Stylecop.dll, Nuget AccessViolationException and CI builds clashing with each other

Observations:
On our Jenkins build server, we were seeing lots of msbuild.exe processes (~100) hanging around after job completion with around 20mb memory usage and 0% CPU activity.
Builds using different versions of stylecop were intermittently failing:
workspace\packages\StyleCop.MSBuild.4.7.41.0\tools\StyleCop.targets(109,7):
error MSB4131: The "ViolationCount" parameter is not supported by the "StyleCopTask" task.
Verify the parameter exists on the task, and it is a gettable public instance property.
Nuget.exe was intermittently exiting with the following access violation error (0x0000005):
.\workspace\.nuget\nuget install .\workspace\packages.config -o .\workspace\packages"
exited with code -1073741819.
MsBuild was launched in the following way via a Jenkins Matrix job, with 'BuildInParallel' enabled:
`msbuild /t:%Targets% /m
/p:Client=%Client%;LOCAL_BUILD=%LOCAL_BUILD%;BUILD_NUMBER=%BUILD_NUMBER%;
JOB_NAME=%JOB_NAME%;Env=%Env%;Configuration=%Configuration%;Platform=%Platform%;
Clean=%Clean%; %~dp0\_Jenkins\Build.proj`
After a lot of digging around and trying various things to no effect, I eventually ended up creating a new minimal solution which reproduced the issue with very little else going on. The issue turned out to be caused by msbuild's multi-core parallelisation - the 'm' parameter.
The 'm' parameter tells msbuild to spawn "nodes", these will remain alive after the build has ended, and are then re-used by new builds!
The StyleCop 'ViolationCount' error was caused by a given build re-using an old version of the stylecop.dll from another build's workspace, where ViolationCount was not supported. This was odd, because the CI workspace only contained the new version. It seems that once the StyleCop.dll was loaded into a given MsBuild node, it would remain loaded for the next build. I can only assume this is because StyleCop loads some sort of singleton into the nodes processs? This also explains the file-locking between builds.
The nuget access violation crash has now gone (with no other changes), so is evidently related to the above node re-use issue.
As the 'm' parameter defaults to the number of cores - we were seeing 24 msbuild instances created on our build server for a given job.
The following posts were helpful:
msbuild.exe staying open, locking files
http://www.hanselman.com/blog/FasterBuildsWithMSBuildUsingParallelBuildsAndMulticoreCPUs.aspx
http://stylecop.codeplex.com/discussions/394606
https://github.com/Glimpse/Glimpse/issues/115
http://msdn.microsoft.com/en-us/library/vstudio/ms164311.aspx
The fix:
Add the line set MSBUILDDISABLENODEREUSE=1 to the batch file which launches msbuild
Launch msbuild with /m:4 /nr:false
The 'nr' paremeter tells msbuild to not use "Node Reuse" - so msbuild instances are closed after the build is completed and no longer clash with each other - resulting in the above errors.
The 'm' parameter is set to 4 to stop too many nodes spawning per-job
I had the same issue. One old reference I found was in csproj files
<PropertyGroup>
<StyleCopMSBuildTargetsFile>..\packages\StyleCop.MSBuild.4.7.48.0\tools\StyleCop.targets</StyleCopMSBuildTargetsFile>
Also, I deleted the entire "Packages" folder that's located in the same folder as sln file after I closed the visual studio. It triggered VS to rebuild the folder and let go of the cache of the old version of stylecop
I've had the same issue for a while, builds were taking over 6 minutes to finish after some digging I found our it's node reuse fault so adding /m:4 /nr:false fixing my issue immediately

How can I remotely start/stop a service using MSBuild?

I'd like to remotely start or stop a windows service on another machine using MSBuild. To accomplish this, I wrote this script:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="C:\Program Files (x86)\MSBuild\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<Target Name="MyTarget">
<ServiceController MachineName="Box2" ServiceName="MyService" Action="Stop" />
</Target>
</Project>
When I run that on a machine that can see Box2, I get this:
Project
"C:\Scripts\Test.xml" on node 1 (default
targets).
C:\Scripts\Test.xml(4,5): error : Couldn't
find the 'MyService'
service on 'Box2' Done Building
Project
"C:\Scripts\Test.xml" (default targets) --
FAILED.
I know that I have the service name correct (I copied and pasted it from the actual service list), and I'm pretty sure that it can see Box2 because if I change it to a machine name that doesn't exist (e.g. Box2asdf), it takes about 10 seconds to come back (with the exact same error, mind you), as opposed to the nearly immediate response that I get when I provide the correct machine name.
How might I debug this issue?
You might try this instead...
You can use the command line program sc and execute that...
ie
SC \ServerName stop ServiceName
http://support.microsoft.com/kb/166819
For more information on how to execute a command from msbuild check this out..
execute a command with parameters using msbuild
The community tasks should work. Just use Sc query to check that the service does work. as for using msbuild its still using msbuild if you wrap sc in an exec?
At least you dont have a dependency on a third party dll in your build process.
ServiceController Target internally uses ServiceController Class. But it doesn't return the reason why it couldn't find the service. If you are shure that both computer and service names are correct, the next thing I can suggest to analyze is access violation problems.
And #jsobo's answer can be very useful to diagnose the actual reason because it can show native errors without .Net exception wrappers around them:
sc.exe \Box2 stop MyService