MSBuild Issue while deploying files - msbuild

I am deploying some files on the server. But when I am doing this, build is deleting all the files and folder which are residing at that location. I don't want to delete all the files from the server. I want to exclude one folder (folder name is Temp) from the destination folder. Temp folder should not get deleted while deleting other files. How to do that?
Here is TFS Build Definition
<PropertyGroup Condition=" '$(DeployEnvironment)' == 'Dev' ">
<DeployPath>\\server1\D$\temp\reports</DeployPath>
</PropertyGroup>
<Target Name="CoreCompileSolution" />
<Target Name="AfterCompile">
<Message Importance ="high" Text="Solution Root: $(SolutionRoot)" />
<Message Importance ="high" Text="Out Dir: $(OutDir)" />
<Copy SourceFiles="#(RPTFiles)" DestinationFolder="$(OutDir)_PublishedWebsites\Reports\" />
</Target>
<Target Name="AfterDropBuild" >
<CreateItem Exclude="$(DeployPath)\Temp*.*">
<Output ItemName="PreviousDeployment" TaskParameter="Include" />
</CreateItem>
</Target>

Why are you using a Copy task? I think it is intended to be used for local manipulations during build, rather than deployment (because it does not give you a chance to easily configure behaviour).
I suggest that instead of copy tsak you use one of the following options
Non-web applications - use Robocopy:
/XD dirs [dirs]... : eXclude Directories matching given names/paths.
XF and XD can be used in combination e.g.
ROBOCOPY c:\source d:\dest /XF *.doc *.xls /XD c:\unwanted /S
see this link for usage guide. You either run it from the command line (using <Exec Command="" > task, or employ MBuiild Community Tasksproject which has a nice wrapper.
Web applications: you should use Web Deploy for your deployments. You an either use MSBuild integration (VS 2010 and later, see this blog series for guidance on setup and configure on VS2010 NB: it has been much simplified in VS 2012, but I don't have a link to share at the moment) or run it from command line (prior to VS 2010):
<Exec Command=""$(WebDeployToolPath)" -verb:sync - source:dirPath='$(MSBuildProjectDirectory)\Published\' -dest:dirPath='$(DeployDirectoryLocalPath)',computerName=$(DeployTargetURL),userName='$(DeployUserName)',password='$(Password)',authType='Basic' -skip:skipaction='Delete',objectname='filePath',absolutepath='app_offline.htm' -skip:skipaction='Delete',objectname='filePath',absolutepath='logs\\.*' -skip:skipaction='Delete',objectname='dirPath',absolutepath='logs\\.*' -skip:skipaction='Delete',objectname='filePath',absolutepath='UserFiles\\.*' -skip:skipaction='Delete',objectname='dirPath',absolutepath='UserFiles\\.*' -verbose -allowUntrusted" />
NB using skip:skipaction='Delete.. to skip removing files and folders.
Update
It looks like I've undestood this a bit incorrect (I supposed, deployment happenned in AfterCompile target, however, as I see now, TFS uses CoreDropBuild target to do the deployment.
So I think, what you need is to override CoreDropBuild target as described: here. (although, I've never tried this).
You can either use Copy task as the author of the thread, or go with Robocopy/webdeploy based on your personal preference.

Related

Build script to copy files from various source folder to various destination folder

The basic requirement is to copy the various files and folders from different solution/project directories to the single build_output folder(/subfolders).
Currently, I am doing this operation using the Robocopy commands. The only issue is my script is too long just using multiple Robocopy commands.
<Copy SourceFiles="$(Docs)\Manual.pdf" DestinationFolder="$(BuildPath)\Help"/>
<RoboCopy Source="$(Web1)" Destination="$(BuildPath)" Files="*.aspx" Options="/E"/>
<RoboCopy Source="$(Web1)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<RoboCopy Source="$(Web2)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<!--- 100s of such RoboCopy & Copy commands (note that in last two commands i need to copy from different sources to same destination -->
How this job is implemented in real enterprise applications, so that
the build script is concise and clear.
Is my thinking below is the way to approach the solution. If yes, can anybody provide me sample steps using MSBuild or CommandScript easily. (free to use any MSBuild extensions)
Define the mapping of the all source folders, file types (can be xyz.png/.png/.*) and the destination path.
Copy the files (Robocopy) using the above mentioned mappings using a single target or task)
Is there any other better way to do this problem?
Insights/Solution ???
I do exactly this sort of thing to stage build output for harvesting by the installer build. I have a custom targets file for consistent processing and have some msbuild property files with the item groups describing that needs to be done.
<ItemGroup Label="AcmeComponent1Payload">
<FileToHarvest Include="$(SourceRoot)AcmeProjects\ServerManager\$(Configuration)\**\*;
$(SourceRoot)Library\SQLServerCompact\**\*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.*"
Exclude="$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\IntegrationTests.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Practices.*.xml;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\obj\**\*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.VisualStudio.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Web.*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.xml">
<Group>AcmeServerManager</Group>
<SubDir>Utilities\</SubDir>
</FileToHarvest>
</ItemGroup>
The custom targets file has the functionality to process it.
<Target Name="CopyFiles">
<Copy Condition="#(FileToHarvest)!=''"
SourceFiles="#(FileToHarvest)"
DestinationFiles="#(FileToHarvest->'$(OutputPath)\%(Group)\%(SubDir)%(RecursiveDir)%(Filename)%(Extension)')"
OverwriteReadOnlyFiles="true"
SkipUnchangedFiles="true" />
</Target>
You can make the properties file as simple or as complicated as you like. I use multiple ones and import them into the project file using wildcards.
Thanks #daughey, I got my first part working, where I need to copy from different sources to the same destination.
<!--Declare an ItemGroup that points to source Locations-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit"/>
<ItemToCopy Include="$(Utilities)\Service"/>
<ItemToCopy Include="$(Web1)\NET"/>
</ItemGroup>
<!--Declare an ItemGroup that points to destination Locations-->
<ItemGroup>
<DestLocations Include="$(BuildPath)" />
</ItemGroup>
<Target Name="CopyFiles">
<!-- Run the copy command to copy the item to your dest locations-->
<!-- The % sign says to use Batching. So Copy will be run for each unique source ItemToCopy(s) in the DestLocation.-->
<RemoveDir Directories="$(BuildPath)"/>
<Message Importance="high" Text="Deploy folder is $(BuildPath)"/>
<RoboCopy Source="%(ItemToCopy.FullPath)" Destination="$(BuildPath)" Files="*.dll"/>
</Target>
After struggling with Item Metadata for Task Batching, the second part is also working great.
Scenario: Copy files from list of source directories to output directory on their respective sub-folders
$(Web1)\Audit*.dll => $(BuildPath)\Audit*.dll
$(Utilities)\Service*.jpg => $(BuildPath)\Utilities\Service*.jpg
Solution
<!--Modify an ItemGroup with metadata-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit">
<ToPath>$(BuildPath)\Audit</ToPath>
<FileType>*.dll</FileType>
</ItemToCopy>
<ItemToCopy Include="$(Utilities)\Service">
<ToPath>$(BuildPath)\Utilities\Service</ToPath>
<FileType>*.jpg;*.bmp</FileType>
</ItemToCopy>
</ItemGroup>
<Target Name="CopyBatch">
<RoboCopy Source="%(ItemToCopy.Identity)" Destination="%(ItemToCopy.ToPath)" Files="%(ItemToCopy.Filetype)"/>
</Target>

.NET Core 2.1 : How to trigger Copy task defined in the project file during debugging in the Visual Studio 2017?

There are some files residing in other directories that, I would like to copy to project folder automatically before build and publishing.
After some research and experimentation, I have come up with the following .csproj file.
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RuntimeFrameworkVersion>2.1.4</RuntimeFrameworkVersion>
<TieredCompilation>true</TieredCompilation>
<PreserveCompilationContext>true</PreserveCompilationContext>
</PropertyGroup>
<ItemGroup>
<APIDefinition Include="D:\SomePlace\*.API.*.yaml" />
</ItemGroup>
<Target Name="CopyFiles" BeforeTargets="Compile;Build;Publish">
<Copy SourceFiles="#(APIDefinition)" DestinationFolder="wwwroot" />
<Copy SourceFiles="D:\SomePlaceElse\BaseAPISettings.json" DestinationFolder="$(MSBuildProjectDirectory)" />
</Target>
<ItemGroup>
<Compile Remove="wwwroot\**\*;node_modules;bower_components" />
<None Update="**.user;**.vspscc">
<CopyToPublishDirectory>Never</CopyToPublishDirectory>
</None>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.4" />
</ItemGroup>
</Project>
Here I have defined CopyFiles target, which should be run before the targets I've placed there. This target uses Copy task to copy YAML format API definition files and base API settings to the project directory.
This works well during build, publish etc. Also, if I delete the local file in the IDE, it instantly recopies it from the source.
Sometimes I make changes to these files between debugging sessions. Then, when I start debugging from Visual Studio, since the project files aren't changed, obviously the already built project is run.
Since the project is not built, my copy tasks are not triggered, and I end up with stale files during debuging.
Is there anything I can do to have my Copy tasks triggered, when I do "Start Debugging F5" in the IDE, regardless of the project build state ?
P.S. : I'm using Visual Studio 2017 15.8.5 and targeting .NET Core 2.1.4 runtime, if it makes any difference.
To integrate fully into the up-to-date check of the project system inside Visual Studio, I susggest the following changes:
Make the items' source and target paths known before
Register them to the up-to-date check system. (Also needs a hack to make sure the project source code is recompiled so that the output will have a newer time stamp)
Make the MSBuild target itself incremental. This also helps for command-line builds when the files don't have to be copied.
The complete changes look like this:
<ItemGroup>
<CustomCopyFile Include="..\TestFiles\*.API.*.yaml"
TargetPath="wwwroot\%(Filename)%(Extension)" />
<CustomCopyFile Include="..\TestFiles\BaseAPISettings.json"
TargetPath="%(Filename)%(Extension)" />
<UpToDateCheckInput Include="#(CustomCopyFile)" />
<UpToDateCheckBuild Include="#(CustomCopyFile->'%(TargetPath)')"
Original="#(CustomCopyFile)" />
<CustomAdditionalCompileInputs Include="#(CustomCopyFile->'%(TargetPath)')" />
</ItemGroup>
<Target Name="CopyFiles"
BeforeTargets="BeforeBuild;BeforePublish"
Inputs="#(CustomCopyFile)"
Outputs="#(CustomCopyFile->'%(TargetPath)')">
<Copy SourceFiles="#(CustomCopyFile)"
DestinationFiles="#(CustomCopyFile->'%(TargetPath)')" />
</Target>
CustomCopyFile now collects all the source files and we put the expected destination file name into the TargetPath metadata.
UpToDateCheckInput items tell Visual Studio to rebuild the project if one of these items change.
UpToDateCheckBuild items instruct Visual Studio to only check these items against special source items. This is redundant for this example project but may be helpful if the target path wasn't inside the project directory but some intermediate output (obj..) folder and no re-evaluation would see these new files. It would also be helpful if the files were also modified as part of processing (e.g. replacing variables inside the files).
CustomAdditionalCompileInputs is a hack here since the items are copied to the project folder and are considered to be "inputs to the output" automatically.. So we force the project to recompile if our source files change. If we don't do so, it would never consider the project up-to-date after a change to the source yaml files since they would be newer than the compiled app.dll file.

Why is MSBuild trying to run the projects it's building?

I'm currently writing an msbuild script to build a solution I've been working on, as well as run its tests. On my development machine, this works as expected. However, when I try to run the same build script on our build server, I get several failures. I've tracked the source of the problem down to the fact that my build script appears to be trying to run the .exe file associated with my application. This line during the script execution tipped me off, since it doesn't run that command on my dev box:
MSIAuthoring:
Building MSI
"C:\Program Files (x86)\Jenkins\workspace\Test Build\BuildArtifacts\MsiBuildTool.exe" "/MBSBUILD:MsiBuildTool"
I'm fairly new to build scripting, but my understanding is that the build script shouldn't be trying to run my program unless I explicitly tell it to do so. Does anyone know what might be causing this?
For reference, here is my build script:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="RunTests"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<BuildArtifactsDir Include="BuildArtifacts\"/>
<SolutionFile Include="MsiBuildTool.sln"/>
<NUnitConsole Include="C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe"/>
<UnitTestsDll Include="BuildArtifacts\MsiBuildToolUnitTests.dll"/>
<TestResultsPath Include="BuildArtifacts\TestResults.xml"/>
</ItemGroup>
<PropertyGroup>
<Configuration Condition="'$(Configuration)' == ''">Release</Configuration>
<Platform Condition="'$(Platform)' == ''">Any CPU</Platform>
</PropertyGroup>
<Target Name="Init" DependsOnTargets="Clean">
<MakeDir Directories="#(BuildArtifactsDir)"/>
</Target>
<Target Name="Clean">
<RemoveDir Directories="#(BuildArtifactsDir)"/>
</Target>
<Target Name ="Compile" DependsOnTargets="Init">
<MSBuild Projects="#(SolutionFile)"
Targets ="Build"
Properties ="OutDir=%(BuildArtifactsDir.FullPath);Configuration=$(Configuration);Platform=$(Platform)"/>
</Target>
<Target Name="RunTests" DependsOnTargets="Compile">
<Exec Command='"#(NUnitConsole)" #(UnitTestsDll) /xml=#(TestResultsPath)'/>
</Target>
</Project>
Update:
After some digging through the output, I found that "MSIAuthoring" step was the result of the Wix# library that I'm using. As described by this thread: https://wixsharp.codeplex.com/discussions/644609#
I disabled the MSIAuthoring step by removing this line in my .csproj files:
<Import Project="..\packages\WixSharp.1.0.22.3\build\WixSharp.targets" Condition="Exists('..\packages\WixSharp.1.0.22.3\build\WixSharp.targets')" />
You're building solution file, thus MSBuild will generate msbuild-xml script first and then will build it. To find why it's being called on build machine but not on your dev machine - follow this advice and obtain generated MSBuild scripts from your dev environment and your build server. Then compare it.
Also enable diagnostic logging (/verbosity:diag in the command line) as Lex Li advised, and you'll see detailed decisions why each target being run or not - grep logs for something like "Conditions A, B, C on target BuildMSI evaluated to False" and this will show you the difference between environments.
It might be some type of post-build script on one of the projects which builds MSI only if it's being run not on dev environment - check actual build script to find where it comes from. Also check that it's really related to your build script, and it's not an extra build step in your Jenkins build configuration.

MSBuild doesn't respect PublishUrl property for my ClickOnce app

I'm trying to make a batch file to publish the few ClickOnce application we have in one click. I'm using msbuild for that, and as an example the below command line shows how I'm doing it:
msbuild
MyApp.sln
/t:Publish
/p:Configuration=Release
/p:PublishUrl="C:\Apps\"
/v:normal > Log.txt
(wrapped for easier reading)
when I run the above command it builds and publish the application in the release directory, i.e. bin\release! Any idea why msbuild doesn't respect PublishUrl property in my example above?
PS: I tried also different combinations including remove 'Configuration', use 'Rebuild' and 'PublishOnly' as targets, and remove the the quotation marks but without any success.
You are setting the wrong property. Try PublishDir instead.
You can pass it into MSBuild as you are or you can set it in the project file (or maybe the sln file too, not sure I always use the project file.) like this
<PropertyGroup>
<PublishDir>C:\Dev\Release\$(BuildEnvironment)\</PublishDir>
</PropertyGroup>
I've just done a few blog posts on MsBuild and ClickOnce stuff, check it out you 'should' find them useful...
Some features are done by Visual-Studio and not by the MSBuild-script. So the click-once-deployment behaves differently when it's executed from the command-line.
The ApplicationRevision isn't increased with every build. This works only when is exectued from Visual Studio
In in somecases, the PublishUrl isn't used. Quote from MSDN:
For example, you could set the PublishURL to an FTP path and set the InstallURL to a Web URL. In this case, the PublishURL is only used in the IDE to transfer the files, but not used in the command-line builds. Finally, you can use UpdateUrl if you want to publish a ClickOnce application that updates itself from a separate location from which it is installed.
I've created a special MSBuild-file which does this things. It runs the publish-target and copies then the files to the right location.
An example of the build-file, as requested by alhambraeidos. It basically runs the regular VisualStudio-build and then copies the click-once data to the real release folder. Note that removed some project-specific stuff, so it's maybe broken. Furthermore it doesn't increase the build-number. Thats done by our Continues-Build-Server:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Publish" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<!-- the folder of the project to build -->
<ProjLocation>..\YourProjectFolder</ProjLocation>
<ProjLocationReleaseDir>$(ProjLocation)\bin\Release</ProjLocationReleaseDir>
<ProjPublishLocation>$(ProjLocationReleaseDir)\app.publish</ProjPublishLocation>
<!-- This is the web-folder, which provides the artefacts for click-once. After this
build the project is actually deployed on the server -->
<DeploymentFolder>D:\server\releases\</DeploymentFolder>
</PropertyGroup>
<Target Name="Publish" DependsOnTargets="Clean">
<Message Text="Publish-Build started for build no $(ApplicationRevision)" />
<MSBuild Projects="$(ProjLocation)/YourProject.csproj" Properties="Configuration=Release" Targets="Publish"/>
<ItemGroup>
<SchoolPlannerSetupFiles Include="$(ProjPublishLocation)\*.*"/>
<SchoolPlannerUpdateFiles Include="$(ProjPublishLocation)\Application Files\**\*.*"/>
</ItemGroup>
<Copy
SourceFiles="#(SchoolPlannerSetupFiles)"
DestinationFolder="$(DeploymentFolder)\"
/>
<Copy
SourceFiles="#(SchoolPlannerUpdateFiles)"
DestinationFolder="$(DeploymentFolder)\Application Files\%(RecursiveDir)"
/>
<CallTarget Targets="RestoreLog"/>
</Target>
<Target Name="Clean">
<Message Text="Clean project:" />
<MSBuild Projects="$(ProjLocation)/YourProject.csproj" Properties="Configuration=Release" Targets="Clean"/>
</Target>
</Project>
I'll put in my 2 cents, this syntax seems to work (right or wrong):
/p:publishUrl="C:\\_\\Projects\\Samples\\artifacts\\Web\\"
For me, the soultion was to escape the path.
Instead of:
/p:PublishUrl="C:\Apps\"
Put:
/p:PublishUrl="C:\\Apps\\"

MSBuild and _PublishedWebsites

After MSbuild has built my solution (with an asp.net website), and the webdeployment project has built and put the website in the directory _PublishedWebsites:
c:\mybuilds\buildName\Daily_20090519.3\Release_PublishedWebsites\MyWebsite.
How do I copy this to the fixed directory where IIS points to for the test website?
I have found loads of code snippets, but I cannot seem to find one that will take into account the fact that this directory name changes.
This is pretty easy. You can edit the project and insert something similar to the following.
<PropertyGroup>
<OutputDest>$(MSBuildProjectDirectory)\..\OutputCopy\</OutputDest>
</PropertyGroup>
<Target Name="AfterBuild">
<!-- Create an item with all the output files -->
<ItemGroup>
<_OutputFiles Include="$(OutputPath)**\*" Exclude="$(OutputPath)obj\**\*" />
</ItemGroup>
<!-- You probably don't want to include the files in the obj folder so exclude them. -->
<Message Text="OutputDest : $(OutputDest)" />
<Copy SourceFiles="#(_OutputFiles)"
DestinationFiles="#(_OutputFiles->'$(OutputDest)%(RecursiveDir)%(Filename)%(Extension)')"/>
</Target>
Is this what you are looking for?
My Book: Inside the Microsoft Build Engine : Using MSBuild and Team Foundation Build
I'm using different technique.
<PropertyGroup>
<BinariesRoot>c:\BinariesForIis\</BinariesRoot>
</PropertyGroup>
The c:\BinariesForIis\ will be used for direct output compiled binaries (before copy to ...\Daily_20090519.3\Release_ ...).