Deploying from MSBuild without overwriting specific files - msbuild

So here's what I want to do:
I want a build script that will xcopy deploy build outputs for a legacy winform app to a given directory. I want to specify a list of files to not overwrite (some config files).
I would rather have the list of files to not overwrite be passed as a parameter than hard code them.
This seems to be really unexpectedly hard. Here's what I have so far:
<!-- A property that is passed a semicolon delimited list of file names -->
<PropertyGroup>
<ProtectedFiles/>
</PropertyGroup>
<--! An ItemGroup to pick up the files>
<ItemGroup>
<FilesToDelete Include=$(DeploymentTargetFolder)\*.* Exclude="#(ProtectedFiles->'$(DeployTargetFolder)\%(identity)')"
<ItemGroup/>
<--! the delete isn't working, so I will stop just with that to keep the code brief -->
<Delete Files="#(FilesToDelete)"/>
The delete just ignores the exclude files and deletes everything
Is there a better way to do this? It doesn't seem too crazy -- I just want to
Delete all files from the target directory, except for the config files
Copy all of the files from the build outputs to the target directory, without overwriting the config files.

The first problem with your particular markup appears to confuse MsBuild $(properties) with MsBuild %(items) and MsBuild #(itemgroups).
ProtectedFiles is a property:
<!-- A property that is passed a semicolon delimited list of file names -->
<PropertyGroup>
<ProtectedFiles/>
</PropertyGroup>
But it's being treated as an Item and wouldn't have any %item.metadata:
<--! An ItemGroup to pick up the files>
<ItemGroup>
<FilesToDelete Include=$(DeploymentTargetFolder)\*.* Exclude="#(ProtectedFiles->'$(DeployTargetFolder)\%(identity)')"
<ItemGroup/>
Save the following markup locally as "foo.xml", then call "msbuild.exe foo.xml" and observe the output:
<Project ToolsVersion="4.0" DefaultTargets="foo" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<FilesProp>FileA.txt;FileB.txt</FilesProp>
</PropertyGroup>
<ItemGroup>
<ProtectedFiles Include="FileA.txt" />
<ProtectedFiles Include="FileA.txt" />
</ItemGroup>
<Target Name="foo">
<Message Importance="high" Text="ProtectedFiles ItemGroup: #(ProtectedFiles)" />
<Message Importance="high" Text="ProtectedFiles ItemGroup transform: #(ProtectedFiles->'%(Identity)')" />
<Message Importance="high" Text="FilesProp Property: $(FilesProp)" />
<Message Importance="high" Text="FilesProp Property: #(FilesProp->'%(FilesProp.Identity)')" />
</Target>
</Project>
Will yield the following output:
foo:
ProtectedFiles ItemGroup: FileA.txt;FileA.txt
ProtectedFiles ItemGroup transform: FileA.txt;FileA.txt
FilesProp Property: FileA.txt;FileB.txt
FilesProp Property:
If you're unable to change the design and need to convert a Property comprising a semi-colon delimited list of file paths, use the MsBuild <CreateItem /> task.
Add this markup to foo.xml occurring after the Foo target, then invoke msbuild again, but using the "bar" target (e.g. msbuild.exe foo.xml /t:bar)
<Target Name="bar">
<CreateItem Include="$(FilesProp)">
<Output TaskParameter="Include" ItemName="TheFiles"/>
</CreateItem>
<Message Text="TheFiles ItemGroup: #(TheFiles)" Importance="high" />
<Message Text="Output each item: %(TheFiles.Identity)" Importance="high" />
</Target>
Will yield the following output:
bar:
TheFiles ItemGroup: FileA.txt;FileB.txt
Output each item: FileA.txt
Output each item: FileB.txt
Next you should rethink some of your assumptions. I don't believe the file extension should be the determining factor when deciding which files to update, rather you should rely on MsBuild's ability to build tasks incrementally allowing it to perform a task only if the inputs are newer than the outputs. You can do this by using an MsBuild <Copy /> task configured to skip unchanged files.
Add this markup to the above Xml file, then modify the $(SourceFolder) and $(TargetFolder) to point to a source folder you'd like to copy recursively, and a destination folder to place the files. Build using "msbuild.exe foo.xml /t:Deployment" and observe the output.
<Target Name="Deployment">
<PropertyGroup>
<SourceFolder>c:\sourcefolder\</SourceFolder>
<TargetFolder>c:\destinationfolder\</TargetFolder>
</PropertyGroup>
<CreateItem Include="$(SourceFolder)\**\*.*">
<Output TaskParameter="Include" ItemName="FilesToCopy" />
</CreateItem>
<Copy SourceFiles="#(FilesToCopy)" DestinationFolder="$(TargetFolder)%(RecursiveDir)" SkipUnchangedFiles="true" />
</Target>
Without modifying any of the source files, run the command again and note that no files were copied.
Modify a file in the source folder, then run the command again. Notice that only the updated files were copied?
I hope this gets you on the right track.

There seems to be an already existing post, similar to this. Please check this Trying to exclude certain extensions doing a recursive copy (MSBuild)

Related

In MSBuild, is there a way to extract part of a path and save that for use [duplicate]

I use the following to get a list of project files that need to be compiled. Each project is stored in a subdirectory of the projects directory.
<ItemGroup>
<dprs Include="c:\projects\**\*.dpr" />
</ItemGroup>
Is there a task that I can use to extract to extract the directory that each project file is in? I know I can write my own task to do this but I was hoping that one already exists and that I simply have not found it yet.
If I understand the question correctly, you shouldn't need a task - you can do this with well-known meta data. Does this do the trick?
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
<ItemGroup>
<dprs Include="c:\projects\**\*.dpr" />
</ItemGroup>
<Target Name="Default">
<CreateItem Include="%(dprs.RelativeDir)">
<Output ItemName="_ProjectFileLocations" TaskParameter="Include" />
</CreateItem>
<Message Text="#(_ProjectFileLocations->'%(FullPath)', '%0D%0A')" />
</Target>
</Project>
From the tests I ran, it shouldn't list a directory twice in the new item group.

Remove Files and Folders Copied From AfterBuild Target

I would like to avoid hard coding the dll and folder names in the AfterClean target, is there a dynamic way to do this? Ideally it would only delete the files and folders created by the Copy in the AfterBuild target.
I tried to simplify this by changing the DestinationFolder to include a subdirectory in the OutputPath. The AfterClean target would only have to remove that subdirectory at this point. However, some of the library's DLLImport paths don't take that subdirectory into consideration which results in a crash.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="AfterBuild">
<ItemGroup>
<NativeLibs Include="$(MSBuildThisFileDirectory)..\lib\native\**\*.*" />
</ItemGroup>
<Copy SourceFiles="#(NativeLibs)" DestinationFolder="$(OutputPath)\%(RecursiveDir)" />
</Target>
<Target Name="AfterClean">
<Delete Files="$(OutputPath)\LumiAPI.dll" />
<Delete Files="$(OutputPath)\LumiCore.dll" />
<Delete Files="$(OutputPath)\LumiInOpAPI.dll" />
<RemoveDir Directories="$(OutputPath)\SPM" />
<RemoveDir Directories="$(OutputPath)\plugin" />
</Target>
</Project>
Project Structure:
src
ConsumingProject
ConsumingProject.csproj
ConsumingProject.sln
packages
my-project.5.7.0.12
build
lib
native
plugin
VenusDvc.dll
SPM
sSPM_1.bin
LumiAPI.dll
LumiCore.dll
LumiInOpAPI.dll
net45
my-project.5.7.0.12.nupkg
Essentially I want to delete all the files and folders that were copied from the native folder to the output of the project (ie LumiAPI.dll, LumiCore.dll, SPM (folder), eSPM_1.bin, etc). However I want it to be generic enough so that if I add another folder to the native directory, it will delete those folders/files as well.
Use a seperate target which lists input and output files, then use that list in both other targets. Note this uses the DestinationFiles attribute from the Copy task instead of DestinationFolders. And it might print some messages about non-existing directories being passed to RemoveDir because the top directory gets removed already before child directories.
update since you don't want to remove the root output directory as it still has files, figured applying the 'only remove output directory if it's empty' principle for any destination directory is probably the safest way to go. Credits go to the answer here.
<Target Name="GetMyOutputFiles">
<ItemGroup>
<NativeLibs Include="$(MSBuildThisFileDirectory)..\lib\native\**\*.*" />
<!--Now add some metadata: output dir and output file-->
<NativeLibs>
<DestinationDir>$(OutputPath)\%(RecursiveDir)</DestinationDir>
<Destination>$(OutputPath)\%(RecursiveDir)%(FileName)%(Extension)</Destination>
</NativeLibs>
</ItemGroup>
</Target>
<Target Name="AfterBuild" DependsOnTargets="GetMyOutputFiles">
<!--Copy one-to-one-->
<Copy SourceFiles="#(NativeLibs)" DestinationFiles="#(NativeLibs->'%(Destination)')" />
</Target>
<Target Name="AfterClean" DependsOnTargets="GetMyOutputFiles">
<Delete Files="#(NativeLibs->'%(Destination)')" />
<!--Find number of files left in each destination directory-->
<ItemGroup>
<NativeLibs>
<NumFiles>0</NumFiles>
<!--Condition is to avoid errors when e.g. running this target multiple times-->
<NumFiles Condition="Exists(%(DestinationDir))">$([System.IO.Directory]::GetFiles("%(DestinationDir)", "*", System.IO.SearchOption.AllDirectories).get_Length())</NumFiles>
</NativeLibs>
</ItemGroup>
<!--Only remove empty directories, use 'Distinct' to skip duplicate directories-->
<RemoveDir Directories="#(NativeLibs->'%(DestinationDir)'->Distinct())" Condition="%(NumFiles)=='0'" />
</Target>

Incremental Build of Nuget Packages

I want to execute an msbuild project which uses batching to determine that one or more csproj projects have been freshly-built, and therefore require fresh nuget packaging. The script I've made so far seems like a reasonable start, but it the incremental-build mechanism isn't working. The MainBuild target executes every time, no matter what.
Here is what I have:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="MainBuild" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)'=='' ">Debug</Configuration>
<Content>content\plugins\</Content>
</PropertyGroup>
<ItemGroup>
<Nuspec Include="$(MSBuildProjectDirectory)\plugins\*\*.nuspec" />
</ItemGroup>
<Target Name="MainBuild"
Inputs="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll"
Outputs="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" >
<ItemGroup>
<Inputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll" />
<Outputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" />
</ItemGroup>
<Message Text="INPUTS: %(Inputs.FullPath)" />
<Message Text="OUTPUTS: #(Outputs->'%(FullPath)')" />
<Copy SourceFiles="#(Inputs)" DestinationFiles="#(Outputs->'%(FullPath)')" />
</Target>
</Project>
The Copy task is just a debugging placeholder for calling-out to nuget and creating a new package.
The idea is that if any files in the bin\Debug directory are newer than the corresponding .nuspec file (found two folders above bin\Debug), then the MainBuild target should execute.
Any ideas?
p.s. The Inputs and Outputs attributes of the Target presumably each create an item. I think it strange that the items created can't be referenced inside the target. In the above example, I had to make a target-interna dynamic ItemGroup to re-create the items, just so that I could access them. Is there a way around that?
I read this in the MSBuild Batching documentation
If a task inside of a target uses batching, MSBuild needs to determine
if the inputs and outputs for each batch of items is up-to-date.
Otherwise, the target is executed every time it is hit.
Which may be the cuprit. Try changing your copy target to use batching instead of an ite transform (I don't think using item metadata in an item group satisfies the above requirement).
<Target Name="MainBuild"
Inputs="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll"
Outputs="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" >
<ItemGroup>
<Inputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll" />
<Outputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" />
</ItemGroup>
<Message Text="INPUTS: %(Inputs.FullPath)" />
<Message Text="OUTPUTS: #(Outputs->'%(FullPath)')" />
<Copy SourceFiles="#(Inputs)" DestinationFiles="%(Outputs.FullPath)" />
</Target>
It looks like the number of inputs may be different than the number of outputs (I suspect there is more than one .dll files in the output directory for each project), which will also cause the target to execute.

Additional paths in msbuild script

How to specify additional assembly reference paths for the MSBuild tasks?
I have following script so far, but can't figure out how to specify additional search paths.
<ItemGroup>
<ProjectsToBuild Include="..\Main\Main.sln" />
</ItemGroup>
<!-- The follwing paths should be added to reference search paths for the build tasks -->
<ItemGroup>
<MyAddRefPath Include="$(MSBuildProjectDirectory)\..\..\Build\Lib1" />
<MyAddRefPath Include="$(MSBuildProjectDirectory)\..\..\Build\Lib2" />
</ItemGroup>
<MSBuild
Projects="#(ProjectsToBuild)"
Properties="Configuration=Debug;OutputPath=$(BuildOutputPath)">
</MSBuild>
UPDATE:
Please show one complete working script which invokes original project, such as an SLN with multiple additional reference paths.
No suggestions on how to improve the project structure please.
I know how to build a good structure, but now it's the task of building an existing piece of crap.
I have finaly figured out how to do it:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<ProjectsToBuild Include="ConsoleApplication1\ConsoleApplication1.csproj" />
</ItemGroup>
<ItemGroup>
<AdditionalReferencePaths Include="..\Build\ClassLibrary1" />
<AdditionalReferencePaths Include="..\Build\ClassLibrary2" />
</ItemGroup>
<PropertyGroup>
<BuildOutputPath>..\Build\ConsoleApplication1</BuildOutputPath>
</PropertyGroup>
<Target Name="MainBuild">
<PropertyGroup>
<AdditionalReferencePathsProp>#(AdditionalReferencePaths)</AdditionalReferencePathsProp>
</PropertyGroup>
<MSBuild
Projects="ConsoleApplication1\ConsoleApplication1.csproj"
Properties="ReferencePath=$(AdditionalReferencePathsProp);OutputPath=$(BuildOutputPath)"
>
</MSBuild>
</Target>
The property you want to modify is AssemblySearchPaths. See the ResolveAssemblyReference task more information.
<Target Name="AddToSearchPaths">
<CreateProperty Value="x:\path\to\assemblies;$(AssemblySearchPaths)">
<Output PropertyName="AssemblySearchPaths" TaskParameter="Value" />
</CreateProperty>
</Target>
Making use of item groups, as in your example, it would look like:
<Target Name="AddToSearchPaths">
<CreateProperty Value="#(MyAddRefPath);$(AssemblySearchPaths)">
<Output PropertyName="AssemblySearchPaths" TaskParameter="Value" />
</CreateProperty>
</Target>
Looking in %WINDIR%\Microsoft.NET\Framework\v2.0.50727\Microsoft.Common.targets, you can see that the ResolveAssemblyReference Task is executed as part of the ResolveAssemblyReferences target. Thus, you want the newly added target to modify the AssemblySearchPaths property before ResolveAssemblyReferences is executed.
You've stated that you want to be able to modify the assembly search paths without modifying the project files directly. In order to accomplish that requirement you need to set an environment variable that will override the AssemblySearchPaths. With this technique you will need to provide every assembly reference path used by all the projects in the solutions. (Modifying the projects or copies of the projects would be easier. See final comments.)
One technique is to create a batch file that runs your script at sets the environment variable:
set AssemblySearchPaths="C:\Tacos;C:\Burritos;C:\Chalupas"
msbuild whatever.msbuild
Another way is to define a PropertyGroup in your custom msbuild file (otherwise known as the "hook" needed to make this work):
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<ProjectsToBuild Include="..\Main\Main.sln" />
</ItemGroup>
<PropertyGroup>
<AssemblySearchPaths>$(MSBuildProjectDirectory)\..\..\Build\Lib1;$(MSBuildProjectDirectory)\..\..\Build\Lib2</AssemblySearchPaths>
</PropertyGroup>
<Target Name="Build">
<MSBuild Projects="#(ProjectsToBuild)" Properties="AssemblySearchPaths=$(AssemblySearchPaths);Configuration=Debug;OutputPath=$(OutputPath)" />
</Target>
</Project>
Now if it were me, and for whatever unexplained reason I couldn't modify the project files to include the updated references that I am going to build with, I would make copies of the project files, load them into the IDE, and correct the references in my copies. Synching the projects becomes a simple diff/merge operation which is automatic with modern tools like mercurial (heck I'm sure clearcase could manage it too).
...and remember that you don't need to use a target for this, you can use project-scoped properties or items, as...
<ItemGroup>
<MyAddRefPath Include="$(MSBuildProjectDirectory)\..\..\Build\Lib1" />
<MyAddRefPath Include="$(MSBuildProjectDirectory)\..\..\Build\Lib2" />
</ItemGroup>
<PropertyGroup>
<MyAddRefPath>$(MSBuildProjectDirectory)\..\..\Build\Lib3</MyAddRefPath>
<!-- add in the property path -->
<AssemblySearchPaths>$(MyAddRefPath);$(AssemblySearchPaths)</AssemblySearchPaths>
<!-- add in the item paths -->
<AssemblySearchPaths>#(MyAddRefPath);$(AssemblySearchPaths)</AssemblySearchPaths>
</PropertyGroup>
...and if you do need to do this in a target to pick up paths from a dynamically populated item group, use inline properties, not the CreateProperty task (if you are not stuck in v2.0)
<Target Name="AddToSearchPaths">
<PropertyGroup>
<!-- add in the item paths -->
<AssemblySearchPaths>#(MyDynamicAddRefPath);$(AssemblySearchPaths)</AssemblySearchPaths>
</PropertyGroup>
</Target>

MSBuild several metadata requests in the same expression

I am writing my first MSBuild script and ran into a problem.
I have several projects, defined in an itemgroup
<ItemGroup>
<Projects Include="Project1Dir\Project1.csproj"/>
<Projects Include="Project2Dir\Project2.csproj"/>
</ItemGroup>
Then, on deployment step, I am trying to do this:
The following should collect all the files for deployment into separate itemgroups for each project ("Project1deploymentFiles" and "Project2deploymentFiles")
<CreateItem Include="$(WebPublishDir)\%(Projects.Filename)\**\*.*">
<Output ItemName="%(Projects.Filename)deploymentFiles" TaskParameter="Include"/>
</CreateItem>
Thes line, should copy each project's files into separate folder
<Copy SourceFiles="#(%(Projects.Filename)deploymentFiles)" DestinationFolder="$(DeploymentDir)\%(Projects.Filename)\%(RecursiveDir)\" />
But it seems that MSBuild resolves %(RecursiveDir) metadata to empty string, as all the files are copied to the same root folder (different for each project).
Any suggestions what am I doing wrong here?
I've found a solution myself:
<CreateItem Include="$(WebPublishDir)\%(Projects.Filename)\**\*.*" AdditionalMetadata="ProjectDir=%(Projects.Filename)\">
<Output ItemName="deploymentFiles" TaskParameter="Include"/>
</CreateItem>
<Copy SourceFiles="#(deploymentFiles)" DestinationFolder="$(DeploymentDir)\%(ProjectDir)\%(RecursiveDir)\" />
Main idea here is to use one item for all projects and just add AdditionalMetadata to values, containing projectname