MSBuild CopyTask: Copying the same file to multiple locations - msbuild

Is there a way to get the CopyTask to copy the same file to multiple locations?
eg. I've generated an AssemblyInfo.cs file and want to copy it across to all my projects before building.

Check out the RoboCopy build task which is part of the Community Build Tasks library which you can find here. RoboCopy can copy one source file to multiple destinations.
On a side note: why don't you use one AssemblyInfo file on solution level and link to that in your projects if you need the same information in every project? Check out my accepted answer on this question: Automatic assembly version number management in VS2008

Right, well maybe I should attempt to do the things I want to do before asking for help :)
<ItemGroup>
<AssemblyInfoSource
Include="AssemblyInfo.cs;AssemblyInfo.cs" />
<AssemblyInfoDestination
Include="$(Destination1)\AssemblyInfo.cs;$(Destination2)\AssemblyInfo.cs" />
</ItemGroup>
<Copy SourceFiles="#(AssemblyInfoSource)" DestinationFiles="#(AssemblyInfoDestination)" />

I had a need to copy the contents of a directory to multiple locations, this is what I came up with that works. So I am posting it here ins case anyone else is in similar need and comes across this question like I did.
<!-- Create a list of the objects in PublishURL so it will copy to multiple directories -->
<ItemGroup>
<PublishUrls Include="$(PublishUrl)"/>
</ItemGroup>
<PropertyGroup>
<Files>$(OutputPath)\**\*</Files>
</PropertyGroup>
<!-- CopyNewFiles will copy all the files in $(OutputPath) to all the directories in the
in $(PublishUrl). $(PublishUrl) can be a single directory, or a list of directories
separated by a semicolon -->
<Target Name ="CopyNewFiles">
<!-- Get list of all files in the output directory; Cross product this with all
the output directories. -->
<CreateItem Include ="$(Files)"
AdditionalMetadata="RootDirectory=%(PublishUrls.FullPath)">
<Output ItemName ="OutputFiles" TaskParameter ="Include"/>
</CreateItem>
<Message Text="'#(OutputFiles)' -> '%(RootDirectory)\%(RecursiveDir)'"/>
<Copy SourceFiles="#(OutputFiles)"
DestinationFolder ="%(RootDirectory)\%(RecursiveDir)"/>
</Target>
If you want to copy AssemblyInfo.cs to Folders A and B you would set the property Files="AssemblyInfo.cs" and PublishUrls="A;B"
What makes this work is the extra metadata in the CreateItem task AdditionalMetadata="RootDirectory=%(PublishUrls.FullPath)" so for each files found in File it creates 1 entry for each item found in PublishUrls. In your case of a single file the equivelent in writing out the xml would be:
<ItemGroup>
<OutputFiles Include="AssemblyInfo.cs">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="AssemblyInfo.cs">
<RootDirectory>B</RootDirectory>
</OutputFiles>
</ItemGroup>
Now if you copied the contents of a folder that had files 1.txt and 2.txt copied to A and B the equivalent xml would be:
<ItemGroup>
<OutputFiles Include="1.txt">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="2.txt">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="1.txt">
<RootDirectory>B</RootDirectory>
</OutputFiles>
<OutputFiles Include="2.txt">
<RootDirectory>B</RootDirectory>
</OutputFiles>
</ItemGroup>

Related

csproj include (recursive) folders based on a list (batching)

I want to add some files recursively to a single project that are located in folders inside my solution root.
Right now, I found a working solution to add those files by specifying each folder manually:
<ItemGroup>
<Content Include="..\Folder1\**\*">
<Link>Folder1\%(RecursiveDir)\%(Filename)%(Extension)</Link>
</Content>
<Content Include="..\Folder2\**\*">
<Link>Folder2\%(RecursiveDir)\%(Filename)%(Extension)</Link>
</Content>
<Content Include="..\Folder3\**\*">
<Link>Folder3\%(RecursiveDir)\%(Filename)%(Extension)</Link>
</Content>
</ItemGroup>
As those can be a handfull folders and they can be different depending on my solution, I want to have an easy list to define the folder names.
Something like this:
<PropertyGroup>
<DefinedResourceFolders>Folder1;Folder2;Folder3</DefinedResourceFolders>
</PropertyGroup>
This would also allow me to input this as a property directly when calling msbuild to extend the files that are going to the build output, maybe.
I tried to look into the MSBuild Batching documentation and several other online sources, but I could not figure it out. Most batching examples I found were working with Targets, not Includes into the solution items.
Is this possible? How would I define the <ItemGroup> for the content then?
P.S.: I don't care about wildcard issues with .csproj files when new files are added, etc. This will either be a single "Resources" project only containing those displayed files, or I am using my external .props file that I am importing into each .csproj file anyway.
Suppose you want to include all files in folders root/Folder1, root/Folder2 and root/Folder3 recursively.
This is how the builds (both VS and CLI) do what you want. However, VS will not show the files as part of the project.
<Project Sdk="Microsoft.NET.Sdk" InitialTargets="__AddBatchedContent;$(InitialTargets)">
<!-- the usual properties -->
<!-- You could of course define the folder array directly in an Item,
but this is what you wanted :-) -->
<PropertyGroup>
<Lookups>Folder1;Folder2;Folder3</Lookups>
</PropertyGroup>
<ItemGroup>
<LookupDir Include="$(Lookups)" />
</ItemGroup>
<Target Name="__AddBatchedContent">
<ItemGroup>
<!-- save the original source folder of the globbed file name in custom
metadata "Folder" so we can use it later as its output base folder -->
<__BatchedFiles Include="..\%(LookupDir.Identity)\**\*" Folder="%(LookupDir.Identity)" />
<Content Include="#(__BatchedFiles)" Link="%(Folder)\%(RecursiveDir)\%(Filename)%(Extension)" CopyToOutputDirectory="Always" />
</ItemGroup>
</Target>
</Project>
Note that if you want to put this in a .targets file to use this technique in multiple projects, you should replace the ..\ with $(MSBuildThisFileDirectory)..\ so that the path is relative to the (known) location of the .targets file instead of the (unknown) location of the importing project.
Thanks to #Ilya Kozhevnikov for the basis of this answer.
You are not the only one who would like this to be simpler :-): https://github.com/dotnet/msbuild/issues/3274

MSBuild - Copy file to output directory if the file isn't in the output directory

I have a nuget with a .targets file that tells the consuming project to copy all files within a "Dependencies" folder to the output directory.
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<Copy SourceFiles="#(Files)"
DestinationFolder="$(TargetDir)" />
</Target>
This nuget is consumed by two projects: Project A and Project B. For this question, let's say we have a System.Runtime.InteropServices.RuntimeInformation.dll that is one of the dependencies within this nuget. The output directory of Project A does not already have System.Runtime.InteropServices.RuntimeInformation.dll, so it gets copied to the output directory when the project is built. Project B however already contains System.Runtime.InteropServices.RuntimeInformation.dll in the output directory. This causes a runtime issue at startup since the targets file is trying to overwrite the existing DLL of the same name with the System.Runtime.InteropServices.RuntimeInformation.dll file from within the nuget (which is a dependency of other files within the output directory).
So, how can I adjust my .targets file to only copy in files that do not already exist within the output directory based on name, and not size or date modified?
There are several ways but probably the most succinct change to your example code would be the following:
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<Copy SourceFiles="#(Files)" DestinationFolder="$(TargetDir)" Condition="!Exists('$(TargetDir)/%(Filename)%(Extension)')" />
</Target>
The change is adding a Condition on the Copy that is using the metadata of the #(Files) collection to test that the file does not exist in $(TargetDir).
Because of the use of metadata, the Copy is a task batch. Essentially the #(Files) collection is divided into batches by %(Filename)%(Extension) and there is a separate Copy task invoked for each batch.
If there is a large number of files in the Dependencies folder, the following variant may provide better performance.
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<ItemGroup>
<FilesToCopy Include="#(Files)" Condition="!Exists('$(TargetDir)/%(Filename)%(Extension)')" />
</ItemGroup>
<Copy SourceFiles="#(FilesToCopy)" DestinationFolder="$(TargetDir)" />
</Target>
The task batching is moved to the definition of a new ItemGroup collection and the Copy task is invoked once for the set of files. The potential performance improvement is that the implementation of the Copy task tries to parallelize copies, which it can't do when invoked per file.

Build script to copy files from various source folder to various destination folder

The basic requirement is to copy the various files and folders from different solution/project directories to the single build_output folder(/subfolders).
Currently, I am doing this operation using the Robocopy commands. The only issue is my script is too long just using multiple Robocopy commands.
<Copy SourceFiles="$(Docs)\Manual.pdf" DestinationFolder="$(BuildPath)\Help"/>
<RoboCopy Source="$(Web1)" Destination="$(BuildPath)" Files="*.aspx" Options="/E"/>
<RoboCopy Source="$(Web1)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<RoboCopy Source="$(Web2)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<!--- 100s of such RoboCopy & Copy commands (note that in last two commands i need to copy from different sources to same destination -->
How this job is implemented in real enterprise applications, so that
the build script is concise and clear.
Is my thinking below is the way to approach the solution. If yes, can anybody provide me sample steps using MSBuild or CommandScript easily. (free to use any MSBuild extensions)
Define the mapping of the all source folders, file types (can be xyz.png/.png/.*) and the destination path.
Copy the files (Robocopy) using the above mentioned mappings using a single target or task)
Is there any other better way to do this problem?
Insights/Solution ???
I do exactly this sort of thing to stage build output for harvesting by the installer build. I have a custom targets file for consistent processing and have some msbuild property files with the item groups describing that needs to be done.
<ItemGroup Label="AcmeComponent1Payload">
<FileToHarvest Include="$(SourceRoot)AcmeProjects\ServerManager\$(Configuration)\**\*;
$(SourceRoot)Library\SQLServerCompact\**\*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.*"
Exclude="$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\IntegrationTests.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Practices.*.xml;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\obj\**\*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.VisualStudio.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Web.*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.xml">
<Group>AcmeServerManager</Group>
<SubDir>Utilities\</SubDir>
</FileToHarvest>
</ItemGroup>
The custom targets file has the functionality to process it.
<Target Name="CopyFiles">
<Copy Condition="#(FileToHarvest)!=''"
SourceFiles="#(FileToHarvest)"
DestinationFiles="#(FileToHarvest->'$(OutputPath)\%(Group)\%(SubDir)%(RecursiveDir)%(Filename)%(Extension)')"
OverwriteReadOnlyFiles="true"
SkipUnchangedFiles="true" />
</Target>
You can make the properties file as simple or as complicated as you like. I use multiple ones and import them into the project file using wildcards.
Thanks #daughey, I got my first part working, where I need to copy from different sources to the same destination.
<!--Declare an ItemGroup that points to source Locations-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit"/>
<ItemToCopy Include="$(Utilities)\Service"/>
<ItemToCopy Include="$(Web1)\NET"/>
</ItemGroup>
<!--Declare an ItemGroup that points to destination Locations-->
<ItemGroup>
<DestLocations Include="$(BuildPath)" />
</ItemGroup>
<Target Name="CopyFiles">
<!-- Run the copy command to copy the item to your dest locations-->
<!-- The % sign says to use Batching. So Copy will be run for each unique source ItemToCopy(s) in the DestLocation.-->
<RemoveDir Directories="$(BuildPath)"/>
<Message Importance="high" Text="Deploy folder is $(BuildPath)"/>
<RoboCopy Source="%(ItemToCopy.FullPath)" Destination="$(BuildPath)" Files="*.dll"/>
</Target>
After struggling with Item Metadata for Task Batching, the second part is also working great.
Scenario: Copy files from list of source directories to output directory on their respective sub-folders
$(Web1)\Audit*.dll => $(BuildPath)\Audit*.dll
$(Utilities)\Service*.jpg => $(BuildPath)\Utilities\Service*.jpg
Solution
<!--Modify an ItemGroup with metadata-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit">
<ToPath>$(BuildPath)\Audit</ToPath>
<FileType>*.dll</FileType>
</ItemToCopy>
<ItemToCopy Include="$(Utilities)\Service">
<ToPath>$(BuildPath)\Utilities\Service</ToPath>
<FileType>*.jpg;*.bmp</FileType>
</ItemToCopy>
</ItemGroup>
<Target Name="CopyBatch">
<RoboCopy Source="%(ItemToCopy.Identity)" Destination="%(ItemToCopy.ToPath)" Files="%(ItemToCopy.Filetype)"/>
</Target>

Remove Files and Folders Copied From AfterBuild Target

I would like to avoid hard coding the dll and folder names in the AfterClean target, is there a dynamic way to do this? Ideally it would only delete the files and folders created by the Copy in the AfterBuild target.
I tried to simplify this by changing the DestinationFolder to include a subdirectory in the OutputPath. The AfterClean target would only have to remove that subdirectory at this point. However, some of the library's DLLImport paths don't take that subdirectory into consideration which results in a crash.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="AfterBuild">
<ItemGroup>
<NativeLibs Include="$(MSBuildThisFileDirectory)..\lib\native\**\*.*" />
</ItemGroup>
<Copy SourceFiles="#(NativeLibs)" DestinationFolder="$(OutputPath)\%(RecursiveDir)" />
</Target>
<Target Name="AfterClean">
<Delete Files="$(OutputPath)\LumiAPI.dll" />
<Delete Files="$(OutputPath)\LumiCore.dll" />
<Delete Files="$(OutputPath)\LumiInOpAPI.dll" />
<RemoveDir Directories="$(OutputPath)\SPM" />
<RemoveDir Directories="$(OutputPath)\plugin" />
</Target>
</Project>
Project Structure:
src
ConsumingProject
ConsumingProject.csproj
ConsumingProject.sln
packages
my-project.5.7.0.12
build
lib
native
plugin
VenusDvc.dll
SPM
sSPM_1.bin
LumiAPI.dll
LumiCore.dll
LumiInOpAPI.dll
net45
my-project.5.7.0.12.nupkg
Essentially I want to delete all the files and folders that were copied from the native folder to the output of the project (ie LumiAPI.dll, LumiCore.dll, SPM (folder), eSPM_1.bin, etc). However I want it to be generic enough so that if I add another folder to the native directory, it will delete those folders/files as well.
Use a seperate target which lists input and output files, then use that list in both other targets. Note this uses the DestinationFiles attribute from the Copy task instead of DestinationFolders. And it might print some messages about non-existing directories being passed to RemoveDir because the top directory gets removed already before child directories.
update since you don't want to remove the root output directory as it still has files, figured applying the 'only remove output directory if it's empty' principle for any destination directory is probably the safest way to go. Credits go to the answer here.
<Target Name="GetMyOutputFiles">
<ItemGroup>
<NativeLibs Include="$(MSBuildThisFileDirectory)..\lib\native\**\*.*" />
<!--Now add some metadata: output dir and output file-->
<NativeLibs>
<DestinationDir>$(OutputPath)\%(RecursiveDir)</DestinationDir>
<Destination>$(OutputPath)\%(RecursiveDir)%(FileName)%(Extension)</Destination>
</NativeLibs>
</ItemGroup>
</Target>
<Target Name="AfterBuild" DependsOnTargets="GetMyOutputFiles">
<!--Copy one-to-one-->
<Copy SourceFiles="#(NativeLibs)" DestinationFiles="#(NativeLibs->'%(Destination)')" />
</Target>
<Target Name="AfterClean" DependsOnTargets="GetMyOutputFiles">
<Delete Files="#(NativeLibs->'%(Destination)')" />
<!--Find number of files left in each destination directory-->
<ItemGroup>
<NativeLibs>
<NumFiles>0</NumFiles>
<!--Condition is to avoid errors when e.g. running this target multiple times-->
<NumFiles Condition="Exists(%(DestinationDir))">$([System.IO.Directory]::GetFiles("%(DestinationDir)", "*", System.IO.SearchOption.AllDirectories).get_Length())</NumFiles>
</NativeLibs>
</ItemGroup>
<!--Only remove empty directories, use 'Distinct' to skip duplicate directories-->
<RemoveDir Directories="#(NativeLibs->'%(DestinationDir)'->Distinct())" Condition="%(NumFiles)=='0'" />
</Target>

MSBuild project file: Copy item to specific location in output directory

In the process of cleaning up the folder/file structure on a project I inherited, I'm running into a problem with organizing the required external libraries. I want to keep them in their own .\dll\ folder, but they're not being copied to the build directory properly. They should be in the root build directory, but they're being moved to a subfolder instead.
My .csproj file contains the following xml:
<Project>
<ItemGroup>
<None Include="dlls\libraryA.dll">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
Then, on build, the libraryA.dll file is copied to the bin\Debug\dll\ folder, but I want it in the bin\Debug\ folder.
I tried this and msbuild always wants to copy the files using their directory path, but there is a workaround...
Edit the csproj file and after this line:
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
Add these lines:
<PropertyGroup>
<PrepareForRunDependsOn>$(PrepareForRunDependsOn);MyCopyFilesToOutputDirectory</PrepareForRunDependsOn>
</PropertyGroup>
<Target Name="MyCopyFilesToOutputDirectory">
<Copy SourceFiles="#(None)" DestinationFolder="$(OutDir)" />
</Target>
The copy of the output files happens in the PrepareForRun target. This adds your own target to the list of targets that are executed as part of PrepareForRun.
This example copies all items in the None item group. You could create your own item group (e.g. MyFiles) and do the copy on that item group if you have other "None" files you don't want copied. When I tried this I had to change the item group name by editing the csproj file directly. Visual Studio did not allow me to set the item group of a file from the UI, but after I edited the csproj and changed it, Visual Studio displayed my custom item group name correctly.
If you only want to change it for one file, it may be easier to use the property:
<None Include="dlls\libraryA.dll">
<Link>%(Filename)%(Extension)</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
Including content files in .csproj that are outside the project cone
This approach works
If you need to force copy of a specific file/nuget package into an asp.net core project (2.2), add at the end of your csproj :
<!-- Force copy MathNet because we need it in compilation -->
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="Build">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll'))" />
</Target>
<ItemGroup>
<ContentWithTargetPath Include="..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<TargetPath>MathNet.Numerics.dll</TargetPath>
</ContentWithTargetPath>
</ItemGroup>
In SDK-style csproj you can write something like:
<Target Name="CopyFilesTargetName" AfterTargets="Build">
<Copy SourceFiles="$(OutDir)\dlls\Some.dll;$(OutDir)\dlls\SomeOther.dll" DestinationFolder="$(OutDir)" />
</Target>
You can also use <Move instead of <Copy to move files