MSBuild - Copy file to output directory if the file isn't in the output directory - msbuild

I have a nuget with a .targets file that tells the consuming project to copy all files within a "Dependencies" folder to the output directory.
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<Copy SourceFiles="#(Files)"
DestinationFolder="$(TargetDir)" />
</Target>
This nuget is consumed by two projects: Project A and Project B. For this question, let's say we have a System.Runtime.InteropServices.RuntimeInformation.dll that is one of the dependencies within this nuget. The output directory of Project A does not already have System.Runtime.InteropServices.RuntimeInformation.dll, so it gets copied to the output directory when the project is built. Project B however already contains System.Runtime.InteropServices.RuntimeInformation.dll in the output directory. This causes a runtime issue at startup since the targets file is trying to overwrite the existing DLL of the same name with the System.Runtime.InteropServices.RuntimeInformation.dll file from within the nuget (which is a dependency of other files within the output directory).
So, how can I adjust my .targets file to only copy in files that do not already exist within the output directory based on name, and not size or date modified?

There are several ways but probably the most succinct change to your example code would be the following:
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<Copy SourceFiles="#(Files)" DestinationFolder="$(TargetDir)" Condition="!Exists('$(TargetDir)/%(Filename)%(Extension)')" />
</Target>
The change is adding a Condition on the Copy that is using the metadata of the #(Files) collection to test that the file does not exist in $(TargetDir).
Because of the use of metadata, the Copy is a task batch. Essentially the #(Files) collection is divided into batches by %(Filename)%(Extension) and there is a separate Copy task invoked for each batch.
If there is a large number of files in the Dependencies folder, the following variant may provide better performance.
<ItemGroup>
<Files Include="$(MSBuildThisFileDirectory)/../contentFiles/Dependencies/*.*" />
</ItemGroup>
<Target Name="CopyDependencies" AfterTargets="Build">
<ItemGroup>
<FilesToCopy Include="#(Files)" Condition="!Exists('$(TargetDir)/%(Filename)%(Extension)')" />
</ItemGroup>
<Copy SourceFiles="#(FilesToCopy)" DestinationFolder="$(TargetDir)" />
</Target>
The task batching is moved to the definition of a new ItemGroup collection and the Copy task is invoked once for the set of files. The potential performance improvement is that the implementation of the Copy task tries to parallelize copies, which it can't do when invoked per file.

Related

How to create a custom task with a dependency for incremental build?

I have a build step that parses a csv file and generates numerous .cs files, and I am trying to make a build target that will let me generated them incrementally/only when the source file has changed.
I have created a Target (in TechTreeGenerator.targets):
<Target Name="BuildTechTreeGenerator" BeforeTargets="BeforeBuild" Inputs="#(TechTree)" Outputs="$(ProjectDir)..\Mods\AutoGen\AutoGen.lastbuild">
<TechTreeGenerator SourceCsv="#(TechTree)" OutputFolder="$(ProjectDir)..\Mods\AutoGen" TemplateFolder="$(ProjectDir)..\TechTreeTemplates">
<Output TaskParameter="DestinationFiles" ItemName="CompileFiles"/>
</TechTreeGenerator>
<ItemGroup>
<CompileFiles>
<Link>%(Filename)%(Extension)</Link>
</CompileFiles>
</ItemGroup>
<ItemGroup>
<Compile Include="#(CompileFiles);" />
</ItemGroup>
</Target>
Where TechTreeGenerator is a code Task that runs the tool, and outputs a list of all files that are generated from the csv file (even if they haven't been re-generated). It also always outputs a "AutoGen.lastbuild" file to be compared with the csv file to tell if it needs to be rebuilt.
However, the lastbuild in the Outputs of the Target doesn't seem to have any affect. Deleting it doesn't rebuild the file.
Edit: I forgot to add in .csproj the project I'm referencing it as:
<Import Project="TechTreeGenerator.targets" />
<ItemGroup>
<TechTree Include="..\EcoTechTree.csv" />
</ItemGroup>
Forget about the .lastbuild file. Just put as the input the .csv file. And put as the output an itemgroup the .cs files you generate.
<Target Name="bla" Inputs="#(CSVFilePath)" Outputs="#(YourCSfiles)">

Build script to copy files from various source folder to various destination folder

The basic requirement is to copy the various files and folders from different solution/project directories to the single build_output folder(/subfolders).
Currently, I am doing this operation using the Robocopy commands. The only issue is my script is too long just using multiple Robocopy commands.
<Copy SourceFiles="$(Docs)\Manual.pdf" DestinationFolder="$(BuildPath)\Help"/>
<RoboCopy Source="$(Web1)" Destination="$(BuildPath)" Files="*.aspx" Options="/E"/>
<RoboCopy Source="$(Web1)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<RoboCopy Source="$(Web2)\Images" Destination="$(BuildPath)\Images" Files="*.jpg;*.png" Options="/E"/>
<!--- 100s of such RoboCopy & Copy commands (note that in last two commands i need to copy from different sources to same destination -->
How this job is implemented in real enterprise applications, so that
the build script is concise and clear.
Is my thinking below is the way to approach the solution. If yes, can anybody provide me sample steps using MSBuild or CommandScript easily. (free to use any MSBuild extensions)
Define the mapping of the all source folders, file types (can be xyz.png/.png/.*) and the destination path.
Copy the files (Robocopy) using the above mentioned mappings using a single target or task)
Is there any other better way to do this problem?
Insights/Solution ???
I do exactly this sort of thing to stage build output for harvesting by the installer build. I have a custom targets file for consistent processing and have some msbuild property files with the item groups describing that needs to be done.
<ItemGroup Label="AcmeComponent1Payload">
<FileToHarvest Include="$(SourceRoot)AcmeProjects\ServerManager\$(Configuration)\**\*;
$(SourceRoot)Library\SQLServerCompact\**\*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.*"
Exclude="$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\IntegrationTests.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Practices.*.xml;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\obj\**\*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.VisualStudio.*;
$(SourceRoot)Server Manager Projects\AcmeServerManager\$(Configuration)\**\Microsoft.Web.*;
$(SourceRoot)Utility Projects\PropertyDataValidator\PropertyDataValidator\bin\$(Configuration)\PropertyDataValidator.xml">
<Group>AcmeServerManager</Group>
<SubDir>Utilities\</SubDir>
</FileToHarvest>
</ItemGroup>
The custom targets file has the functionality to process it.
<Target Name="CopyFiles">
<Copy Condition="#(FileToHarvest)!=''"
SourceFiles="#(FileToHarvest)"
DestinationFiles="#(FileToHarvest->'$(OutputPath)\%(Group)\%(SubDir)%(RecursiveDir)%(Filename)%(Extension)')"
OverwriteReadOnlyFiles="true"
SkipUnchangedFiles="true" />
</Target>
You can make the properties file as simple or as complicated as you like. I use multiple ones and import them into the project file using wildcards.
Thanks #daughey, I got my first part working, where I need to copy from different sources to the same destination.
<!--Declare an ItemGroup that points to source Locations-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit"/>
<ItemToCopy Include="$(Utilities)\Service"/>
<ItemToCopy Include="$(Web1)\NET"/>
</ItemGroup>
<!--Declare an ItemGroup that points to destination Locations-->
<ItemGroup>
<DestLocations Include="$(BuildPath)" />
</ItemGroup>
<Target Name="CopyFiles">
<!-- Run the copy command to copy the item to your dest locations-->
<!-- The % sign says to use Batching. So Copy will be run for each unique source ItemToCopy(s) in the DestLocation.-->
<RemoveDir Directories="$(BuildPath)"/>
<Message Importance="high" Text="Deploy folder is $(BuildPath)"/>
<RoboCopy Source="%(ItemToCopy.FullPath)" Destination="$(BuildPath)" Files="*.dll"/>
</Target>
After struggling with Item Metadata for Task Batching, the second part is also working great.
Scenario: Copy files from list of source directories to output directory on their respective sub-folders
$(Web1)\Audit*.dll => $(BuildPath)\Audit*.dll
$(Utilities)\Service*.jpg => $(BuildPath)\Utilities\Service*.jpg
Solution
<!--Modify an ItemGroup with metadata-->
<ItemGroup>
<ItemToCopy Include="$(Web1)\Audit">
<ToPath>$(BuildPath)\Audit</ToPath>
<FileType>*.dll</FileType>
</ItemToCopy>
<ItemToCopy Include="$(Utilities)\Service">
<ToPath>$(BuildPath)\Utilities\Service</ToPath>
<FileType>*.jpg;*.bmp</FileType>
</ItemToCopy>
</ItemGroup>
<Target Name="CopyBatch">
<RoboCopy Source="%(ItemToCopy.Identity)" Destination="%(ItemToCopy.ToPath)" Files="%(ItemToCopy.Filetype)"/>
</Target>

How do I use my .targets file in Visual Studio with custom build actions?

I am a beginner with MSBuild. So far I have been able to create a custom task called 'MakeTextFile' which creates a text file in C:\ based on the contents property you pass it. This works running from a command line prompt.
I have also included this in my .targets file (under the project tag):
<ItemGroup>
<AvailableItemName Include="CreateTextFileAction" />
</ItemGroup>
When I use the Import tag on my client applications .csproj I can now set items build actions to 'CreateTextFileAction', however the action never triggers (as no text file on C:\ is created)
How do I get all the file paths of items that were marked with my build action 'CreateTextFileAction' and pass them onto my custom task?
For reference, my .targets file:
<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<AvailableItemName Include="CreateTextFileAction" />
</ItemGroup>
<UsingTask AssemblyFile="CustomMSBuildTask.dll" TaskName="CustomMSBuildTask.MakeTextFile" />
<Target Name="MyTarget">
<MakeTextFile Contents="TODO HOW DO I GRAB MARKED FILES?" />
</Target>
</Project>
A csproj file has a defined set of targets. The three main entry points are Build, Rebuild and Clean. These targets each have a set of dependencies. If you write your own targets to be part of the standard csproj build you need to find a suitable injection point within these dependencies.
For ease of use there are two standard targets for you to override called BeforeBuild and AfterBuild. If you define this in the csproj file (after the import of the csharp targets file) and call your custom task in there then it should work (or at least move further along).
<Target Name="BeforeBuild">
<MakeTextFile Contents="TODO HOW DO I GRAB MARKED FILES?" />
</Target>

MSBuild project file: Copy item to specific location in output directory

In the process of cleaning up the folder/file structure on a project I inherited, I'm running into a problem with organizing the required external libraries. I want to keep them in their own .\dll\ folder, but they're not being copied to the build directory properly. They should be in the root build directory, but they're being moved to a subfolder instead.
My .csproj file contains the following xml:
<Project>
<ItemGroup>
<None Include="dlls\libraryA.dll">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
Then, on build, the libraryA.dll file is copied to the bin\Debug\dll\ folder, but I want it in the bin\Debug\ folder.
I tried this and msbuild always wants to copy the files using their directory path, but there is a workaround...
Edit the csproj file and after this line:
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
Add these lines:
<PropertyGroup>
<PrepareForRunDependsOn>$(PrepareForRunDependsOn);MyCopyFilesToOutputDirectory</PrepareForRunDependsOn>
</PropertyGroup>
<Target Name="MyCopyFilesToOutputDirectory">
<Copy SourceFiles="#(None)" DestinationFolder="$(OutDir)" />
</Target>
The copy of the output files happens in the PrepareForRun target. This adds your own target to the list of targets that are executed as part of PrepareForRun.
This example copies all items in the None item group. You could create your own item group (e.g. MyFiles) and do the copy on that item group if you have other "None" files you don't want copied. When I tried this I had to change the item group name by editing the csproj file directly. Visual Studio did not allow me to set the item group of a file from the UI, but after I edited the csproj and changed it, Visual Studio displayed my custom item group name correctly.
If you only want to change it for one file, it may be easier to use the property:
<None Include="dlls\libraryA.dll">
<Link>%(Filename)%(Extension)</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
Including content files in .csproj that are outside the project cone
This approach works
If you need to force copy of a specific file/nuget package into an asp.net core project (2.2), add at the end of your csproj :
<!-- Force copy MathNet because we need it in compilation -->
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="Build">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll'))" />
</Target>
<ItemGroup>
<ContentWithTargetPath Include="..\packages\MathNet.Numerics.4.8.1\lib\netstandard2.0\MathNet.Numerics.dll">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<TargetPath>MathNet.Numerics.dll</TargetPath>
</ContentWithTargetPath>
</ItemGroup>
In SDK-style csproj you can write something like:
<Target Name="CopyFilesTargetName" AfterTargets="Build">
<Copy SourceFiles="$(OutDir)\dlls\Some.dll;$(OutDir)\dlls\SomeOther.dll" DestinationFolder="$(OutDir)" />
</Target>
You can also use <Move instead of <Copy to move files

MSBuild CopyTask: Copying the same file to multiple locations

Is there a way to get the CopyTask to copy the same file to multiple locations?
eg. I've generated an AssemblyInfo.cs file and want to copy it across to all my projects before building.
Check out the RoboCopy build task which is part of the Community Build Tasks library which you can find here. RoboCopy can copy one source file to multiple destinations.
On a side note: why don't you use one AssemblyInfo file on solution level and link to that in your projects if you need the same information in every project? Check out my accepted answer on this question: Automatic assembly version number management in VS2008
Right, well maybe I should attempt to do the things I want to do before asking for help :)
<ItemGroup>
<AssemblyInfoSource
Include="AssemblyInfo.cs;AssemblyInfo.cs" />
<AssemblyInfoDestination
Include="$(Destination1)\AssemblyInfo.cs;$(Destination2)\AssemblyInfo.cs" />
</ItemGroup>
<Copy SourceFiles="#(AssemblyInfoSource)" DestinationFiles="#(AssemblyInfoDestination)" />
I had a need to copy the contents of a directory to multiple locations, this is what I came up with that works. So I am posting it here ins case anyone else is in similar need and comes across this question like I did.
<!-- Create a list of the objects in PublishURL so it will copy to multiple directories -->
<ItemGroup>
<PublishUrls Include="$(PublishUrl)"/>
</ItemGroup>
<PropertyGroup>
<Files>$(OutputPath)\**\*</Files>
</PropertyGroup>
<!-- CopyNewFiles will copy all the files in $(OutputPath) to all the directories in the
in $(PublishUrl). $(PublishUrl) can be a single directory, or a list of directories
separated by a semicolon -->
<Target Name ="CopyNewFiles">
<!-- Get list of all files in the output directory; Cross product this with all
the output directories. -->
<CreateItem Include ="$(Files)"
AdditionalMetadata="RootDirectory=%(PublishUrls.FullPath)">
<Output ItemName ="OutputFiles" TaskParameter ="Include"/>
</CreateItem>
<Message Text="'#(OutputFiles)' -> '%(RootDirectory)\%(RecursiveDir)'"/>
<Copy SourceFiles="#(OutputFiles)"
DestinationFolder ="%(RootDirectory)\%(RecursiveDir)"/>
</Target>
If you want to copy AssemblyInfo.cs to Folders A and B you would set the property Files="AssemblyInfo.cs" and PublishUrls="A;B"
What makes this work is the extra metadata in the CreateItem task AdditionalMetadata="RootDirectory=%(PublishUrls.FullPath)" so for each files found in File it creates 1 entry for each item found in PublishUrls. In your case of a single file the equivelent in writing out the xml would be:
<ItemGroup>
<OutputFiles Include="AssemblyInfo.cs">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="AssemblyInfo.cs">
<RootDirectory>B</RootDirectory>
</OutputFiles>
</ItemGroup>
Now if you copied the contents of a folder that had files 1.txt and 2.txt copied to A and B the equivalent xml would be:
<ItemGroup>
<OutputFiles Include="1.txt">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="2.txt">
<RootDirectory>A</RootDirectory>
</OutputFiles>
<OutputFiles Include="1.txt">
<RootDirectory>B</RootDirectory>
</OutputFiles>
<OutputFiles Include="2.txt">
<RootDirectory>B</RootDirectory>
</OutputFiles>
</ItemGroup>