ItemGroup With Custom Metadata regarding files - msbuild

I’m trying to create a “Files” task item group with a metadata attribute called “TargetPath” populated with the relative path to a file.
Example:
For these paths:
D:\Test\Blah.exe
D:\Test\Config\fun.config
D:\Test\en-US\my.resources.dll
The output should be:
File Target = Blah.exe
File Target = Config\fun.config
File Target = en-US\my.resources.dll
Here is my best attempt... hopefully this makes my question clearer:
<ItemGroup>
<Files Include="d:\test\**\*" >
<TargetPath>%(RecursiveDir)%(Filename)%(Extension)</TargetPath>
</Files>
</ItemGroup>
<Message Text="File Target = #(Files->'%(TargetPath)')"/>
I'd like "TargetPath" populated correctly... currently it appears to be null or empty. Anyone know what I'm missing?
Edit:
Yes, I realize I can do this:
<Message Text="File Target = #(Files->'%(RecursiveDir)%(Filename)%(Extension)')"/>
However, I'm building up this ItemGroup to use the ResolveManifestFiles MSBuild task, which requires that I build up a TaskItem with the TargetPath metadata attribute to be able to customize that value.

You're trying to assign dynamic metadata to an itemgroup before it's created. In your example there's no need to create custom metadata since this information is already part of the well-known metadata, so you can just do:
<ItemGroup>
<Files Include="d:\test\**\*" ></Files>
</ItemGroup>
<Message Text="File Target = #(Files->'%(RecursiveDir)%(Filename)%(Extension)')"/>
Or:
<Message Text="File Target = %(Files.RecursiveDir)%(Files.Filename)%(Files.Extension)"/>
EDIT:
This example uses CreateItem task to dynamically update the itemgroup:
<ItemGroup>
<Files Include="d:\test\**\*" ></Files>
</ItemGroup>
<CreateItem
Include="#(Files)"
AdditionalMetadata="TargetPath=%(RecursiveDir)%(Filename)%(Extension)">
<Output TaskParameter="Include" ItemName="Files"/>
</CreateItem>

Modern MSBuild doesn't require CreateTask (since .NET 3.5).
You can do it like this:
<ItemGroup>
<Files Include="d:\test\**\*" />
<FilesWithMetadata Include="%(Files.Identity)" >
<TargetPath>%(RecursiveDir)%(Filename)%(Extension)</TargetPath>
</FilesWithMetadata>
</ItemGroup>

I am liking the CreateItem method to use like this:
<ItemGroup>
<Reference Include="Newtonsoft.Json, Version=8.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
<HintPath>..\packages\Newtonsoft.Json.8.0.2\lib\net45\Newtonsoft.Json.dll</HintPath>
<Private>True</Private>
</Reference>
...
</ItemGroup>
<CreateItem Include="#(Reference)" Condition="'%(Reference.Private)' == 'True'" AdditionalMetadata="TargetPath=$([System.IO.Path]::GetFileName(%(Reference.HintPath)))">
<Output TaskParameter="Include" ItemName="DLLFiles"/>
</CreateItem>
<Message Text="HintPaths: "#(DLLFiles->'$(OutputPath)%(TargetPath)')"" Importance="high" />
I am using Transforms to just get only the file name.
Output:
HintPaths:
"bin\Release\log4net.dll;bin\Release\Newtonsoft.Json.dll;bin\Release\RabbitMQ.Client.dll;bin\Release\ReflectSoftware.Insight.dll"

Related

Filter an existing itemgroup so it only includes files that match some condition

How do you filter an existing ItemGroup based on a specific condition, such as file extension or the item's metadata?
For this example, I'll use the file extension. I'm trying to filter the 'None' ItemGroup defined by VS so that my target can operate on all files of a given extension.
For example, the following may be defined:
<ItemGroup>
<None Include="..\file1.ext" />
<None Include="..\file2.ext" />
<None Include="..\file.ext2" />
<None Include="..\file.ext3" />
<None Include="..\file.ext4" />
</ItemGroup>
I want to filter the 'None' ItemGroup above so it only includes the ext extension. Note that I do not want to specify all the extensions to exclude, as they'll vary per project and I'm trying to make my target reusable without modification.
I've tried adding a Condition within a target:
<Target Name="Test">
<ItemGroup>
<Filtered
Include="#(None)"
Condition="'%(Extension)' == 'ext'"
/>
</ItemGroup>
<Message Text="None: '%(None.Identity)'"/>
<Message Text="Filtered: '%(Filtered.Identity)'"/>
</Target>
But sadly, it doesn't work. I get the following for output:
Test:
None: '..\file1.ext'
None: '..\file2.ext'
None: '..\file.ext2'
None: '..\file.ext3'
None: '..\file.ext4'
Filtered: ''
<ItemGroup>
<Filtered Include="#(None)" Condition="'%(Extension)' == '.ext'" />
</ItemGroup>
For advanced filtering I suggest using the RegexMatch from MSBuild Community Tasks.
In this Example we will Filter for Versionnumbers
<RegexMatch Input="#(Items)" Expression="\d+\.\d+\.\d+.\d+">
<Output ItemName ="ItemsContainingVersion" TaskParameter="Output" />
</RegexMatch>
Install MSBuild Community Tasks via Nuget: PM> Install-Package MSBuildTasks or download it here
Then Import it in your MSBuild Script:
<PropertyGroup>
<MSBuildCommunityTasksPath>..\.build\</MSBuildCommunityTasksPath>
</PropertyGroup>
<Import Project="$(MSBuildCommunityTasksPath)MsBuild.Community.Tasks.Targets" />

MSBuild reference assembly not being included in build

I can build my project with the following command...
csc /reference:lib\Newtonsoft.Json.dll SomeSourceFile.cs
... but when I use this command...
msbuild MyProject.csproj
... with the following .csproj file my .dll reference isn't included. Any thoughts?
<PropertyGroup>
<AssemblyName>MyAssemblyName</AssemblyName>
<OutputPath>bin\</OutputPath>
</PropertyGroup>
<ItemGroup>
<Compile Include="SomeSourceFile.cs" />
</ItemGroup>
<ItemGroup>
<Reference Include="Newtonsoft.Json">
<HintPath>lib\Newtonsoft.Json.dll</HintPath>
</Reference>
</ItemGroup>
<Target Name="Build">
<MakeDir Directories="$(OutputPath)" Condition="!Exists('$(OutputPath)')" />
<Csc Sources="#(Compile)" OutputAssembly="$(OutputPath)$(AssemblyName).exe" />
</Target>
You didn't get your Reference group hooked up to the Csc task. Also the references the way you specified could not be used directly inside the task. Tasks that ship with MSBuild include ResolveAssemblyReference, that is able to transform short assembly name and search hints into file paths. You can see how it is used inside c:\Windows\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets.
Without ResolveAssemblyReference, the simplest thing that you can do is to write it like this:
<PropertyGroup>
<AssemblyName>MyAssemblyName</AssemblyName>
<OutputPath>bin\</OutputPath>
</PropertyGroup>
<ItemGroup>
<Compile Include="SomeSourceFile.cs" />
</ItemGroup>
<ItemGroup>
<Reference Include="lib\Newtonsoft.Json.dll" />
</ItemGroup>
<Target Name="Build">
<MakeDir Directories="$(OutputPath)" Condition="!Exists('$(OutputPath)')" />
<Csc Sources="#(Compile)" References="#(Reference)" OutputAssembly="$(OutputPath)$(AssemblyName).exe" />
</Target>
Notice that Reference item specifies direct path to the referenced assembly.
What you've done is to overload the default Build target typically imported via Microsoft.CSharp.targets. In the default Build target, it takes the item array #(Compile), in which your .cs source files are resident, and also the #(Reference) array, among other things, and composites the proper call to the C# compiler. You've done no such thing in your own minimal Build target, which effectivly ignores the declaration of #(Reference) and only supplies #(Compile) to the Csc task.
Try adding the References="#(References)" attribute to the Csc task.

Add Custom Metadata to Already-defined ItemGroup from Another ItemGroup

I have the following:
<ItemGroup>
<Files Include="C:\Versioning\**\file.version" />
<ItemGroup>
<ReadLinesFromFile File="%(Files.Identity)">
<Output TaskParameter="Lines" ItemName="_Version"/>
</ReadLinesFromFile>
where each file.version file contains simply one line which is - you guessed it - a version of the form Major.Minor.Build.Revision.
I want to be able to associate each item in the Files ItemGroup with its _Version by adding the latter as metadata, so that I can do something like:
<Message Text="%(Files.Identity): %(Files.Version)" />
and have MSBuild print out a nice list of file-version associations.
Is this possible?
This can be achieved by using target batching to add your Version member to the metadata. This involves moving your ReadLinesFromFile operation to its own target, using the #(Files) ItemGroup as an input.
This causes the target to be executed for each item in your ItemGroup, allowing you to read the contents (i.e. version number) from each individual file and subsequently update that item to add the Version metadata:
<Project DefaultTargets="OutputFilesAndVersions" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Files Include="C:\Versioning\**\file.version" />
</ItemGroup>
<Target Name="OutputFilesAndVersions" DependsOnTargets="RetrieveVersions">
<Message Text="#(Files->'%(Identity): %(Version)')" />
</Target>
<Target Name="RetrieveVersions" Inputs="#(Files)" Outputs="%(Files.Identity)">
<ReadLinesFromFile File="%(Files.Identity)">
<Output TaskParameter="Lines" PropertyName="_Version"/>
</ReadLinesFromFile>
<PropertyGroup>
<MyFileName>%(Files.Identity)</MyFileName>
</PropertyGroup>
<ItemGroup>
<Files Condition="'%(Files.Identity)'=='$(MyFileName)'">
<Version>$(_Version)</Version>
</Files>
</ItemGroup>
</Target>
</Project>

In MSBuild, can I use the String.Replace function on a MetaData item?

In MSBuild v4 one can use functions (like string.replace) on Properties. But how can I use functions on Metadata?
I'd like to use the string.replace function as below:
<Target Name="Build">
<Message Text="#(Files->'%(Filename).Replace(".config","")')" />
</Target>
Unfortunately this outputs as (not quite what I was going for):
log4net.Replace(".config","");ajaxPro.Replace(".config","");appSettings.Replace(".config","");cachingConfiguration20.Replace(".config","");cmsSiteConfiguration.Replace(".config","");dataProductsGraphConfiguration.Replace(".config","");ajaxPro.Replace(".config","");appSettings.Replace(".config","");cachingConfiguration20.Replace(".config","");cmsSiteConfiguration
Any thoughts?
You can do this with a little bit of trickery:
$([System.String]::Copy('%(Filename)').Replace('config',''))
Basically, we call the static method 'Copy' to create a new string (for some reason it doesn't like it if you just try $('%(Filename)'.Replace('.config',''))), then call the replace function on the string.
The full text should look like this:
<Target Name="Build">
<Message Text="#(Files->'$([System.String]::Copy("%(Filename)").Replace(".config",""))')" />
</Target>
Edit: MSBuild 12.0 seems to have broken the above method. As an alternative, we can add a new metadata entry to all existing Files items. We perform the replace while defining the metadata item, then we can access the modified value like any other metadata item.
e.g.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Files Include="Alice.jpg"/>
<Files Include="Bob.not-config.gif"/>
<Files Include="Charlie.config.txt"/>
</ItemGroup>
<Target Name="Build">
<ItemGroup>
<!--
Modify all existing 'Files' items so that they contain an entry where we have done our replace.
Note: This needs to be done WITHIN the '<Target>' (it's a requirment for modifying existing items like this
-->
<Files>
<FilenameWithoutConfig>$([System.String]::Copy('%(Filename)').Replace('.config', ''))</FilenameWithoutConfig>
</Files>
</ItemGroup>
<Message Text="#(Files->'%(FilenameWithoutConfig)')" Importance="high" />
</Target>
</Project>
Result:
D:\temp>"c:\Program Files (x86)\MSBuild\12.0\Bin\MSBuild.exe" /nologo test.xml
Build started 2015/02/11 11:19:10 AM.
Project "D:\temp\test.xml" on node 1 (default targets).
Build:
Alice;Bob.not-config;Charlie
Done Building Project "D:\temp\test.xml" (default targets).
I needed to do something similar, the following worked for me.
<Target Name="Build">
<Message Text="#(Files->'%(Filename)'->Replace('.config', ''))" />
</Target>
Those functions works in properties only (as I know). So create target which will perform operation throw batching:
<Target Name="Build"
DependsOnTargets="ProcessFile" />
<Target Name="ProcessFile"
Outputs="%(Files.Identity)">
<PropertyGroup>
<OriginalFileName>%(Files.Filename)</OriginalFileName>
<ModifiedFileName>$(OriginalFileName.Replace(".config",""))</ModifiedFileName>
</PropertyGroup>
<Message Text="$(ModifiedFileName)" Importance="High"/>
</Target>
Do you really need in your example such kind of task? I mean there exists MSBuild Well-known Item Metadata
EDIT: I should specify that this task processes all items in #(Files).
i dont think you can use functions directly with itemgroups and metadata (that would be easy)
However you can use batching:
Taking the ideas from this post:
array-iteration
I was trying to trim an itemgroup to send to a commandline tool (i needed to lose .server off the filename)
<Target Name="ProcessFile" DependsOnTargets="FullPaths">
<ItemGroup>
<Environments Include="$(TemplateFolder)\$(Branch)\*.server.xml"/>
</ItemGroup>
<MSBuild Projects=".\Configure.msbuild"
Properties="CurrentXmlFile=%(Environments.Filename)"
Targets="Configure"/>
</Target>
<Target Name="Configure" DependsOnTargets="FullPaths">
<PropertyGroup>
<Trimmed>$(CurrentXmlFile.Replace('.server',''))</Trimmed>
</PropertyGroup>
<Message Text="Trimmed: $(Trimmed)"/>
<Exec Command="ConfigCmd $(Trimmed)"/>
</Target>
For MSBuild 12.0, here's an alternative.
<Target Name="Build">
<Message Text="$([System.String]::Copy("%(Files.Filename)").Replace(".config",""))" />
</Target>
Got the same problem (except with MakeRelative), so I passed with another solution : Using good old CreateItem that take a string and transform to Item :)
<ItemGroup>
<_ToUploadFTP Include="$(PublishDir)**\*.*"></_ToUploadFTP>
</ItemGroup>
<CreateItem Include="$([MSBuild]::MakeRelative('c:\$(PublishDir)','c:\%(relativedir)%(filename)%(_ToUploadFTP.extension)'))">
<Output ItemName="_ToUploadFTPRelative" TaskParameter="Include"/>
</CreateItem>
<FtpUpload Username="$(UserName)"
Password="$(UserPassword)"
RemoteUri="$(FtpHost)"
LocalFiles="#(_ToUploadFTP)"
RemoteFiles="#(_ToUploadFTPRelative->'$(FtpSitePath)/%(relativedir)%(filename)%(extension)')"
UsePassive="$(FtpPassiveMode)" ></FtpUpload>

MSBuild Copy task not copying files the first time round

I created a build.proj file which consists of a task to copy files that will be generated after the build is complete. The problem is that these files are not copied the first time round and I have to run msbuild again on the build.proj so that the files can be copied. Please can anyone tell me whats wrong with the following build.proj file:
<Configuration Condition="'$(Configuration)' == ''">Debug</Configuration>
<SourcePath Condition="'$(SourcePath)' == ''">$(MSBuildProjectDirectory)</SourcePath>
<BuildDir>$(SourcePath)\build</BuildDir>
</PropertyGroup>
<ItemGroup>
<Projects
Include="$(SourcePath)\src\myApp\application.csproj">
</Projects>
</ItemGroup>
<Target Name="Build">
<Message text = "Building project" />
<MSBuild
Projects="#(Projects)"
Properties="Configuration=$(Configuration)" />
</Target>
<ItemGroup>
<OutputFiles Include ="$(MSBuildProjectDirectory)\**\**\bin\Debug\*.*"/>
</ItemGroup>
<Target Name="CopyToBuildFolder">
<Message text = "Copying build items" />
<Copy SourceFiles="#(OutputFiles)" DestinationFolder="$(BuildDir)"/>
</Target>
<Target Name="All"
DependsOnTargets="Build; CopyToBuildFolder"/>
</Project>
The itemgroups are evaluated when the script is parsed. At that time your files aren't there yet. To be able to find the files you'll have to fill the itemgroup from within a target.
<!-- SQL Scripts which are needed for deployment -->
<Target Name="BeforeCopySqlScripts">
<CreateItem Include="$(SolutionRoot)\04\**\Databases\**\*.sql">
<Output ItemName="CopySqlScript" TaskParameter="Include"/>
</CreateItem>
</Target>
This example creates the ItemGroup named "CopySqlScript" using the expression in the Include attribute.
Edit:
Now I can read your script: add the CreateItem tag within your CopyToBuildFolder target