I have an ItemGroup that contains some Items that are duplicates of one another. I would like to execute a task on a modified version of all of the Items in the ItemGroup. But, so far, I have not been able to find a way that does not remove the duplicated Items.
Does such exist?
<ItemGroup>
<MyMessage Include="foo" />
<MyMessage Include="bar" />
<MyMessage Include="baz" />
<MyMessage Include="foo" />
<MyMessage Include="baz" />
<MyMessage Include="baz" />
</ItemGroup>
<Target Name="DemoBug">
<Message Importance="High" Text="FIRST VERSION USING ItemGroup ..."/>
<Message Importance="High" Text="#(MyMessage)"/>
<Message Importance="High" Text="SECOND VERSION USING Batching and Metadata ..."/>
<Message Importance="High" Text="someprefix;%(MyMessage.Identity);somesuffix"/>
</Target>
In my example above, the FIRST VERSION output has all the Items in it, but has only executed Message task once on the concatenated values of the Items. So this isn't what I want, at all.
5> FIRST VERSION USING ItemGroup ...
5> foo;bar;baz;foo;baz;baz
The SECOND VERSION is executing Message task several times, like I want, but it is stripping the duplicates, which I don't want it to do.
5> SECOND VERSION USING Batching and Metadata ...
5> someprefix;foo;somesuffix
5> someprefix;bar;somesuffix
5> someprefix;baz;somesuffix
Is what I want to do fundamentally impossible (in which case it's maybe time to switch to PowerShell for my task), or is there some reasonable way to do it?
(In my real task, the Items came from ReadLinesFromFile and, after various processing, will eventually end up in WriteLinesToFile. It works fine, except when duplicated lines are encountered)
Msbuild task batching is not looping. It does what it is supposed to do, grouping the items in batches that have the same metadata values. In your example it's grouping by %(Identity).
Would item transformation work in your case? Something like
<Message Importance="High" Text="#(MyMessage->'someprefix;%(Identity);somesuffix')"/>
Related
Is there a way to overcome Targets one-to-one mapping when you have multiple outputs? Seems like that should be possible, but I cannot find out how, given I'm pretty new to MsBuild I'm probably missing something.
The following piece of msbuild script is from microsoft's documentation. What should I change when I have multiple backup folders? So a list #(BackupFolders) and I would like to keep the incremental behaviour of the build?
<Target Name="Backup" Inputs="#(Compile)"
Outputs="#(Compile->'$(BackupFolder)%(Identity).bak')">
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder)%(Identity).bak')" />
</Target>
First of all, the Inputs and Outputs attributes on the Target node are for incremental builds. They need to have the same amount of entries in order for msbuild to understand which items should be filtered when building. MSBuild checks if the output is already there and up-to-date, and if so, the matching input item is filtered from the input list. If you don't care for incremental building, you can skip this mechanism altogether. If inputs and outputs don't match (or are not present), msbuild will always execute the target with all items, because it can't decide which items lead to which output.
Second, what these attributes expect is a list of items. This doesn't have to be one list, it could be an arbitrary list. So it's perfectly fair to extend your example like this:
<Target Name="Backup" Inputs="#(Compile);#(Compile2)"
Outputs="#(Compile->'$(BackupFolder)%(Identity).bak');#(Compile2->'$(BackupFolder)%(Identity).bak')">
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder)%(Identity).bak')" />
<Copy SourceFiles="#(Compile2)" DestinationFiles=
"#(Compile2->'$(BackupFolder)%(Identity).bak')" />
</Target>
But you want to copy the same items to different backup folders, right? So something like this should do:
<Target Name="Backup">
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder)%(Identity).bak')" />
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder2)%(Identity).bak')" />
</Target>
With two backup folders, an item may actually already be up-to-date in one folder, but missing in the other. You could define one as the "main" backup folder, and tell MSBuild to use this as reference for incremental builds.
Edit: For incremental builds to two locations, probably the easiest solution is to combine two targets, both building incrementally:
<Target Name="Backup" DependsOnTargets="_Backup1;_Backup2">
</Target>
<Target Name="_Backup1" Inputs="#(Compile)"
Outputs="#(Compile->'$(BackupFolder)%(Identity).bak')">
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder)%(Identity).bak')" />
</Target>
<Target Name="_Backup2" Inputs="#(Compile)"
Outputs="#(Compile->'$(BackupFolder2)%(Identity).bak')">
<Copy SourceFiles="#(Compile)" DestinationFiles=
"#(Compile->'$(BackupFolder2)%(Identity).bak')" />
</Target>
For a C++ project, I want to autogenerate a defs.h file with project definitions, such as the date, git commit, ... to automate the versioning process of my application.
Therefore I am trying to create a MSBuild Target that will extract the latest git tag, git commit, and the current date and save it to a temporary gitinfo.txt file.
Another build target will depend on that file and generate a .h file.
In order to avoid unnecessary recompiles of my project, the .h file and for that reason the gitinfo.txt file shall only be rewritten, if any of the information has changes.
So my idea is the following:
Calculate git and date info
If available, read in the existing gitinfo.txt
Compare the calculated values to those in the txt file
If anything has changed, rewrite the gitinfo.txt
I've mastered steps 1. and 2., however I am not sure how to process the values after reading them.
<!-- The purpose of this target is to update gitinfo.txt if git information (commit...) has changed -->
<Target
Name="GetHeaderInfos"
BeforeTargets="ClCompile"
Outputs="$(IntDir)\gitinfo.txt"
>
<!-- Get information about the state of this repo-->
<GitDescribe>
<Output TaskParameter="Tag" PropertyName="NewGitTag" />
<Output TaskParameter="CommitHash" PropertyName="NewGitCommitHash" />
<Output TaskParameter="CommitCount" PropertyName="NewGitCommitCount" />
</GitDescribe>
<!-- Get the current date -->
<Time Format="dd.MM.yyyy">
<Output TaskParameter="FormattedTime" PropertyName="NewBuildDate" />
</Time>
<ReadLinesFromFile File="$(IntDir)\gitinfo.txt" Condition="Exists('$(IntDir)\gitinfo.txt')">
<Output TaskParameter="Lines" ItemName="Version" />
</ReadLinesFromFile>
<!-- Comparison here! HOW TO DO IT PROPERLY -->
<PropertyGroup>
<TagChanged> <!-- `$(NewGitTag)` == `$(Version)[0]` --> </TagChanged>
<!-- Other comparisons -->
</PropertyGroup>
</Target>
And this could be the content of gitinfo.txt
v4.1.4
04fe34ab
1
31.07.2016
I am not quite sure how to compare the values now. I need to compare $(NewGitTag) to the first value in the $(Version) version variable, and so on.
I haven't found an example, that actually accesses the variables after reading them from a file. The official documentation provides no help, nor have I found anything on stackoverflow or the likes.
I only know that the $(Version) variable holds a list, and I can batch process it. How can I compare its content to the defined variables $(NewGitTag), $(NewGitCommitHash), $(NewGitCommitCount) and $(NewBuildDate)?
Suppose we start with this data:
<ItemGroup>
<Version Include="v4.1.4;04fe34ab;1;31.07.2016"/>
</ItemGroup>
<PropertyGroup>
<GitTag>v4.1.4</GitTag>
<GitSHA>04fe34ab</GitSHA>
<Count>1</Count>
<Date>31.07.2016</Date>
</PropertyGroup>
Then here are at least 3 ways to achieve comparision (apart from the one mentioned in the comment) and there are probably other ways as well (I'll post them if I can come up with something else):
Just compare the items
I'm not sure why you want to compare everything seperately when this works just as well: compare the whole ItemGroup at once.
<Target Name="Compare1">
<PropertyGroup>
<VersionChanged>True</VersionChanged>
<VersionChanged Condition="'#(Version)' == '$(GitTag);$(GitSHA);$(Count);$(Date)'">False</VersionChanged>
</PropertyGroup>
<Message Text="VersionChanged = $(VersionChanged)" />
</Target>
Batch and check if there's one difference
Each item of Version is compared with e.g. GitTag via batching. The result will be False;False;False;False if there's a difference, else it will be True;False;False;False. Count the distinct elements and if it's 2 it means we got the latter so GitTag did not change. Note this obviousle only works if each of your source items can never have the same value as one of the other items.
<Target Name="Compare2">
<PropertyGroup>
<TagChanged>True</TagChanged>
<TagChanged Condition="'#(Version->Contains($(GitTag))->Distinct()->Count())' == '2'">False</TagChanged>
</PropertyGroup>
<Message Text="TagChanged = $(TagChanged)" />
</Target>
you can then compare the other items as well and combine the result.
Use an inline task to access items by index
This comes closest to what's in your question, but it does need a bit of inline code.
<UsingTask TaskName="IndexItemGroup" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll" >
<ParameterGroup>
<Items Required="true" ParameterType="Microsoft.Build.Framework.ITaskItem[]"/>
<Index Required="true" ParameterType="System.Int32"/>
<Item Output="true" ParameterType="Microsoft.Build.Framework.ITaskItem"/>
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[Item = Items[ Index ];]]>
</Code>
</Task>
</UsingTask>
<Target Name="Compare3">
<IndexItemGroup Items="#(Version)" Index="1">
<Output PropertyName="OldGitSHA" TaskParameter="Item"/>
</IndexItemGroup>
<PropertyGroup>
<SHAChanged>True</SHAChanged>
<SHAChanged Condition="'$(GitSHA)' == '$(OldGitSHA)'">False</SHAChanged>
</PropertyGroup>
<Message Text="OldGitSHA = $(OldGitSHA), changed = $(SHAChanged)" />
</Target>
I'm trying to create a reusable Target in msbuild, following the basic model outlined in How to invoke the same msbuild target twice?
I'm stuck trying to pass a property that I want interpreted as a list. I haven't found an example online that deals with this situation. As I understand it, the problem is that Properties is already treated as a list item, so it doesn't like having a list item passed in as well. Is there a way to get msbuild to pack and unpack the list correctly here?
Msbuild is complaining with:
error MSB4012: The expression "FilesToZip=#(Scripts)" cannot be used in this context. Item lists cannot be concatenated with other strings where an item list is expected. Use a semicolon to separate multiple item lists.
Here's an example caller:
<Target Name="BuildMigrationZip">
<MSBuild Projects="BuildZip.msbuild"
Targets="BuildZip"
Properties="FilesToZip=#(Scripts);OutputZipFile=$(MigrationPackageFilePath);OutputFolder=$(MigrationPackagePath);Flatten=true"/>
<Message Text="Created database migration zip: $(MigrationPackageFilePath)" Importance="high"/>
</Target>
And the base target:
<Target Name="BuildZip">
<MakeDir Directories="$(OutputFolder)"/>
<Zip Files="#(FilesToZip)"
ZipFileName="$(OutputZipFile)"
Flatten="$(Flatten)"
ParallelCompression="false" />
</Target>
I'm basically at the point of just going back to cut and paste for these, although I want to package up a number of zips here.
UPDATE: The same issue applies to setting Inputs on the reusable target. My question up to this point addresses the raw functionality, but it would be nice to keep dependencies working. So for example:
<Target Name="BuildZip"
Inputs="#(FilesToZip)"
Outputs="$(OutputZipFile)">
<MakeDir Directories="$(OutputFolder)"/>
<Zip Files="#(FilesToZip)"
ZipFileName="$(OutputZipFile)"
Flatten="$(Flatten)"
ParallelCompression="false" />
</Target>
They key is to pass the list around as a property. So when your Scripts list is defined as
<ItemGroup>
<Scripts Include="A"/>
<Scripts Include="B"/>
<Scripts Include="C"/>
</ItemGroup>
then you first convert it into a property (which just makes semicolon seperated items, but msbuild knows how to pass this via the Properties of the MSBuild target) then pass it to the target:
<Target Name="BuildMigrationZip">
<PropertyGroup>
<ScriptsProperty>#(Scripts)</ScriptsProperty>
</PropertyGroup>
<MSBuild Projects="$(MSBuildThisFile)" Targets="BuildZip"
Properties="FilesToZip=$(ScriptsProperty)" />
</Target>
(note I'm using $(MSBuildThisFile) here: you don't necessarily need to create seperate build files for every single target, in fact for small targets like yours it's much more convenient to put it in the same file)
Then in your destination target you turn the property into a list again:
<Target Name="BuildZip">
<ItemGroup>
<FilesToZipList Include="$(FilesToZip)"/>
</ItemGroup>
<Message Text="BuildZip: #(FilesToZipList)" />
</Target>
Output:
BuildZip: A;B;C
Update
When working with Inputs, you cannot pass #(FilesToZip) since that expands to nothing because is not a list: it's a property - which happens to be a number of semicolon-seperated strings. And as such, it is usable for Inputs you just have to expand it as the property it is i.e. $(FilesToZip):
<Target Name="BuildZip"
Inputs="$(FilesToZip)"
Outputs="$(OutputZipFile)">
...
</Target>
Output of second run:
BuildZip:
Skipping target "BuildZip" because all output files are up-to-date with respect to the input files.
I am trying to use an item in a list several times. The list is created from a names of directories using MSBuild extensions:
<MSBuild.ExtensionPack.FileSystem.FindUnder TaskAction="FindDirectories" Recursive="false" Path="path\to\stuff\">
<Output ItemName="AllFoundDirectories" TaskParameter="FoundItems"/>
</MSBuild.ExtensionPack.FileSystem.FindUnder>
I wish to use each item within this list in different instances i.e.:
<Message Text="##teamcity[testStarted name='%(AllFoundDirectories.FileName)']"/>
<!-- do some stuff using %(AllFoundDirectories.FileName) here -->
<Message Text="##teamcity[testFinished name='%(AllFoundDirectories.FileName)' duration='test_duration_in_milliseconds']"/>
The problem I have is that everything in the list is being called the first time MSBuild see's the variable %(AllFoundDirectories.FileName). Is there a way to get a single variable out of the list at a time and use it in various places?
Use target batching on a dependent target.
<Target Name="MakeDirectoryItems">
<MSBuild.ExtensionPack.FileSystem.FindUnder
TaskAction="FindDirectories"
Recursive="false" Path="path\to\stuff\">
<Output
ItemName="AllFoundDirectories"
TaskParameter="FoundItems"
/>
</MSBuild.ExtensionPack.FileSystem.FindUnder>
</Target>
<Target Name="UseDirectoryItemsInBatch"
Outputs="%(AllFoundDirectories.Identity)">
<!--
while inside this target, the value of both #(AllFoundDirectories)
and %(AllFoundDirectories.Meta) will be just the single item
in each batch.
-->
</Target>
<Target Name="Driver"
DependsOnTargets="MakeDirectoryItems;UseDirectoryItemsInBatch">
</Target>
I have a script that attempts to construct an ItemGroup out of all files in a certain directory while excluding files with certain names (regardless of extension).
The list of files to be excluded initially contains file extensions, and I am using Community Tasks' RegexReplace to replace the extensions with an asterisk. I then use this list in the item's Exclude attribute. For some reason the files do not get excluded properly, even though the list appears to be correct.
To try and find the cause I created a test script (below) which has two tasks: first one initialises two properties with the list of file patterns in two different ways. The second task prints both properties and the files resulting from using both these properties in the Exclude attribute.
The properties' values appear to be identical, however the resulting groups are different. How is this possible?
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
DefaultTargets="Init;Test" ToolsVersion="3.5">
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<Target Name="Init">
<ItemGroup>
<OriginalFilenames Include="TestDir\SampleProj.exe"/>
<OriginalFilenames Include="TestDir\SampleLib1.dll"/>
</ItemGroup>
<RegexReplace Input="#(OriginalFilenames)" Expression="\.\w+$" Replacement=".*">
<Output TaskParameter="Output" ItemName="PatternedFilenames"/>
</RegexReplace>
<PropertyGroup>
<ExcludeFilesA>TestDir\SampleProj.*;TestDir\SampleLib1.*</ExcludeFilesA>
<ExcludeFilesB>#(PatternedFilenames)</ExcludeFilesB>
</PropertyGroup>
</Target>
<Target Name="Test">
<Message Text='ExcludeFilesA: $(ExcludeFilesA)' />
<Message Text='ExcludeFilesB: $(ExcludeFilesB)' />
<ItemGroup>
<AllFiles Include="TestDir\**"/>
<RemainingFilesA Include="TestDir\**" Exclude="$(ExcludeFilesA)"/>
<RemainingFilesB Include="TestDir\**" Exclude="$(ExcludeFilesB)"/>
</ItemGroup>
<Message Text="
**AllFiles**
#(AllFiles, '
')" />
<Message Text="
**PatternedFilenames**
#(PatternedFilenames, '
')" />
<Message Text="
**RemainingFilesA**
#(RemainingFilesA, '
')" />
<Message Text="
**RemainingFilesB**
#(RemainingFilesB, '
')" />
</Target>
</Project>
Output (reformatted somewhat for clarity):
ExcludeFilesA: TestDir\SampleProj.*;TestDir\SampleLib1.*
ExcludeFilesB: TestDir\SampleProj.*;TestDir\SampleLib1.*
AllFiles:
TestDir\SampleLib1.dll
TestDir\SampleLib1.pdb
TestDir\SampleLib2.dll
TestDir\SampleLib2.pdb
TestDir\SampleProj.exe
TestDir\SampleProj.pdb
PatternedFilenames:
TestDir\SampleProj.*
TestDir\SampleLib1.*
RemainingFilesA:
TestDir\SampleLib2.dll
TestDir\SampleLib2.pdb
RemainingFilesB:
TestDir\SampleLib1.dll
TestDir\SampleLib1.pdb
TestDir\SampleLib2.dll
TestDir\SampleLib2.pdb
TestDir\SampleProj.exe
TestDir\SampleProj.pdb
Observe that both ExcludeFilesA and ExcludeFilesB look identical, but the resulting groups RemainingFilesA and RemainingFilesB differ.
Ultimately I want to obtain the list RemainingFilesA using the pattern generated the same way ExcludeFilesB is generated. Can you suggest a way, or do I have to completely rethink my approach?
The true cause of this was revealed accidentally when a custom task threw an exception.
The actual value of ExcludeFilesA is TestDir\SampleProj.*;TestDir\SampleLib1.* like one might expect. However the actual value of ExcludeFilesB is TestDir\SampleProj.%2a;TestDir\SampleLib1.%2a.
Presumably Message unescapes the string before using it, but Include and Exclude do not. That would explain why the strings look the same but behave differently.
Incidentally, the execution order doesn't seem to have anything to do with this, and I'm pretty sure (following extensive experimentation) that everything gets executed and evaluated exactly in the order in which it appears in this script.
ItemGroups need to be evaluated before targets execution, and the PatternedFilenames ItemGroup is being created on the fly within its target container.
You could workaround this using the CreateItem task, which will ensure the PatternedFilenames scope throughout the execution:
<RegexReplace Input="#(OriginalFilenames)" Expression="\.\w+$" Replacement=".*">
<Output TaskParameter="Output" ItemName="PatternedFilenames_tmp"/>
</RegexReplace>
<CreateItem Include="#(PatternedFilenames_tmp)">
<Output TaskParameter="Include" ItemName="PatternedFilenames"/>
</CreateItem>