How to pass ItemGroup with metadata between targets? - msbuild

From my understanding, using DependsOnTargets is necessary to pass ItemGroup between targets. I am not sure if there are other ways to pass targets without DependsOnTargets.
I have tested that ItemGroup cannot pass from CallTarget or MSBuild task. A workaround solution is convert the ItemGroup to property (flatten it) and use Properties to pass over.
I define a ItemGroup of File. The File has Value metadata. I would like to execute target and remove one File item for each recursive loop. Here is my script:
<Target Name="MyGroup">
<ItemGroup>
<File Include="5">
<Value>5a</Value>
</File>
<File Include="4">
<Value>4a</Value>
</File>
<File Include="3">
<Value>3a</Value>
</File>
<File Include="2">
<Value>2a</Value>
</File>
<File Include="1">
<Value>1a</Value>
</File>
</ItemGroup>
</Target>
<Target Name="Recursive" DependsOnTargets="MyGroup" Condition="$(Value) > 0">
<ItemGroup>
<File Remove="$(Value)" />
</ItemGroup>
<PropertyGroup>
<Value>$([MSBuild]::Subtract($(Value), 1))</Value>
</PropertyGroup>
<Message Text="File: #(File->'%(Value)') Value=$(Value)" Importance="High" />
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=$(Value)"/>
</Target>
<Target Name="Build" DependsOnTargets="MyGroup">
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=5" />
</Target>
The output is:
File: 4a;3a;2a;1a Value=4
File: 5a;3a;2a;1a Value=3
File: 5a;4a;2a;1a Value=2
File: 5a;4a;3a;1a Value=1
File: 5a;4a;3a;2a Value=0
I expect the output to be:
File: 4a;3a;2a;1a Value=4
File: 3a;2a;1a Value=3
File: 2a;1a Value=2
File: 1a Value=1
File: Value=0
Is there a solution?

Not really, you are trying to circumvent idea of msbuild to build target only once (unless inputs\outputs changes).
And you can't pass itemgroups between MSBuild contexts (when you call MSBuild task - new context is created). Also in your script - ItemGroup is scoped to target - you need to define it globally so it'll be available to other targets.
What you can do is:
Define Global property Value with condition to use the one passed to script if exists.
Change ItemGroup from MyGroup target to global scope.
Define conditions on each element of itemgroup to be emitted depending on the value of Value property
e.g. your script will be something like this:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Value Condition="$(Value) == ''">5</Value>
</PropertyGroup>
<ItemGroup>
<File Include="5" Condition="$(Value) >5">
<Value>5a</Value>
</File>
<File Include="4" Condition="$(Value) >4">
<Value>4a</Value>
</File>
<File Include="3" Condition="$(Value) >3">
<Value>3a</Value>
</File>
<File Include="2" Condition="$(Value) >2">
<Value>2a</Value>
</File>
<File Include="1" Condition="$(Value) >1">
<Value>1a</Value>
</File>
</ItemGroup>
<Target Name="Recursive" Condition="$(Value) >0">
<PropertyGroup>
<Value>$([MSBuild]::Subtract($(Value), 1))</Value>
</PropertyGroup>
<Message Text="File: #(File->'%(Value)') Value=$(Value)" Importance="High" />
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=$(Value)"/>
</Target>
<Target Name="Build">
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=5" />
</Target>
</Project>
As Alexey Shcherbak points out, we can't pass ItemGroup from MSBuild task. I resolve my problem by introducing a new property Done to keep a list of item that has processed.
<Target Name="MyGroup">
<ItemGroup>
<File Include="5">
<Value>5a</Value>
</File>
<File Include="4">
<Value>4a</Value>
</File>
<File Include="3">
<Value>3a</Value>
</File>
<File Include="2">
<Value>2a</Value>
</File>
<File Include="1">
<Value>1a</Value>
</File>
</ItemGroup>
</Target>
<Target Name="Recursive" DependsOnTargets="MyGroup" Condition="$(Value) > 0" >
<ItemGroup>
<File Remove="$(Value)" />
<File Remove="$(Done)" />
</ItemGroup>
<Message Text="File: #(File->'%(Value)') Value=$(Value)" Importance="High" />
<PropertyGroup>
<Done>$(Done);$(Value)</Done>
<Value>$([MSBuild]::Subtract($(Value), 1))</Value>
</PropertyGroup>
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=$(Value);Done=$(Done)" />
</Target>
<Target Name="Build" DependsOnTargets="MyGroup">
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" Properties="Value=5" />
</Target>

Related

How to execute MSBuild target recursively?

I attempt to execute the following target but ends up with circular dependency error. I do have a stop condition $(Value) > 0 in target Recursive:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Build">
<Target Name="Recursive" Condition="$(Value) > 0">
<PropertyGroup>
<Value>$([MSBuild]::Subtract($(Value), 1))</Value>
</PropertyGroup>
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Display" />
</Target>
<Target Name="Display">
<Message Text="Value: $(Value)" />
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Recursive" />
</Target>
<PropertyGroup>
<Value>10</Value>
</PropertyGroup>
<Target Name="Build">
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Display" />
</Target>
</Project>
Your stop condition is fine, but you forgot to pass the recalculated Value property into the next recursion. Change your Recursive target like this:
<Target Name="Recursive" Condition="$(Value) > 0">
<PropertyGroup>
<Value>$([MSBuild]::Subtract($(Value), 1))</Value>
</PropertyGroup>
<MSBuild Projects="$(MSBuildProjectFile)" Targets="Display"
Properties="Value=$(Value)" />
</Target>

Transforming multiple config files for multiple projects via MsBuildProj file

I am trying to run multiple commands on a list of files based on a pattern (all files of form *.config under sub directories of a given directory), like so:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="BuildSolution" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<UsingTask TaskName="TransformXml"
AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v12.0\Web\Microsoft.Web.Publishing.Tasks.dll" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Release</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<TransformConfiguration>Release</TransformConfiguration>
<PublishFolder>$(OutDir)_PublishedWebsites</PublishFolder>
</PropertyGroup>
<Target Name="BuildSolution">
<MSBuild Projects="$(MSBuildProjectDirectory)\SolutionName.sln"
ContinueOnError="false"
Properties="PublishProfile=$(TransformConfiguration);DeployOnBuild=true" />
</Target>
<ItemGroup>
<ConfigFiles Include="$(PublishFolder)\**\*.config" />
</ItemGroup>
<Target Name="TransformWebConfig" AfterTargets="BuildSolution"
Condition="'#(ConfigFiles)'!=''"
Outputs="%(ConfigFiles.Identity)">
<PropertyGroup>
<ConfigFile>%(ConfigFiles.Identity)</ConfigFile>
<BackupFile>$([System.IO.Path]::ChangeExtension($(ConfigFile),".Source.config"))</BackupFile>
<TransformFile>$([System.IO.Path]::ChangeExtension($(ConfigFile),$(TransformConfiguration) + ".Source.config"))</TransformFile>
</PropertyGroup>
<Message Text="$(ConfigFile)" />
<Message Text="$(BackupFile)" />
<Message Text="$(TransformFile)" />
<Copy SourceFiles="$(ConfigFile)"
DestinationFiles="$(BackupFile)" />
<Exec Command="attrib -r $(ConfigFile)" />
<TransformXml Source="$(BackupFile)"
Transform="$(TransformFile)"
Destination="$(ConfigFile)"
StackTrace="false" />
</Target>
</Project>
However, the batch processing of the matching files is not performed.
From outputs I have added I see that the property $(PublishFolder) points to the correct directory, however, the item #(ConfigFiles) is left empty.
I also tried manually listing the directory names and configuration file names like so:
<ItemGroup>
<Sites Include="Site1" />
<Sites Include="Site2" />
</ItemGroup>
<ItemGroup>
<ConfigFiles Include="Web" />
<ConfigFiles Include="NLog" />
</ItemGroup>
<Target Name="TransformWebConfig" AfterTargets="BuildSolution">
<PropertyGroup>
<SiteConfigFile>$(PublishFolder)\%(Sites.Identity)\%(ConfigFiles.Identity)</SiteConfigFile>
</PropertyGroup>
<Message Text="$(SiteConfigFile)" />
<Copy SourceFiles="$(SiteConfigFile).config"
DestinationFiles="$(SiteConfigFile).Source.config"/>
<Exec Command="attrib -r $(SiteConfigFile).config" />
<TransformXml Source="$(SiteConfigFile).Source.config"
Transform="$(SiteConfigFile).$(TransformConfiguration).config"
Destination="$(SiteConfigFile).config"
StackTrace="false" />
</Target>
However, in this case, the transform is only applied on one file in one site.
Any idea what to do to get this working?
Similar questions and MSDN references I have gone through:
MSBUild: Copy files with a name based on the original (following a pattern)
MSBuild multiple outputpath
How to invoke the same msbuild target twice with different parameters from within msbuild project file itself
msbuild array iteration
http://msdn.microsoft.com/en-us/library/ms171454.aspx
Edit:
Moving the ItemGroup under the task enabled reading the file list after the files were created, however, now only the first file from the list is transformed:
<ItemGroup>
<ConfigFiles Include="$(PublishFolder)\**\Web.config;$(PublishFolder)\**\NLog.config"
Exclude="$(PublishFolder)\**\Packages.config;$(PublishFolder)\**\*.*.config;$(PublishFolder)\**\bin\*.config" />
</ItemGroup>
<PropertyGroup>
<ConfigFile>%(ConfigFiles.Identity)</ConfigFile>
<BackupFile>$([System.IO.Path]::ChangeExtension($(ConfigFile),".Source.config"))</BackupFile>
<TransformFilePrefix>$([System.String]::Concat($(TransformConfiguration), ".config"))</TransformFilePrefix>
<TransformFile>$([System.IO.Path]::ChangeExtension($(ConfigFile), $(TransformFilePrefix)))</TransformFile>
</PropertyGroup>
<Message Text="$(PublishFolder)" />
<Message Text="#(ConfigFiles)" />
<Message Text="$(ConfigFile)" />
<Message Text="$(BackupFile)" />
<Message Text="$(TransformFile)" />
<Copy SourceFiles="$(ConfigFile)"
DestinationFiles="$(BackupFile)" />
<Exec Command="attrib -r $(ConfigFile)" />
<TransformXml Source="$(BackupFile)"
Transform="$(TransformFile)"
Destination="$(ConfigFile)"
StackTrace="false" />
Found the solution:
Create one target for creating the item group:
<Target Name="ListWebConfigs" AfterTargets="BuildSolution">
<ItemGroup>
<ConfigFiles Include="$(PublishFolder)\**\Web.config;$(PublishFolder)\**\NLog.config"
Exclude="$(PublishFolder)\**\Packages.config;$(PublishFolder)\**\*.*.config;$(PublishFolder)\**\bin\*.config" />
</ItemGroup>
<Message Text="$(PublishFolder)" />
<Message Text="#(ConfigFiles)" />
</Target>
Then another for the actual transforms:
<!-- \x to prevent MSBuild from skipping "because all output files are up-to-date" -->
<Target Name="TransformWebConfig" AfterTargets="ListWebConfigs" Inputs="#(ConfigFiles)" Outputs="%(ConfigFiles.Identity)\x">
<PropertyGroup>
<ConfigFile>%(ConfigFiles.Identity)</ConfigFile>
<BackupFile>$([System.IO.Path]::ChangeExtension($(ConfigFile),".Source.config"))</BackupFile>
<TransformFilePrefix>$([System.String]::Concat($(TransformConfiguration), ".config"))</TransformFilePrefix>
<TransformFile>$([System.IO.Path]::ChangeExtension($(ConfigFile), $(TransformFilePrefix)))</TransformFile>
</PropertyGroup>
<Message Text="$(PublishFolder)" />
<Message Text="#(ConfigFiles)" />
<Message Text="$(ConfigFile)" />
<Message Text="$(BackupFile)" />
<Message Text="$(TransformFile)" />
<Copy SourceFiles="$(ConfigFile)" DestinationFiles="$(BackupFile)" />
<Exec Command="attrib -r $(ConfigFile)" />
<TransformXml Source="$(BackupFile)"
Transform="$(TransformFile)"
Destination="$(ConfigFile)"
StackTrace="false" />
</Target>

FTP Credentials for MSBuild.ExtensionPack.Communication.Ftp

In my AfterBuild script I use the following method to upload the files to the deployment server:
<MSBuild.ExtensionPack.Communication.Ftp
TaskAction="UploadFiles"
Host="localhost"
FileNames="$(SomeFolder)\$(FileToUpload)"
UserName="myUserName"
UserPassword="myPassword"
RemoteDirectoryName="/" />
How can I load these credentials from a text file or an external source? What are the alternatives? I don't want to hard-code ftp credentials into my cproj files.
I used GranadaCoders method to answer my own question:
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='host']/#value">
<Output PropertyName="FtpHost" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='username']/#value">
<Output PropertyName="FtpUserName" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='password']/#value">
<Output PropertyName="FtpPassword" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<Message Text="Attempting to uploade $(GeneratedZipFile) to $(FtpHost) as read from $(FTP_Credentials_File) ..." Importance="high" />
<MSBuild.ExtensionPack.Communication.Ftp TaskAction="UploadFiles" Condition="Exists('$(FTP_Credentials_File)')" Host="$(FtpHost)" FileNames="$(PublicFolderToDropZip)\$(GeneratedZipFile)" UserName="$(FtpUserName)" UserPassword="$(FtpPassword)" RemoteDirectoryName="/" />
Put the values in an external xml file.
Read the values from the xml file into a variable.
Parameters.xml
<?xml version="1.0" encoding="utf-8"?>
<parameters>
<setParameter name="LineNumber1" value="PeanutsAreCool" />
<setParameter name="LineNumber2" value="" />
</parameters>
MyMsbuild_MsBuildExtensions.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="AllTargetsWrapped">
<Import Project="$(MSBuildExtensionsPath)\ExtensionPack\4.0\MSBuild.ExtensionPack.tasks"/>
<PropertyGroup>
<!-- Always declare some kind of "base directory" and then work off of that in the majority of cases -->
<WorkingCheckout>.</WorkingCheckout>
</PropertyGroup>
<Target Name="AllTargetsWrapped">
<CallTarget Targets="ReadXmlPeekValue" />
</Target>
<Target Name="ReadXmlPeekValue">
<!-- ReadAttribute -->
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(WorkingCheckout)\Parameters.xml" XPath="/parameters/setParameter[#name='LineNumber1']/#value">
<Output PropertyName="MyValue1" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<Message Text="MyValue1 = $(MyValue1)"/>
</Target>
</Project>
OR
MyMsbuild_WithCommunityTasks.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="AllTargetsWrapped">
<!--
<UsingTask AssemblyFile="$(ProgramFiles)\MSBuild\MSBuild.Community.Tasks.dll" TaskName="Version"/>
-->
<Import Project="$(MSBuildExtensionsPath32)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<PropertyGroup>
<!-- Always declare some kind of "base directory" and then work off of that in the majority of cases -->
<WorkingCheckout>.</WorkingCheckout>
</PropertyGroup>
<Target Name="AllTargetsWrapped">
<CallTarget Targets="ReadXmlPeekValue" />
</Target>
<Target Name="ReadXmlPeekValue">
<!-- you do not need a namespace for this example, but I left it in for future reference -->
<XmlPeek Namespaces="<Namespace Prefix='peanutNamespace' Uri='http://schemas.microsoft.com/developer/msbuild/2003'/>"
XmlInputPath=".\Parameters.xml"
Query="/parameters/setParameter[#name='LineNumber1']/#value">
<Output TaskParameter="Result" ItemName="Peeked" />
</XmlPeek>
<Message Text="#(Peeked)"/>
<XmlPeek Namespaces="<Namespace Prefix='peanutNamespace' Uri='http://schemas.microsoft.com/developer/msbuild/2003'/>"
XmlInputPath=".\Parameters.xml"
Query="/parameters/setParameter[#name='LineNumber1']/#value">
<Output TaskParameter="Result" PropertyName="PeekedSingle" />
</XmlPeek>
<Message Text="PeekedSingle = $(PeekedSingle) "/>
</Target>
</Project>
EDIT:
You can add some basic error checking for the values.
See URL here:
http://tutorials.csharp-online.net/MSBuild:_By_Example%E2%80%94Dealing_with_MSBuild_Errors
Short example.. note the condition..and how it checks for an empty string.
<Error Text="Unable to connect to webserver" Code="Deploy" Condition=" '$(WebURL)' == '' "/>

How to simplify MSBuild-targets?

<Target Name="ProtobufCompile"
Inputs="#(ProtocCompile)"
Outputs="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs">
<PropertyGroup>
<protooutdir>$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))</protooutdir>
</PropertyGroup>
<Message Text="%(ProtocCompile.Filename)%(ProtocCompile.Extension)" Importance="high" />
<MakeDir Directories="$(protooutdir)" />
<Exec Command="$(ProtobufCompiler) --protoc_dir=${PROTOBUF_PROTOC_EXECUTABLE}/.. --proto_path=%(ProtocCompile.RootDir)%(ProtocCompile.Directory) -output_directory=$(protooutdir) %(ProtocCompile.FullPath)" />
</Target>
<!-- set Intputs and Outputs -->
<Target Name="ProtobufCSharpCompile"
DependsOnTargets="ProtobufCompile">
<CreateItem Include="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs">
<Output TaskParameter="Include" ItemName="Compile"/>
</CreateItem>
</Target>
<Target Name="ProtobufClean"
BeforeTargets="Clean">
<Delete Files="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs" />
</Target>
This is the piece of target-file. How to simplify this code? How to reduce duplicating of string below?
$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))
Since you already specified a property for that value like so:
<PropertyGroup>
<protooutdir>$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))</protooutdir>
</PropertyGroup>
You can just replace references to that string with the property $(protooutdir) like so:
<CreateItem Include="$(protooutdir)%(ProtocCompile.Filename).cs">
and
<Delete Files="$(protooutdir)%(ProtocCompile.Filename).cs" />

Why doesn't Content Remove work for MSBuild ItemGroup?

I have an AfterCompile target defined in my csproj which involves minifying and combining JS and CSS. I then add them to ItemGroup Content and remove the unnecessary files, however the Remove paramter does not seem to work.
<Target Name="AfterCompile">
<ItemGroup>
<JS_Combine Include="js\c??.*.min.js" />
<CSS_Combine Include="css\c??.*.min.css" />
</ItemGroup>
<!-- Combine JS -->
<ReadLinesFromFile File="%(JS_Combine.Identity)">
<Output TaskParameter="Lines" ItemName="JSLines" />
</ReadLinesFromFile>
<WriteLinesToFile File="js\combined.min.js" Lines="#(JSLines)" Overwrite="true" />
<!-- Combine CSS -->
<ReadLinesFromFile File="%(CSS_Combine.Identity)">
<Output TaskParameter="Lines" ItemName="CSSLines" />
</ReadLinesFromFile>
<WriteLinesToFile File="css\combined.min.css" Lines="#(CSSLines)" Overwrite="true" />
<!-- Tidy up -->
<ItemGroup>
<Content Include="js\combined.min.js" />
<Content Include="css\combined.min.css" />
<Content Remove="#(JS_Combine)" />
<Content Remove="#(CSS_Combine)" />
</ItemGroup>
<!-- DEBUG message -->
<Message Text="DEBUG: #(Content)" Importance="high" />
</Target>
The debug message still shows #(Content) as having the unnecessary js files. Can anyone tell me what's happening?
In order to recreate you situation I created this sample script
<Project DefaultTargets="Demo" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Demo">
<ItemGroup>
<JS_Combine Include="js\c01.min.js;js\c02.min.js;js\c03.min.js;" />
<CSS_Combine Include="css\c01.min.css;css\c02.min.css;css\c03.min.css;" />
</ItemGroup>
<ItemGroup>
<Content Include="#(JS_Combine);#(CSS_Combine)"/>
</ItemGroup>
<Message Text="Content Before: #(Content)" Importance="high" />
<!-- Tidy up -->
<ItemGroup>
<Content Include="js\combined.min.js" />
<Content Include="css\combined.min.css" />
<Content Remove="#(JS_Combine)" />
<Content Remove="#(CSS_Combine)" />
</ItemGroup>
<Message Text="-------------------------"/>
<Message Text="Content After: #(Content)" Importance="high" />
</Target>
</Project>
It works for me here is the results:
Project "C:\Data\Development\My Code\Community\MSBuild\RemoveTest\Remove01.proj" on node
1 (default targets).
Demo:
Content Before: js\c01.min.js;js\c02.min.js;js\c03.min.js;css\c01.min.css;css\c02.min.c
ss;css\c03.min.css
-------------------------
Content After: js\combined.min.js;css\combined.min.css
Done Building Project "C:\Data\Development\My Code\Community\MSBuild\RemoveTest\Remove01.
proj" (default targets).
Are you still having issues with this?