It is possible to do this kind of conversion with msbuild - msbuild

It is possible to do this kind of conversion with msbuild? Transforming metadata into items?
This:
<ItemGroup>
<Group Include="G1">
<A>1</A>
<B>1</B>
</Group>
<Group Include="G2">
<A>2</A>
<B>2</B>
</Group>
</ItemGroup>
To this:
<ItemGroup>
<A>1</A>
<A>2</A>
<B>1</B>
<B>2</B>
</ItemGroup>

You can use batching:
This creates the new item groups A abd B based on Group. The new item groups don't have to use the same name as the metadata. Set ItemName in CreateItem/Output to use a different name.
<?xml version="1.0" encoding="utf-8"?>
<Project
xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
ToolsVersion="4.0"
DefaultTargets="Default">
<ItemGroup>
<Group Include="G1">
<A>1</A>
<B>1</B>
</Group>
<Group Include="G2">
<A>2</A>
<B>2</B>
</Group>
</ItemGroup>
<Target Name="Default">
<Message Text="#(Group)" Importance="High" />
<CreateItem Include="%(Group.A)">
<Output TaskParameter="Include" ItemName="A" />
</CreateItem>
<CreateItem Include="%(Group.B)" AdditionalMetadata="From=%(Group.Identity)">
<Output TaskParameter="Include" ItemName="B" />
</CreateItem>
<Message Text="A=#(A)" Importance="High" />
<Message Text="B=#(B):%(B.From)" Importance="High" />
</Target>
</Project>
The new "B" group also defines a metadata item called From that gives each item the original item group name it was copied from.
Update: With msbuild 3.5 or newer you can also use this instead of CreateItem:
<ItemGroup>
<A Include="%(Group.A)" />
<B Include="%(Group.B)">
<From>%(Group.Identity)</From>
</B>
</ItemGroup>

Related

Copying files from multiple directories in msbuild

I have following directories (for example)
./dirA/file1
./dirA/dir/file2
./dirB/file3
./dirB/dir/file4
./dirC/file5
And I want to have them copied to something different, like:
./dirA_renamed/file1
./dirA_renamed/dir/file2
./dirB_renamed_differently/file3
./dirB_renamed_differently/dir/file4
./dirC_renam/file5
The list of directories and their new names is something that does not change very often, but I'd like to use only one Copy.
I tried following:
<ItemGroup>
<ToCopy Include=".\dirA">
<OutputDirName>dirA_renamed</OutputDirName>
</ToCopy>
<ToCopy Include=".\dirB">
<OutputDirName>dirB_renamed_differently</OutputDirName>
</ToCopy>
<ToCopy Include=".\dirC">
<OutputDirName>dirC_renam</OutputDirName>
</ToCopy>
</ItemGroup>
......
<CreateItem Include="%(ToCopy.Directory)\**\*.*">
<Output TaskParameter="Include" ItemName="FilesToCopy" />
</CreateItem>
<Copy SourceFiles="#(FilesToCopy)" DestinationFolder="#(FilesToCopy->'%(OutputDirName)')" />
But nothing happens. If I output FilesToCopy, it is empty. What am I doing wrong?
The hard coded way is the one I know, but I know that you can use parameters, I just haven't figure it out completely yet, to give you a working sample.
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
ToolsVersion="4.0" DefaultTargets="Build">
<PropertyGroup>
<RootDir>C:\DevDir</RootDir>
<SourceA>dirA</SourceA>
<SourceB>dirB</SourceB>
<SourceC>dirC</SourceC>
<RenameA>dirA_renamed</RenameA>
<RenameB>dirB_renamed_differently</RenameB>
<RenameC>dirC_renam</RenameC>
</PropertyGroup>
<Target Name="CopyDirTest" >
<ItemGroup>
<SourceDirA Include="$(RootDir)$(SourceA)\"/>
<SourceDirB Include="$(RootDir)$(SourceB)\" />
<SourceDirC Include="$(RootDir)$(SourceC)\" />
</ItemGroup>
<ItemGroup>
<SourceAFiles Include="$(RootDir)$(SourceA)\**\*.*" />
<SourceBFiles Include="$(RootDir)$(SourceB)\**\*.*" />
<SourceCFiles Include="$(RootDir)$(SourceC)\**\*.*" />
<SourceAllFiles Include="#(SourceAFiles);#(SourceBFiles);#(SourceCFiles)" />
</ItemGroup>
<CreateItem Include="#(SourceAFiles->Replace($(SourceA), $(RenameA)))">
<Output TaskParameter="Include" ItemName="RenamedSourceA" />
</CreateItem>
<CreateItem Include="#(SourceBFiles->Replace($(SourceB), $(RenameB)))">
<Output TaskParameter="Include" ItemName="RenamedSourceB" />
</CreateItem>
<CreateItem Include="#(SourceCFiles->Replace($(SourceC), $(RenameC)))">
<Output TaskParameter="Include" ItemName="RenamedSourceC" />
</CreateItem>
<ItemGroup>
<RenamedAllFiles Include="#(RenamedSourceA);#(RenamedSourceB);#(RenamedSourceC)" />
</ItemGroup>
<Message Text="%(SourceAllFiles.Identity)" Importance="high" />
<Message Text="%(RenamedAllFiles.Identity)" Importance="high" />
<Copy SourceFiles="#(SourceAllFiles)" DestinationFiles="#(RenamedAllFiles)" />
</Target>
</Project>

FTP Credentials for MSBuild.ExtensionPack.Communication.Ftp

In my AfterBuild script I use the following method to upload the files to the deployment server:
<MSBuild.ExtensionPack.Communication.Ftp
TaskAction="UploadFiles"
Host="localhost"
FileNames="$(SomeFolder)\$(FileToUpload)"
UserName="myUserName"
UserPassword="myPassword"
RemoteDirectoryName="/" />
How can I load these credentials from a text file or an external source? What are the alternatives? I don't want to hard-code ftp credentials into my cproj files.
I used GranadaCoders method to answer my own question:
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='host']/#value">
<Output PropertyName="FtpHost" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='username']/#value">
<Output PropertyName="FtpUserName" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(FTP_Credentials_File)" XPath="/parameters/setParameter[#name='password']/#value">
<Output PropertyName="FtpPassword" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<Message Text="Attempting to uploade $(GeneratedZipFile) to $(FtpHost) as read from $(FTP_Credentials_File) ..." Importance="high" />
<MSBuild.ExtensionPack.Communication.Ftp TaskAction="UploadFiles" Condition="Exists('$(FTP_Credentials_File)')" Host="$(FtpHost)" FileNames="$(PublicFolderToDropZip)\$(GeneratedZipFile)" UserName="$(FtpUserName)" UserPassword="$(FtpPassword)" RemoteDirectoryName="/" />
Put the values in an external xml file.
Read the values from the xml file into a variable.
Parameters.xml
<?xml version="1.0" encoding="utf-8"?>
<parameters>
<setParameter name="LineNumber1" value="PeanutsAreCool" />
<setParameter name="LineNumber2" value="" />
</parameters>
MyMsbuild_MsBuildExtensions.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="AllTargetsWrapped">
<Import Project="$(MSBuildExtensionsPath)\ExtensionPack\4.0\MSBuild.ExtensionPack.tasks"/>
<PropertyGroup>
<!-- Always declare some kind of "base directory" and then work off of that in the majority of cases -->
<WorkingCheckout>.</WorkingCheckout>
</PropertyGroup>
<Target Name="AllTargetsWrapped">
<CallTarget Targets="ReadXmlPeekValue" />
</Target>
<Target Name="ReadXmlPeekValue">
<!-- ReadAttribute -->
<MSBuild.ExtensionPack.Xml.XmlFile TaskAction="ReadAttribute" File="$(WorkingCheckout)\Parameters.xml" XPath="/parameters/setParameter[#name='LineNumber1']/#value">
<Output PropertyName="MyValue1" TaskParameter="Value"/>
</MSBuild.ExtensionPack.Xml.XmlFile>
<Message Text="MyValue1 = $(MyValue1)"/>
</Target>
</Project>
OR
MyMsbuild_WithCommunityTasks.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="AllTargetsWrapped">
<!--
<UsingTask AssemblyFile="$(ProgramFiles)\MSBuild\MSBuild.Community.Tasks.dll" TaskName="Version"/>
-->
<Import Project="$(MSBuildExtensionsPath32)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<PropertyGroup>
<!-- Always declare some kind of "base directory" and then work off of that in the majority of cases -->
<WorkingCheckout>.</WorkingCheckout>
</PropertyGroup>
<Target Name="AllTargetsWrapped">
<CallTarget Targets="ReadXmlPeekValue" />
</Target>
<Target Name="ReadXmlPeekValue">
<!-- you do not need a namespace for this example, but I left it in for future reference -->
<XmlPeek Namespaces="<Namespace Prefix='peanutNamespace' Uri='http://schemas.microsoft.com/developer/msbuild/2003'/>"
XmlInputPath=".\Parameters.xml"
Query="/parameters/setParameter[#name='LineNumber1']/#value">
<Output TaskParameter="Result" ItemName="Peeked" />
</XmlPeek>
<Message Text="#(Peeked)"/>
<XmlPeek Namespaces="<Namespace Prefix='peanutNamespace' Uri='http://schemas.microsoft.com/developer/msbuild/2003'/>"
XmlInputPath=".\Parameters.xml"
Query="/parameters/setParameter[#name='LineNumber1']/#value">
<Output TaskParameter="Result" PropertyName="PeekedSingle" />
</XmlPeek>
<Message Text="PeekedSingle = $(PeekedSingle) "/>
</Target>
</Project>
EDIT:
You can add some basic error checking for the values.
See URL here:
http://tutorials.csharp-online.net/MSBuild:_By_Example%E2%80%94Dealing_with_MSBuild_Errors
Short example.. note the condition..and how it checks for an empty string.
<Error Text="Unable to connect to webserver" Code="Deploy" Condition=" '$(WebURL)' == '' "/>

How to simplify MSBuild-targets?

<Target Name="ProtobufCompile"
Inputs="#(ProtocCompile)"
Outputs="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs">
<PropertyGroup>
<protooutdir>$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))</protooutdir>
</PropertyGroup>
<Message Text="%(ProtocCompile.Filename)%(ProtocCompile.Extension)" Importance="high" />
<MakeDir Directories="$(protooutdir)" />
<Exec Command="$(ProtobufCompiler) --protoc_dir=${PROTOBUF_PROTOC_EXECUTABLE}/.. --proto_path=%(ProtocCompile.RootDir)%(ProtocCompile.Directory) -output_directory=$(protooutdir) %(ProtocCompile.FullPath)" />
</Target>
<!-- set Intputs and Outputs -->
<Target Name="ProtobufCSharpCompile"
DependsOnTargets="ProtobufCompile">
<CreateItem Include="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs">
<Output TaskParameter="Include" ItemName="Compile"/>
</CreateItem>
</Target>
<Target Name="ProtobufClean"
BeforeTargets="Clean">
<Delete Files="$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))%(ProtocCompile.Filename).cs" />
</Target>
This is the piece of target-file. How to simplify this code? How to reduce duplicating of string below?
$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))
Since you already specified a property for that value like so:
<PropertyGroup>
<protooutdir>$(IntermediateOutputPath)$([System.Text.RegularExpressions.Regex]::Replace('%(ProtocCompile.RelativeDir)','\.\.[/\\]',''))</protooutdir>
</PropertyGroup>
You can just replace references to that string with the property $(protooutdir) like so:
<CreateItem Include="$(protooutdir)%(ProtocCompile.Filename).cs">
and
<Delete Files="$(protooutdir)%(ProtocCompile.Filename).cs" />

get list of subdirectories in msbuild

Given a list of directories:
<ItemGroup>
<Dirs Include="Foo\Dir1" />
<Dirs Include="Foo\Dir2" />
</ItemGroup>
How can I get a list of all subdirectories.
Transforming this list with "$(Identity)\**" does not match anything and transforming with "$(Identity)\**\*" and then with RelativeDir yields only directories that contain files.
Currently I have to resort to C#:
<UsingTask TaskName="GetSubdirectories" TaskFactory="CodeTaskFactory"
AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Directories ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
<SubDirectories ParameterType="Microsoft.Build.Framework.ITaskItem[]" Output="true" />
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs"><![CDATA[
var result = new List<ITaskItem>();
foreach (var dirItem in Directories) {
foreach (var dir in Directory.GetDirectories(dirItem.ItemSpec, "*", SearchOption.AllDirectories)) {
if (dir.Contains(#"\.svn\") || dir.EndsWith(#"\.svn")) continue;
result.Add(new TaskItem(dir));
}
}
SubDirectories = result.ToArray();
]]></Code>
</Task>
</UsingTask>
<GetSubdirectories Directories="#(Dirs)">
<Output TaskParameter="SubDirectories" ItemName="SubDirs" />
</GetSubdirectories>
But I would like to know if there is an easier way.
Excerpted from the book "MSBuild Trickery":
<Import Project="EnableAllPropertyFunctions.tasks" />
<Target Name="GetSubdirectories">
<ItemGroup>
<Dirs Include="$([System.IO.Directory]::
EnumerateDirectories(
`.\Foo`,
`*`,
System.IO.SearchOption.AllDirectories))"
/>
</ItemGroup>
<Message Text="%(Dirs.Identity)" />
</Target>
You'll need to first enable the extended property function set by ensuring that the environment variable MSBuildEnableAllPropertyFunctions is set to the value 1 (that is what the imported .tasks file accomplishes, with an inline task).
Once #(Dirs) is set up, you can filter it with the Remove attribute to get rid of the Subversion folders.
<CreateItem Include="$(OutputFolder)\*\*.*">
<Output TaskParameter="Include" ItemName="FilesInSubFolders" />
</CreateItem>
<RemoveDuplicates Inputs="#(FilesInSubFolders->'%(RelativeDir)')">
<Output TaskParameter="Filtered" ItemName="SubDirs"/>
</RemoveDuplicates>
<Message Text="#(SubDirs)"/>
This will put all the immediate subfolder paths into #(SubDirs). If you change Include="$(OutputFolder)\*\*.*" to Include="$(OutputFolder)\**\*.*", it'll include all subfolders recursively.
To expand on Brian's answer with a fully self-contained example:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="GetSubdirectories">
<UsingTask TaskName="SetEnvironmentVariable"
TaskFactory="CodeTaskFactory"
AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v$(MSBuildToolsVersion).dll">
<ParameterGroup>
<Name ParameterType="System.String" Required="true" />
<Value ParameterType="System.String" Required="true" />
</ParameterGroup>
<Task>
<Using Namespace="System" />
<Code Type="Fragment" Language="cs"><![CDATA[
Environment.SetEnvironmentVariable(Name, Value);
]]></Code>
</Task>
</UsingTask>
<Target Name="GetSubdirectories">
<SetEnvironmentVariable Name="MSBuildEnableAllPropertyFunctions" Value="1" />
<ItemGroup>
<Dirs Include="$([System.IO.Directory]::EnumerateFiles('.\Stuff', '*', System.IO.SearchOption.AllDirectories))"/>
</ItemGroup>
<Message Text="%(Dirs.Identity)" />
</Target>
</Project>
I got the UsingTask example from this answer.

Why doesn't Content Remove work for MSBuild ItemGroup?

I have an AfterCompile target defined in my csproj which involves minifying and combining JS and CSS. I then add them to ItemGroup Content and remove the unnecessary files, however the Remove paramter does not seem to work.
<Target Name="AfterCompile">
<ItemGroup>
<JS_Combine Include="js\c??.*.min.js" />
<CSS_Combine Include="css\c??.*.min.css" />
</ItemGroup>
<!-- Combine JS -->
<ReadLinesFromFile File="%(JS_Combine.Identity)">
<Output TaskParameter="Lines" ItemName="JSLines" />
</ReadLinesFromFile>
<WriteLinesToFile File="js\combined.min.js" Lines="#(JSLines)" Overwrite="true" />
<!-- Combine CSS -->
<ReadLinesFromFile File="%(CSS_Combine.Identity)">
<Output TaskParameter="Lines" ItemName="CSSLines" />
</ReadLinesFromFile>
<WriteLinesToFile File="css\combined.min.css" Lines="#(CSSLines)" Overwrite="true" />
<!-- Tidy up -->
<ItemGroup>
<Content Include="js\combined.min.js" />
<Content Include="css\combined.min.css" />
<Content Remove="#(JS_Combine)" />
<Content Remove="#(CSS_Combine)" />
</ItemGroup>
<!-- DEBUG message -->
<Message Text="DEBUG: #(Content)" Importance="high" />
</Target>
The debug message still shows #(Content) as having the unnecessary js files. Can anyone tell me what's happening?
In order to recreate you situation I created this sample script
<Project DefaultTargets="Demo" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Demo">
<ItemGroup>
<JS_Combine Include="js\c01.min.js;js\c02.min.js;js\c03.min.js;" />
<CSS_Combine Include="css\c01.min.css;css\c02.min.css;css\c03.min.css;" />
</ItemGroup>
<ItemGroup>
<Content Include="#(JS_Combine);#(CSS_Combine)"/>
</ItemGroup>
<Message Text="Content Before: #(Content)" Importance="high" />
<!-- Tidy up -->
<ItemGroup>
<Content Include="js\combined.min.js" />
<Content Include="css\combined.min.css" />
<Content Remove="#(JS_Combine)" />
<Content Remove="#(CSS_Combine)" />
</ItemGroup>
<Message Text="-------------------------"/>
<Message Text="Content After: #(Content)" Importance="high" />
</Target>
</Project>
It works for me here is the results:
Project "C:\Data\Development\My Code\Community\MSBuild\RemoveTest\Remove01.proj" on node
1 (default targets).
Demo:
Content Before: js\c01.min.js;js\c02.min.js;js\c03.min.js;css\c01.min.css;css\c02.min.c
ss;css\c03.min.css
-------------------------
Content After: js\combined.min.js;css\combined.min.css
Done Building Project "C:\Data\Development\My Code\Community\MSBuild\RemoveTest\Remove01.
proj" (default targets).
Are you still having issues with this?