Include/Exclude specific files using wildcards in MSBuild - msbuild

I have a directory that contains both an uncompressed and a minified copy of jQuery. Depending on my deployment scenario, I may only want to distribute one or the other. Based on Seva Titov's answer, I have an ItemGroup with the following item:
<jQueryFiles Include="$(MSBuildThisFileDirectory)..\..\js\jquery\*.js"
Exclude="$(MSBuildThisFileDirectory)..\..\js\jquery\*.min.js" />
which will just grab jquery.2.1.1.js and exclude the minified version.
For the opposite, it is even simpler with just:
<jQueryFiles Include="$(MSBuildThisFileDirectory)..\..\js\jquery\*.min.js" />
which will only grab the minified version.
But what if I have a scenario where the folder also includes other files to copy like images?
I can generalize copying non-minified files with recursion by using:
<jQueryFiles Include="$(MSBuildThisFileDirectory)..\..\js\jquery\**\*.*"
Exclude="$(MSBuildThisFileDirectory)..\..\js\jquery\**\*.min.*" />
My question now is, how do I achieve the opposite effect and grab all files with ANY extension except plain ".js"? So I might want "jquery\images\foo.png" AND "jquery\jquery.2.1.1.min.js" but I want to exclude the unminified js files.

This is easy to accomplish using Exclude attribute for the item group element. E.g.:
<jQueryFiles
Include="$(MSBuildThisFileDirectory)..\..\js\jquery\*.js"
Exclude="$(MSBuildThisFileDirectory)..\..\js\jquery\*.min.js" />

Related

MsBuild: Flattening of nested ItemGroup with variable number of metadata

Assume the following ItemGroup structure:
<ItemGroup>
<BinaryFiles Include="C:\">
<Binary>a.dll</Binary>
<Binary>b.dll</Binary>
</BinaryFiles>
<BinaryFiles Include="D:\">
<Binary>my.ddl</Binary>
</BinaryFiles>
</ItemGroup>
I need to flatten this to a string like this:
C:\a.dll;C:\b.dll;D:\my.dll
How would I do that? If it's not possible, is there a better way to do it?
A metadata value can have only one value. Multiple definitions and updates will override the value, so the "C:\" item will only have b.dll as Binary metadata.
If there is only one element in the metadata then #(BinaryFiles->'%(Identity)%(Binary)') would yield the result you wanted.
However, since you are using file based logic, you are better off using a BinaryFiles item for each item:
<BinaryFiles Include="C:\*.dll" />
<BinaryFiles Include="D:\*.dll" />
This will scan for all the dll files. You can even use D:\**\*.dll to scan recursively. Then you can use #(BinaryFiles->'%(FullPath') to get the list of all absolute paths.

How can I change multiple files with XMLUpdate?

I'm using XMLUpdate to update multiple config files in subdirectories.
I thought I would be able to do something like this:
<XmlUpdate Namespace="http://schemas.microsoft.com/.NetConfiguration/v2.0"
XmlFileName="\\$(BuildEnvironment)\websites\*.config"
Xpath="//configuration/appSettings/add[#key='Site']/#value"
Value="sitename"
/>
Where I have the following structure:
Websites
|
|-site1\web.config
|
|-site2\web.config
|
|-site3\web.config
So the idea is that rather than writing the xmlupdate task many times, I would be able to use the above and update many config files at once.
Is this possible?
Yes, I'm pretty sure it is possible, but I think you need to use <ItemGroup> for that to get a collection of files. Something like:
<ItemGroup>
<documentation_files Include="\\$(BuildEnvironment)\websites\**web.config" />
</ItemGroup>
<XmlUpdate
XmlFileName="#(documentation_files)"
Xpath="//configuration/appSettings/add[#key='Site']/#value"
Value="sitename" />
I left the Value static, but if you want to change it to the current folder, you can use something like Value="%(documentation_files.RecursiveDir)".
Note: This code is just an example. You may have to change it a bit to get what you want, but I hope it helps you.

How can I use perform MSBuild batching on Items that don't represent files?

The MSBuild documentation hints in several places that Items aren't necessarily the same as files.
"MSBuild items are inputs into the build system, and they typically represent files."
"Items are objects that typically represent files."
However, I can't seem to find any examples where Items do not represent files. In particular, I would like to perform batching over a set of non-file items. But every item that I create, even from a custom build task, somehow acquires file-like metadata (FullPath, RootDir, Filename, Extension, etc.). Furthermore, I'm confused about the ramifications of setting the Inputs of a target to a set of items that aren't files, and what to use as that target's Outputs.
Does anybody have an example of using non-file Items to perform batching in MSBuild?
edit
Sorry for taking so long to come up with an example. I understand things a bit more, but I'm still uncertain (and there seems to be a complete lack of documentation about this). Everything here is going off my recollection; I'm not at my work computer right now, so I can't verify any of it.
In my experience, MSBuild doesn't like to build multiple configurations of a .sln file in one go. So, this:
msbuild.exe SampleMSBuild.sln /p:Configuration=Debug%3BRelease
(The encoded semicolon being necessary so that it doesn't try to define multiple properties.)
Produces this:
"D:\src\SampleMSBuild\SampleMSBuild.sln" (default target) (1) ->
(ValidateSolutionConfiguration target) ->
D:\src\SampleMSBuild\SampleMSBuild.sln.metaproj : error MSB4126: The
specified solution configuration "Debug;Release|Any CPU" is invalid.
Please specify a valid solution configuration using the Configuration
and Platform properties (e.g. MSBuild.exe Solution.sln
/p:Configuration=Debug /p:Platform="Any CPU") or leave those properties
blank to use the default solution configuration.
[D:\src\SampleMSBuild\SampleMSBuild.sln]
So, it seems like it should be possible to use batching and items to handle this.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
DefaultTarget="Rebuild"
ToolsVersion="4.0">
<ItemGroup>
<Configurations Include="Debug" />-->
<Configurations Include="Release" />-->
</ItemGroup>
<UsingTask TaskName="LogMetadata"
TaskFactory="CodeTaskFactory"
AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Items ParameterType="Microsoft.Build.Framework.ITaskItem[]"
Required="true" />
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[
foreach (var item in Items) {
Console.Write(item.ItemSpec);
Console.Write(" {");
foreach (string metadataKey in item.MetadataNames) {
Console.Write(metadataKey);
Console.Write("=\"");
Console.Write(item.GetMetadata(metadataKey).
ToString().Replace("\"", "\\\""));
Console.Write("\" ");
}
Console.WriteLine("}");
}]]>
</Code>
</Task>
</UsingTask>
<Target Name="Rebuild">
<LogMetadata Items="%(Configurations.Identity)" />
</Target>
</Project>
Which produces this:
Debug {
FullPath="D:\src\SampleMSBuild\Debug"
RootDir="D:\"
Filename="Debug"
Extension=""
RelativeDir=""
Directory="src\SampleMSBuild\"
RecursiveDir=""
Identity="Debug"
ModifiedTime=""
CreatedTime=""
AccessedTime=""
}
Release {
FullPath="D:\src\SampleMSBuild\Release"
RootDir="D:\"
Filename="Release"
Extension=""
RelativeDir=""
Directory="src\SampleMSBuild\"
RecursiveDir=""
Identity="Release"
ModifiedTime=""
CreatedTime=""
AccessedTime=""
}
As you can see, the items have all kinds of file metadata attached to them. I can't drop the Include attribute, since it's required, but I could synthesize the items in a custom task. HOWEVER, when I do that, they still somehow magically gain all the same file metadata.
What are the ramifications of this? Since I haven't specified these as Inputs or Outputs to a target, will the file metadata cause any problems? Will the build system skip over targets, or build more than it needs to, because the files specified in the Items' FullPath metadata do not exist? What if those files did exist? Would it cause any problems?
I use items to build several solutions after each other and doing some "manual" task with them, therefore I define my own items:
<ItemGroup>
<MergeConfigurations Include="project1\project1.SDK.sln">
<MergeOutAssemblyName>product1.dll</MergeOutAssemblyName>
<MergePrimaryAssemblyName>project1.Interfaces.dll</MergePrimaryAssemblyName>
<SolutionBinaries>project1\bin\$(FlavorToBuild)</SolutionBinaries>
</MergeConfigurations>
<MergeConfigurations Include="project1\project1.Plugin.sln">
<MergeOutAssemblyName>product1.dll</MergeOutAssemblyName>
<MergePrimaryAssemblyName>project1.Interfaces.dll</MergePrimaryAssemblyName>
<SolutionBinaries>project1\bin\plugin\$(FlavorToBuild)</SolutionBinaries>
</MergeConfigurations>
<ItemGroup>
Then I use a target to take the information and do what is necessary:
<Target Name="MergeSolution"
Inputs="%(MergeConfigurations.Identity)"
Outputs="%(MergeConfigurations.Identity)\Ignore_this">
<PropertyGroup>
<MergeSolution>%(MergeConfigurations.Identity)</MergeSolution>
<MergeOutAssemblyName>%(MergeConfigurations.MergeOutAssemblyName)</MergeOutAssemblyName>
<MergePrimaryAssemblyName>%(MergeConfigurations.MergePrimaryAssemblyName)</MergePrimaryAssemblyName>
<SolutionBinaries>%(MergeConfigurations.SolutionBinaries)</SolutionBinaries>
</PropertyGroup>
[....]
</Target>
Hope this helps to point you in the direction you need.
Tricky question. I'm an MSBuild noob, and found myself needing to understand batching recently, so figured I'd share some of what I found.
In general, yes, it seems that most everywhere you see samples and discussions about ITaskItems they tend to be about files. But the underlying implementation is very flexible and can deal with many other things as well. In my case, I've been working with strings and XML data.
This MS article gives some great examples of non-file ItemGroups and metadata.
This article was the best summary I can find that talks about the mechanics of Items and how they are different than Properties. It also covers the whole bit about # and % syntax, converting between strings and Items, and a hint as to where those file metadata properties are coming from - MSBuild is optimized for it.
Whenever you have an interface used as a parameter to a task or whatever, there is going to be a default implementation of that interface somewhere. My guess is that your code sample is newing up a number of these default objects under the hood and those define the metadata that are created by default. If you were to implement this interface yourself, I'll wager you could change this behavior. Probably beyond the scope of the question though =)

CruiseControl.net dynamicValue for sourcecontrol branch

maybe i'm just not seeing it, but i'd like a way to "inject" the value for branch (externally somehow) for a set of predefined build projects.
we have builds configured for Project1, Project2 and Project3. But at any time, the projects may take from a different branch, based on merge schedules. i'd like to store the Project=>branch mapping in either an external file or database, then dynamically inject it into the config file when we do a ForceBuild.
The following block is used in all 3 project config files, which are referenced at the bottom (end) of the cnet.config file.
<cb:define name="cvs-block">
<sourcecontrol type="cvs">
<cvsroot>:sspi;username=johnDoe;password=passTheSalt;hostname=127.0.0.1;port=1776:/$(repository)</cvsroot>
<module>"$(module)"</module>
<executable>c:\Program Files (x86)\cvsnt\cvs.exe</executable>
<workingDirectory>D:\CruiseBuild\$(workingDir)</workingDirectory>
<branch>[SOME EXTERNALLY DYNAMIC VALUE]</branch>
<autoGetSource>true</autoGetSource>
<timeout units="minutes">20</timeout>
</sourcecontrol>
</cb:define>
<cb:include href="D:\CruiseBuild\ACME-project1.xml" xmlns:cb="urn:ccnet.config.builder" />
<cb:include href="D:\CruiseBuild\ACME-project2.xml" xmlns:cb="urn:ccnet.config.builder" />
<cb:include href="D:\CruiseBuild\ACME-project3.xml" xmlns:cb="urn:ccnet.config.builder" />
Just generate a file (injectpath.config) with a defined value:
<cb:define branchpath="yourpath"/>
...and then include it into your config file shown above at the top.
Change your [SOME DYNAMIC VALUE] to $(branchpath).

Dynamic Metadata Assignment in an ItemGroup

I have an ItemGroup defined as:
<ItemGroup>
<ProtoFiles Include="Protos\*.proto"/>
</ItemGroup>
It yields a list of all .proto files in a directory of my project. I want each item in the group to include a piece of metadata that specifies the name of the file that will be generated based on the .proto file. I know I can do this:
<ItemGroup>
<ProtoFiles Include="Protos\*.proto">
<OutputFile>%(ProtoFiles.Filename).cs</OutputFile>
</ProtoFiles>
</ItemGroup>
But my problem is that it is not a simple mapping from .proto filename to output filename. There is some tricky logic involved that I need to encapsulate somewhere and call that when assigning metadata. I need something like:
<ItemGroup>
<ProtoFiles Include="Protos\*.proto">
<OutputFile><GetOutputFilename ProtoFilename="%(ProtoFiles.Filename)"/></OutputFile>
</ProtoFiles>
</ItemGroup>
The idea being that my custom GetOutputFilename task would be called in order to get the metadata value.
Is this possible? Am I barking up the wrong tree?
I think it's not, try instead passing the ItemGroup to a task to generate this metadata. Property Functions can operate on metadata values, but unfortunately cannot be used to define metadata.
It's hard to know if the logic is too tricky for MSBuild without knowing exactly what it is. Do you have a custom task that operates on #(ProtoFiles) to generate the output files? If so, why not alter your task (or refactor to a new one) that just calculates the output files without creating them, something like this,
<ProtoTask
Files="#(ProtoFiles)"
... other params
DryRun="true">
<Output
TaskParameter="OutputFiles"
ItemName="ProtoFiles" />
</ProtoFiles>
The task can clone the item array, calculate the metadata value, and assign it to the output item array, which in the example here overwrites the original item array passed into the task.