<Target Name="Build">
...
<MSBuild
Projects="$(MSBuildProjectFile)"
Condition="'#(FilesToCompile)' != ''"
Targets="buildcpp"
Properties="CPPFILE=%(FilesToCompile.FullPath);OBJFILE=$(ObjectFolder)\%(FilesToCompile.Filename).doj;IncludeDirs=$(IncludeDirs)"
/>
FilesToCompile is an ItemGroup of all .cpp files.
When I look at the build log, it shows the target buildcpp being run for each of the files in CPPFILE.
I understand that is what I logically want to happen but my question is, what rule of element <MSBuild> or the MSBuild schema causes task MSBuild to be executed for each value of CPPFILE?
In short, where in the documentation does it state that is what will happen?
I want to pass in an entire ItemGroup once instead of calling the MSBuild target once for each item.
The msbuild concept this is based on is called "batching" - in your case task batching (see MSBuild's task batching documentation).
Any task that contains a %() reference to an item group will be split up into batches that share the same metadata and the task will be executed once for each batch. When using built-in metadata like Identity or FullPath, this essentially means "execute this task for ever item", though there can also be more complex use cases.
Related
I have the following definitions in my working msbuild project...
<MSBuild
Projects="$(MSBuildProjectFile)"
Condition="'#(FilesToCompile)' != ''"
Targets="buildcpp"
Properties="CPPFILE=%(FilesToCompile.FullPath);OBJFILE=$(ObjectFolder)\%(FilesToCompile.Filename).doj;IncludeDirs=$(IncludeDirs)"
/>
...followed by the definition of the target.
Notice how the definition of the target contains a call to another target compilecpp...
<Target Name="buildcpp">
<PropertyGroup>
<CompileDefines Condition="'$(PreprocessorDefinitions)' != ''">-D$(PreprocessorDefinitions.Replace(";"," -D"))</CompileDefines>
</PropertyGroup>
<Exec
EchoOff="true"
StandardOutputImportance="low"
StandardErrorImportance="low"
IgnoreExitCode="true"
ConsoleToMSBuild="true"
Command='
"$(CompilerExe)" ^
$(HWProcessor) ^
$(IncludeDirs) ^
$(CompilerOptions) ^
$(CompileDefines) ^
"$(CPPFILE)" ^
-MM
'>
<Output TaskParameter="ConsoleOutput" PropertyName="output_cppdeps"/>
<Output TaskParameter="ExitCode" PropertyName="exitcode_cppdeps"/>
</Exec>
<MSBuild
Projects="$(MSBuildProjectFile)"
Condition="'$(exitcode_cppdeps)' == '0'"
Targets="compilecpp"
Properties="INPUTFILES=$(BuildCppDeps)"
/>
</Target>
...which uses the property $(OBJFILE) even though it was never passed in by the caller
<Target Name="compilecpp" Inputs="$(INPUTFILES)" Outputs="$(OBJFILE)">
<Message Importance="high" Text="$(CPPFILE): Compiling..."/>
...
QUESTION
Since this msbuild works, I can infer that $(OBJFILE) is accessible; why is it accessible? What are the scope rules for properties?
When using the <MSBuild> task, this performs a new msbuild run similar to running msbuild.exe with arguments. In particular, passing in properties is similar to passing /p:PropName=Value arguments - it defines new "global properties" for this run.
During this inner build, the property is still there and accessible by additional inner builds (buildcpp -> compilecpp) unless overwritten. So OBJFILE is only accessible in compilecpp because it was defined as global property for a parent msbuild run. If compilecpp was somehow invoked directly, the property would not be defined (assuming it not set somewhere else). When you want to stop forwarding a global property, you'd need to use the MSBuild task's RemoveProperties parameter. So if you set RemoveProperties="OBJFILE", then it won't be pased on.
Fyi, in .NET Core projects, RemoveProperties is used to not forward a RuntimeIdentifier from a self-contained apps to referenced projects, which may not be able to build with this property set (due to missing restore information).
For more information, read the properties documentation - especially the section about global properties - and the MSBuild Task documentation (important part is the description for the Properties parameter). However, the fact that global properties are passed on isn't explicitly documented (though implied by the RemoveProperties).
Update: the documentation for global properties was updated to describe this behavior:
Global properties are also forwarded to child projects unless the
RemoveProperties attribute of the MSBuild task is used to specify the
list of proerties not to forward.
I am having a target where i am calling 2 more targets.
Option 1
<Target Name="CoreBuildSubSystem" DependsOnTargets="BuildDotNETSolutions;CopySubSystemDOs;">
</Target>
Option 2
<Target Name="CoreBuildSubSystem">
<MSBuild Targets="BuildDotNETSolutions" BuildInParallel="false"></MSBuild>
<MSBuild Targets="CopySubSystemDOs" BuildInParallel="true"></MSBuild>
</Target>
Are they doing the same activity ? I guess the second method is faster as it is allowing BuildInParallel=True. Is my understanding correct?
These two implementations are effectively the same performance wise as BuildDotNETSolutions and CopySubSystemDOs are executed sequentially.
Stick with Option 1 as it follows better MS Build practices as you state the dependencies of the target in the DependsOnTargets attribute which lets the build engine decide if the targets are out of date if they declare inputs and outputs.
Options 1 is also more maintainable as properties and item groups are naturally accessible and you won’t need to pass them as properties on the MS Build element.
Option 2 currently doesn't allow BuildDotNETSolutions and CopySubSystemDOs to build in parallel. You need to change it as follows to allow the targets to be built in parallel.
However this approach would build the project in a recursive fashion which will make whoever follows you curse your name.
<Target Name="CoreBuildSubSystem">
<MSBuild Projects=$(MSBuildThisFileFullPath)
Targets="BuildDotNETSolutions;CopySubSystemDOs"
BuildInParallel="true" />
</Target>
The MSBuild documentation hints in several places that Items aren't necessarily the same as files.
"MSBuild items are inputs into the build system, and they typically represent files."
"Items are objects that typically represent files."
However, I can't seem to find any examples where Items do not represent files. In particular, I would like to perform batching over a set of non-file items. But every item that I create, even from a custom build task, somehow acquires file-like metadata (FullPath, RootDir, Filename, Extension, etc.). Furthermore, I'm confused about the ramifications of setting the Inputs of a target to a set of items that aren't files, and what to use as that target's Outputs.
Does anybody have an example of using non-file Items to perform batching in MSBuild?
edit
Sorry for taking so long to come up with an example. I understand things a bit more, but I'm still uncertain (and there seems to be a complete lack of documentation about this). Everything here is going off my recollection; I'm not at my work computer right now, so I can't verify any of it.
In my experience, MSBuild doesn't like to build multiple configurations of a .sln file in one go. So, this:
msbuild.exe SampleMSBuild.sln /p:Configuration=Debug%3BRelease
(The encoded semicolon being necessary so that it doesn't try to define multiple properties.)
Produces this:
"D:\src\SampleMSBuild\SampleMSBuild.sln" (default target) (1) ->
(ValidateSolutionConfiguration target) ->
D:\src\SampleMSBuild\SampleMSBuild.sln.metaproj : error MSB4126: The
specified solution configuration "Debug;Release|Any CPU" is invalid.
Please specify a valid solution configuration using the Configuration
and Platform properties (e.g. MSBuild.exe Solution.sln
/p:Configuration=Debug /p:Platform="Any CPU") or leave those properties
blank to use the default solution configuration.
[D:\src\SampleMSBuild\SampleMSBuild.sln]
So, it seems like it should be possible to use batching and items to handle this.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
DefaultTarget="Rebuild"
ToolsVersion="4.0">
<ItemGroup>
<Configurations Include="Debug" />-->
<Configurations Include="Release" />-->
</ItemGroup>
<UsingTask TaskName="LogMetadata"
TaskFactory="CodeTaskFactory"
AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
<ParameterGroup>
<Items ParameterType="Microsoft.Build.Framework.ITaskItem[]"
Required="true" />
</ParameterGroup>
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[
foreach (var item in Items) {
Console.Write(item.ItemSpec);
Console.Write(" {");
foreach (string metadataKey in item.MetadataNames) {
Console.Write(metadataKey);
Console.Write("=\"");
Console.Write(item.GetMetadata(metadataKey).
ToString().Replace("\"", "\\\""));
Console.Write("\" ");
}
Console.WriteLine("}");
}]]>
</Code>
</Task>
</UsingTask>
<Target Name="Rebuild">
<LogMetadata Items="%(Configurations.Identity)" />
</Target>
</Project>
Which produces this:
Debug {
FullPath="D:\src\SampleMSBuild\Debug"
RootDir="D:\"
Filename="Debug"
Extension=""
RelativeDir=""
Directory="src\SampleMSBuild\"
RecursiveDir=""
Identity="Debug"
ModifiedTime=""
CreatedTime=""
AccessedTime=""
}
Release {
FullPath="D:\src\SampleMSBuild\Release"
RootDir="D:\"
Filename="Release"
Extension=""
RelativeDir=""
Directory="src\SampleMSBuild\"
RecursiveDir=""
Identity="Release"
ModifiedTime=""
CreatedTime=""
AccessedTime=""
}
As you can see, the items have all kinds of file metadata attached to them. I can't drop the Include attribute, since it's required, but I could synthesize the items in a custom task. HOWEVER, when I do that, they still somehow magically gain all the same file metadata.
What are the ramifications of this? Since I haven't specified these as Inputs or Outputs to a target, will the file metadata cause any problems? Will the build system skip over targets, or build more than it needs to, because the files specified in the Items' FullPath metadata do not exist? What if those files did exist? Would it cause any problems?
I use items to build several solutions after each other and doing some "manual" task with them, therefore I define my own items:
<ItemGroup>
<MergeConfigurations Include="project1\project1.SDK.sln">
<MergeOutAssemblyName>product1.dll</MergeOutAssemblyName>
<MergePrimaryAssemblyName>project1.Interfaces.dll</MergePrimaryAssemblyName>
<SolutionBinaries>project1\bin\$(FlavorToBuild)</SolutionBinaries>
</MergeConfigurations>
<MergeConfigurations Include="project1\project1.Plugin.sln">
<MergeOutAssemblyName>product1.dll</MergeOutAssemblyName>
<MergePrimaryAssemblyName>project1.Interfaces.dll</MergePrimaryAssemblyName>
<SolutionBinaries>project1\bin\plugin\$(FlavorToBuild)</SolutionBinaries>
</MergeConfigurations>
<ItemGroup>
Then I use a target to take the information and do what is necessary:
<Target Name="MergeSolution"
Inputs="%(MergeConfigurations.Identity)"
Outputs="%(MergeConfigurations.Identity)\Ignore_this">
<PropertyGroup>
<MergeSolution>%(MergeConfigurations.Identity)</MergeSolution>
<MergeOutAssemblyName>%(MergeConfigurations.MergeOutAssemblyName)</MergeOutAssemblyName>
<MergePrimaryAssemblyName>%(MergeConfigurations.MergePrimaryAssemblyName)</MergePrimaryAssemblyName>
<SolutionBinaries>%(MergeConfigurations.SolutionBinaries)</SolutionBinaries>
</PropertyGroup>
[....]
</Target>
Hope this helps to point you in the direction you need.
Tricky question. I'm an MSBuild noob, and found myself needing to understand batching recently, so figured I'd share some of what I found.
In general, yes, it seems that most everywhere you see samples and discussions about ITaskItems they tend to be about files. But the underlying implementation is very flexible and can deal with many other things as well. In my case, I've been working with strings and XML data.
This MS article gives some great examples of non-file ItemGroups and metadata.
This article was the best summary I can find that talks about the mechanics of Items and how they are different than Properties. It also covers the whole bit about # and % syntax, converting between strings and Items, and a hint as to where those file metadata properties are coming from - MSBuild is optimized for it.
Whenever you have an interface used as a parameter to a task or whatever, there is going to be a default implementation of that interface somewhere. My guess is that your code sample is newing up a number of these default objects under the hood and those define the metadata that are created by default. If you were to implement this interface yourself, I'll wager you could change this behavior. Probably beyond the scope of the question though =)
So I'm kinda getting the hang of writing in MSBuild.
I like the idea storing things in ItemGroups because its easy to iterate, and you can have multple fields.
That makes them sort of like a class (more like a struct) but they respond to the iteration syntax of targets and tasks in a way that feels like a lambda expression.
However I continue to run into situation where I want to access a value of a particular item in an item group, that is, to access it like a property.
In that case I run into the issue of how to isolate a single item within the group.
When the item group is batching in a target or task, because of addressing using a metadata name or because of addressing with the common itemgroup name, you can utilize the value of the 'current' item i.e. #(ItemsName) or %(MetaDataName).
But I want to do things like: use an Item group as a System.Configuration class that contains the values of the entries in a section of a config file. Therefore the normal thing would be to name the ItemGroup itself to match the section name in the config file. however, the ItemGroup is not an addressable element that is accessible through the build engine interface, only the items themselves are addressable.
It might be nice to individually name the items in an ItemGroup rather than name them all the same and use the Include or a metadata field like to distinguish among them. This makes them behave like properties in that they are individually addressable as distinct items. so you could easily use their values in Conditions this way: '#(UniqueItemName->'%(Value)').
However, then the iterable features are essentially lost.
To narrow this down, presume i have a config file that gets read into an Item group by and xml task so that the element names in a section become the name of the items in the item group and attributes of each config file element are attributes that become metadata:
<configItemFlag name="displayDebugMessages" value="true" note="use with abandon" />
<configItemFlag name="displaySecurityValueMessages" value="false" note="use with caution" />
When I want to test this in a Condition, I need to narrow it down to something like this:
<Messge Text="Debug Message: you are debugging!" Condition="'#(configItemFlag->'%(Name)')' == 'displayDebugMessages' AND '#(configItemFlag->'%(Value)')' == 'true'/>
But this only evaluates the comparison and frequently does not evaluate to a single boolean.
So is there any way to syntacticly get this down to a dependable test?
Does this work for what you are trying to do?
<ItemGroup>
<ConfigItemFlag Include="displayDebugMessages">
<Value>true</Value>
<Note>use with abandon</Note>
</ConfigItemFlag>
<ConfigItemFlag Include="displaySecurityValueMessages">
<Value>false</Value>
<Note>use with caution</Note>
</ConfigItemFlag>
</ItemGroup>
<Target Name="Build">
<Message
Condition="
'%(ConfigItemFlag.Identity)' == 'displayDebugMessages' AND
'%(Value)' == 'true'"
Text="Debug Message: you are debugging, %(Note)!"
/>
</Target>
Output:
Build:
Debug Message: you are debugging, use with abandon!
(response to comment)
...the only thing I can offer to be able to use meta as properties isn't all that great, unless the target will make heavy use of them throughout. Basically it involves flattening each item to properties by batching on the item and creating local properties with each batch.
<Target Name="BuildOne"
Outputs="%(ConfigItemFlag.Identity)">
<!-- flatten this batch to properties -->
<PropertyGroup>
<_Identity>%(ConfigItemFlag.Identity)</_Identity>
<_Value>%(ConfigItemFlag.Value)</_Value>
<_Note>%(ConfigItemFlag.Note)</_Note>
</PropertyGroup>
<!-- use meta as properties -->
<Message
Condition="
'$(_Identity)' == 'displayDebugMessages' AND
'$(_Value)' == 'true'"
Text="Debug Message: you are debugging, $(_Note)!"
/>
</Target>
<Target Name="Build" DependsOnTargets="BuildOne"
/>
It seems like you are running into some of the limitations of the msbuild scripting language. Have you thought about writing a custom task to perform what you are looking for? That way you would be able to bring the full power of a full programming language to bear against the simple conditional check you want to perform.
I need to create multiple /testcontainer: parameters to feed into a task that exec's MsTest.
I have the following :
<ItemGroup>
<TestFiles Include="$(ProjectPath)\**\UnitTest.*.dll" />
</ItemGroup>
for each match in TestFiles I would like to build a string like so:
"/testcontainer:UnitTest.SomeLibrary1.dll"
"/testcontainer:UnitTest.SomeLibrary2.dll"
"/testcontainer:UnitTest.SomeLibrary3.dll"
I am trying to use the internals of MSBuild without having to create a custom task, is this possible ?
TIA
It really depends on the usage of this afterwards. For example the task that you are sending it to, does it accept in an item list and do you want to invoke it once or multiple times?
If you want to invoke it once then you use the #(...) syntax, and if you want to invoke it many times then you do batching with the %(...) syntax.
To invoke once
<Message Text="Test Files: #(TestFiles->'/testcontainer:%(RecursiveDir)%(Filename)%(Extension)')"/>
To invoke many times
<Message Text="Test Files: /testcontainer:%(TestFiles.RecursiveDir)%(TestFiles.Filename)%(TestFiles.Extension)"/>
More info on batching at http://sedotech.com/Resources#batching
Try this:?
<Message Text="TestFiles= #(TestFiles-> '"%(Fullpath)"', ' ')" />
References:
MSBuild transforms
MSBuild: How to display an item list, separated by a comma.