MSBuild Validating Properties - msbuild

I'm working on a reusable MSBuild Target that will be consumed by several other tasks. This target requires that several properties be defined. What's the best way to validate that properties are defined, throwing an Error if the are not?
Two attempts that I almost like:
<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="3.5" DefaultTarget="Release" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Release">
<Error
Text="Property PropA required"
Condition="'$(PropA)' == ''"/>
<Error
Text="Property PropB required"
Condition="'$(PropB)' == ''"/>
<!-- The body of the task -->
</Target>
</Project>
Here's an attempt at batching. It's ugly because of the extra "Name" parameter. Is it possible to use the Include attribute instead?
<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="3.5" DefaultTarget="Release" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Release">
<!-- MSBuild BuildInParallel="true" Projects="#(ProjectsToBuild)"/ -->
<ItemGroup>
<RequiredProperty Include="PropA"><Name>PropA</Name></RequiredProperty>
<RequiredProperty Include="PropB"><Name>PropB</Name></RequiredProperty>
<RequiredProperty Include="PropC"><Name>PropC</Name></RequiredProperty>
</ItemGroup>
<Error
Text="Property %(RequiredProperty.Name) required"
Condition="'$(%(RequiredProperty.Name))' == ''" />
</Target>
</Project>

Great question! I have written about this in depth in my book and in a blog post, Elements of Reusable MSBuild Scripts: Validation. My approach will cover properties and items.
Here is the run down. In the shared .targets file create a validation target, and this should be one of the first targets declared in the file so that users can easily locate it.
Properties
Inside the validation target define your properties like this:
<_RequiredProperties Include="Root">
<Value>$(Root)</Value>
</_RequiredProperties>
I place the name of the property in the include and its value inside of the Value metadata.The reason why I do this is so that I can detect when Value is blank and then I use the include value to report the name of the missing property back to the user.
Items
Inside the target place the required items inside of an item like:
<_RequiredItems Include="AllConfigurations">
<RequiredValue>#(AllConfigurations)</RequiredValue>
</_RequiredItems>
Similar to the properties, inside the include you place the name of the item and then the value to check inside of RequiredValue metadata. In this example it just checks to ensure the the AllConfiguraitons item is not empty. If you want to make sure that a given metadata value is specified on all items then do something like:
<_RequiredItems Include = "AllConfigurations.Configuration">
<RequiredValue>%(AllConfigurations.Configuration </RequiredValue>
</_RequiredItems>
If you want to make sure that a file exists then add the additional metadata, RequiredFilePath.
<_RequiredItems Include ="ProjectsToBuild">
<RequiredValue>%(ProjectsToBuild.Identity)</RequiredValue>
<RequiredFilePath>%(ProjectsToBuild.Identity)</RequiredFilePath>
</_RequiredItems>
Validation
Here is what you need to perform the validation
Complete example
Here is the full example
<Target Name="ValidateBuildSettings">
<ItemGroup>
<_RequiredProperties Include="Root">
<Value>$(Root)</Value>
</_RequiredProperties>
<_RequiredProperties Include="BuildInstallRoot">
<Value>$(BuildInstallRoot)</Value>
</_RequiredProperties>
<_RequiredProperties Include="SourceRoot">
<Value>$(SourceRoot)</Value>
</_RequiredProperties>
<!--
_RequiredItems is the item where required items should be placed.
The following metadata is significant:
REQUIRED METADATA:
Identity = This will basically be used to identify the specific required item
RequiredValue = This is the specific value that will be validated to exist
OPTIONAL METADATA
RequiredFilePath = Populate this with a path that should exists, if it is not empty
then it will be checked to exist on disk.
-->
<_RequiredItems Include="AllConfigurations">
<RequiredValue>#(AllConfigurations)</RequiredValue>
</_RequiredItems>
<_RequiredItems Include = "AllConfigurations.Configuration">
<RequiredValue>%(AllConfigurations.Configuration </RequiredValue>
</_RequiredItems>
<_RequiredItems Include ="ProjectsToBuild">
<RequiredValue>%(ProjectsToBuild.Identity)</RequiredValue>
<RequiredFilePath>%(ProjectsToBuild.Identity)</RequiredFilePath>
</_RequiredItems>
</ItemGroup>
<!-- Raise an error if any value in _RequiredProperties is missing -->
<Error Condition =" '%(_RequiredProperties.Value)'=='' "
Text=" Missing required property [%(_RequiredProperties.Identity)]" />
<!-- Raise an error if any value in _RequiredItems is empty -->
<Error Condition = " '%(_RequiredItems.RequiredValue)'=='' "
Text = " Missing required item value [%(_RequiredItems.Identity)] " />
<!-- Validate any file/directory that should exist -->
<Error Condition = " '%(_RequiredItems.RequiredFilePath)' != '' and !Exists('%(_RequiredItems.RequiredFilePath)') "
Text = " Unable to find expeceted path [%(_RequiredItems.RequiredFilePath)] on item [%(_RequiredItems.Identity)] " />
</Target>

Related

How to reference ItemGroup Identity within the definition of the var

I have a target like below. It needs to replace content of a file with new content. I have multiple files I am matching with ItemGroup.
I couldn't figure out a way to get this working.
Here is my target definition.
<ItemGroup>
<PRSetting Include="$(settings_root)\**\settings_config_*.xml">
<NewContent>$([System.IO.File]::ReadAllText('%(Identity)')).Replace('[config.version]', '$(PR_Version)'))</NewContent>
</PRSetting>
</ItemGroup>
<Target Name="PrepSettings" Inputs="#(PRSetting)"
Outputs="#(PRSetting->'$out\$Filename.xml')" >
<Message Text="%(PRSetting.Identity) new contents:" />
<Message Text="%(PRSetting.NewContent)"/>
</Target>
I hope I explained it right what I am trying to do. When the target is built, I am getting an error that the path to File::ReadFile() can't be empty string. I am using VS 2019. This is work in progress. I am yet to figure out how to save the new content in destination file.
Update
I have the Itemgroup outside. I updated the question. The reason it is outside is because the target inputs parameter needs it.
Try the following and see if it works:
<Target Name="PrepSettings">
<ItemGroup>
<PRSetting Include="$(settings_root)\**\settings_config_*.xml" />
<PRSetting>
<NewContent Condition="%(Identity) != ''">$([System.IO.File]::ReadAllText('%(Identity)')).Replace('[config.version]', '$(PR_Version)'))</NewContent>
</PRSetting>
</ItemGroup>
<Message Text="#(PRSetting.Identity) new contents:" />
<Message Text="%(PRSetting.NewContent)"/>
</Target>
There are two changes:
There seems to be an issue with an Include that doesn't use an existing ItemGroup and metadata that is self-referencing. So, setting up PRSetting is split in two.
First, establish PRSetting with the Include.
Second, revisit to add the NewContent metadata item.
Add Condition="%(Identity) != ''" on the NewContent metadata.
I'm not able to fully test your exact scenario at present but I tested an analogue.
Here is my test analogue:
<Target Name="PrepSettings">
<ItemGroup>
<PRSetting Include="1;4;2;3"/>
<PRSetting>
<NewContent Condition="%(Identity) != ''">$([MSBuild]::Add('%(Identity)', '1'))</NewContent>
</PRSetting>
</ItemGroup>
<Message Text="PRSetting is #(PRSetting->'(%(Identity),%(NewContent))');" />
</Target>
The output is
PrepSettings:
PRSetting is (1,2);(4,5);(2,3);(3,4);
Regarding your code change to move the ItemGroup outside the target:
The technique of splitting the ItemGroup as written won't work, but if you are using VS2017 or later and working with .NET Core/.NET 5+ you can use Update.
As shown in the question, the Outputs attribute has syntax errors. I assume Outputs="#(PRSetting->'$(OutputPath)\%(Filename).xml')" (or something close) is intended.
As shown in the question, the Outputs attribute will never be satisfied because PrepSettings doesn't create the files. I assume PrepSettings as shown is not complete.

Msbuild- 'DependsOnTargets' that contain condition

I tried to have a condition on a Target tag, but resulted with the error:
target has a reference to item metadata. References
to item metadata are not allowed in target conditions unless they are part of an item transform.
So i found this work around:
How to add item transform to VS2012 .proj msbuild file
and tried to implement it, but i can't figure up what i am doing wrong because it's not working as expected.
<CallTarget Targets="CopyOldWebConfigJs" />
<Target Name="CopyOldWebConfigJs"
Inputs="#(ContentFiltered)"
Outputs="%(Identity).Dummy"
DependsOnTargets="webConfigJsCase">
<Message Text="web.config.js Case" />
</Target>
<!-- New target to pre-filter list -->
<Target Name="webConfigJsCase"
Inputs="#(FileToPublish)"
Outputs="%(Identity).Dummy">
<ItemGroup>
<ContentFiltered Condition="$([System.Text.RegularExpressions.Regex]::IsMatch('%(FileToPublish.Filename)%(FileToPublish.Extension)', 'web.config.js'))" />
</ItemGroup>
</Target>
I thought that Inputs="#(ContentFiltered)" will contain the lines that DependsOnTargets="webConfigJsCase" find.
But when i run it , i am getting this message: Skipping target "CopyOldWebConfigJs" because it has no inputs.
I know for a fact that the regex work, and it do find a filename_ext that equals web.config.js so it return True
What do i do or understand wrong?
In <ItemGroup><Item/></ItemGroup>, no change will be made to the Item item because no action was specified. If you want to add entries to the item, you must specify Include="".
The <Item/> documentation describes the various attributes for item elements inside of an <ItemGroup/>. Note that at the top-level of an MSBuild file, directly under the <Project/> element, you would use the attributes Include and Exclude while in a <Target/> you would use the attributes Include and Remove. Not including any attributes at all is nonsensical and—as far as I know—no different than simply deleting the entire line. I am surprised MSBuild doesn’t throw an error or warning this is almost certainly a mistake and not intentional.
The Inputs and Outputs attributes on your <Target Name="webConfigJsCase"/> are unnecessary. In fact, they slow MSBuild down by making it loop over the target unnecessarily. You can filter just in the <Item/> like this:
<Target Name="webConfigJsCase">
<ItemGroup>
<ContentFiltered Condition="'%(FileToPublish.Filename)%(FileToPublish.Extension)' == 'web.config.js'" Include="#(FileToPublish)" />
</ItemGroup>
</Target>
Additionally, I assume that you intended your regular expression to match web.config.js but not match webaconfigbjs. You don’t need to use an advanced feature like Regular Expressions here because MSBuild’s built-in condition operators already support simple string comparison. If fixed the condition above to be more readable.

How to use property functions inside transform modifiers in msbuild?

I have not found a correct syntax that would allow a transform modifiers to use property functions. For example, if we want to trim each filename in a list starting with the "lib" string:
<ListWithoutLib>#(MyOriginalList->%(Filename.TrimStart("lib"))</ListWithoutLib>
Is there any msbuild voodoo that could be written to accomplish this?
There is more than one issue in the question.
First, TrimStart doesn't have an overload that accepts a string and it doesn't remove a substring from the beginning of a string. Instead TrimStart accepts a set of chars and remove all instances of each char from the beginning of string.
For example in C#
"libFoo.dll".TrimStart('l', 'i', 'b')
will produce 'Foo.dll' and
"bilbobaggins.dll".TrimStart('l', 'i', 'b')
will produce 'obaggins.dll'.
But that issue is secondary to the question being asked.
You cannot use a property function inside of a metadata transform but you can apply a method of the string class as an item function.
<ItemGroup>
<MyOriginalList Include="libapple;libboat;cat;dog" />
</ItemGroup>
<PropertyGroup>
<ListWithoutLib>#(MyOriginalList->TrimStart('l'))</ListWithoutLib>
</PropertyGroup>
<Target Name="Default">
<Message Text="ListWithoutLib = $(ListWithoutLib)" />
</Target>
<!--
Output:
Default:
ListWithoutLib = ibapple;ibboat;cat;dog
-->
See String item functions.
This example shows how TrimStart can be called on an item collection but it doesn't do what you indicated that you need.
A solution to what you seem to need might be as follows:
<ItemGroup>
<SourceList Include="libapple;libboat;cat;dog"/>
</ItemGroup>
<PropertyGroup>
<libPrefix>lib</libPrefix>
</PropertyGroup>
<ItemGroup>
<MyOriginalList Include="#(SourceList)">
<IsStartsWithLibPrefix>$([System.String]::Copy('%(Filename)').StartsWith('$(libPrefix)'))</IsStartsWithLibPrefix>
<NameWithoutPrefix Condition="!'%(IsStartsWithLibPrefix)'">%(Filename)</NameWithoutPrefix>
<NameWithoutPrefix Condition="'%(IsStartsWithLibPrefix)'">$([System.String]::Copy('%(Filename)').Substring($(libPrefix.Length)))</NameWithoutPrefix>
</MyOriginalList>
</ItemGroup>
<Target Name="Default">
<Message Text="List of Filename:" />
<Message Text="#(MyOriginalList->' %(Filename)','%0d%0a')" />
<Message Text="List of NameWithoutPrefix:" />
<Message Text="#(MyOriginalList->' %(NameWithoutPrefix)','%0d%0a')" />
</Target>
<!--
Output:
Default:
List of Filename:
libapple
libboat
cat
dog
List of NameWithoutPrefix:
apple
boat
cat
dog
-->
There is a difference between item functions and property functions. Item functions are limited to being called on a collection reference, e.g. #(MyOriginalList). Item batching could be used to split the collection into references to the items with and without the lib prefix. Batching requires a target or task to batch over. The code that is shown forgoes using item functions and instead uses property functions. This allows all the 'work' to be done in the ItemGroup definition.
Assuming the inputs are not static, we need to determine which filenames have the prefix.
Although metadata values are strings, string methods can't be called on metadata values. A well known work-around is to create a new string object from the metadata value and then call a string method. [System.String]::Copy('%(Filename)') is creating a string object that is a copy of the value of the Filename metadata. The code $([System.String]::Copy('%(Filename)').StartsWith('$(libPrefix)')) is a property function call and the IsStartsWithLibPrefix metadata will be set to either True or False.
Because ItemGroups can be self-referential, we can use the IsStartsWithLibPrefix metadata to define the NameWithoutPrefix metadata. The NameWithoutPrefix definition has a condition. For a given item in the MyOriginalList ItemGroup only one variant of the NameWithoutPrefix definition will be used. For Items that have the prefix another property function call is used. Substring is called to create a new string instance that doesn't have the prefix.

MSBuild: How to update default metadata in an item?

I have the following problem in scripting with MSBuild:
I create a default item "itemA" with two metadata "metadata1" and "metadata2", whereby metadata2 refers to metadata1.
When I define itemA later and overwrite metadata1, the metadata2 contains still the default value of metadata1. How can I make the metadata2 to refer to the "new" metadata1?
Illustration in code as below:
<ItemDefinitionGroup>
<itemA>
<Metadata1>default</Metadata1>
<Metadata2>%(itemA.Metadata1)</Metadata2>
</itemA>
</ItemDefinitionGroup>
<ItemGroup>
<itemA Include="first" >
<Metadata1>m_data1</Metadata1>
</itemA>
</ItemGroup>
But see the print
<Message Text="itemA.Metadata1 = %(itemA.Metadata1)" />
<Message Text="itemA.Metadata2 = %(itemA.Metadata2)" />
delivers:
itemA.Metadata1 = m_data1 ***<-- correctly updated***
itemA.Metadata2 = default ***<-- why showing the default value, not* m_data1??**
how can I make itemA.Metadata2 to have the same value as itemA.Metadata1 after it has been updated?
I think this is not possible because order of evaluation Item Definitions - Value Sources - Note:
Item metadata from an ItemGroup is not useful in an ItemDefinitionGroup metadata declaration because ItemDefinitionGroup elements are processed before ItemGroup elements.
You have to override itemA's Metadata2 value in ItemGroup
<ItemDefinitionGroup>
<itemA>
<Metadata1>default</Metadata1>
<Metadata2>%(Metadata1)</Metadata2>
</itemA>
</ItemDefinitionGroup>
<ItemGroup>
<itemA Include="first" >
<Metadata1>m_data1</Metadata1>
<Metadata2>%(Metadata1)</Metadata2>
</itemA>
</ItemGroup>
As palo states, since Metadata2 has already been evaluated, you'll have to explicitly overwrite the value. Your change to Metadata1 won't automatically propagate to other places where it was referenced during initialization.
However, you can "re-evaluate" your items' metadata by starting a new instance of MSBuild and passing the updated metadata in as a property. Running msbuild /t:Wrapper on this project from the command line will result in Metadata1 and Metadata2 printing the same value:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<DefaultMetadata1 Condition="DefaultMetadata1==''">default</DefaultMetadata1>
</PropertyGroup>
<ItemDefinitionGroup>
<itemA>
<Metadata1>$(DefaultMetadata1)</Metadata1>
<Metadata2>%(itemA.Metadata1)</Metadata2>
</itemA>
</ItemDefinitionGroup>
<ItemGroup>
<itemA Include="first" >
<Metadata1>m_data1</Metadata1>
</itemA>
</ItemGroup>
<Target Name="Wrapper">
<MSBuild
Projects="$(MSBuildProjectFile)"
Targets="Worker"
Properties="DefaultMetadata1=%(itemA.Metadata1)"
/>
</Target>
<Target Name="Worker">
<Message Text="itemA.Metadata1 = %(itemA.Metadata1)" />
<Message Text="itemA.Metadata2 = %(itemA.Metadata2)" />
</Target>
</Project>
The usefulness of this approach will depend on what you're trying to accomplish. You can undoubtedly find an alternate solution using properties instead of item metadata.
While the solution above works for the case you describe, it can quickly get out of hand. There's probably a more simple solution that may involve some redundant code.
My recommendation would be to use the simple solution and eliminate as much redundancy as you reasonably can without inventing new ways to get around MSBuild's small feature set. Clever tricks here probably won't save you that many LOC at the end of the day and may result in less readable code, making it more difficult for newcomers to understand what's going on.

how to get extension name (without dot) in MSBuild

I have an ItemGroup, and I use its metadata as identifiers in my MSBuild project for batch processing. For example:
<BuildStep
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Name="RunUnitTestsStep-%(TestSuite.Filename)-%(TestSuite.Extension)"
Message=" - Unit Tests: %(TestSuite.Filename): %(TestSuite.Extension)">
<Output
TaskParameter="Id"
PropertyName="RunUnitTestsStepId-%(TestSuite.Filename)-%(TestSuite.Extension)" />
</BuildStep>
However, this will not work, because there is a dot in the Extension, which is invalid character for an Id (in the BuildStep task). Thus, the MSBuild always fails on the BuildStep task.
I've been trying to remove the dot, but with no luck. Maybe there is a way to add some metadata to en existing ItemGroup? Ideally, I would like to have something like %(TestSuite.ExtensionWithoutDot). How can I achieve that?
I think you are slightly confused about what the <Output> element is doing here - it will create a property named with the value in the PropertyName attribute, and will set the value of that property to be value of the Id output from the BuildStep task. You have no influence on the value of Id - you just store it in a property for later reference in order to set the status of the build step
With that in mind, I can't see why you are concerned that the Property created would have a name that would include the concatenation of the extension. As long as the property name is unique, you can reference it later in a subsequent BuildStep task, and I presume your testsuite filename is enough to indicate uniqueness.
In fact, you could avoid having to create unique properties that track each testsuite/buildstep pair if you did Target batching:
<Target Name="Build"
Inputs="#(TestSuite)"
Outputs="%(Identity).Dummy">
<!--
Note that even though it looks like we have the entire TestSuite itemgroup here,
We will only have ONE - ie we will execute this target *foreach* item in the group
See http://beaucrawford.net/post/MSBuild-Batching.aspx
-->
<BuildStep
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Name="RunUnitTestsStep-%(TestSuite.Filename)-%(TestSuite.Extension)"
Message=" - Unit Tests: %(TestSuite.Filename): %(TestSuite.Extension)">
<Output
TaskParameter="Id"
PropertyName="TestStepId" />
</BuildStep>
<!--
..Do some stuff here..
-->
<BuildStep Condition=" Evaluate Success Condition Here "
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Id="$(TestStepId)"
Status="Succeeded" />
<BuildStep Condition=" Evaluate Failed Condition Here "
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
BuildUri="$(BuildUri)"
Id="$(TestStepId)"
Status="Failed" />
</Target>