In MSBuild, why isn't Item Metadata, within a property, being resolved? - msbuild

Below is a portion of a MSBuild file that I'm working on:
<ItemGroup>
<Tests Include="$(SolutionDir)\**\bin\$(TestPlatform)\$(Configuration)\*.Tests.dll" />
</ItemGroup>
<PropertyGroup>
<TestProperties>/testcontainer:%(Tests.FullPath)</TestProperties>
</PropertyGroup>
I want to have a property that holds a command line switch. However, when I try to use $(TestProperties) in an Exec Command string, %(Tests.FullPath) is never resolved to the absolute path of the Tests item. Instead, it's always processed literally, as "%(Tests.FullPath)".
Am I doing something wrong or is this a standard MSBuild behavior? If the latter, is there a way for me to workaround this?
Thanks
P.S. - I realize I probably don't need to access the FullPath property since my Include value is an absolute path. However, I'd still like to understand the issue, along with how to handle it.

You have a syntax error. Item lists are referenced via the # character and item meta data is referenced via %. Reference the MSBuild Special Character Reference for details. To access the well known item metadata, you need to apply a transform inside the Property itself.
<ItemGroup>
<Tests Include="MyFile.txt" />
</ItemGroup>
<PropertyGroup>
<TestProperties>/testcontainer:#(Tests->'%(FullPath)')</TestProperties>
</PropertyGroup>
You can find more help here

Related

How do I group files by the top folder they are in and have a task act on those files?

I am attempting to link resource files that were organized by locale folders into their own .resources.dll assembly. There are more than 750 locales that are dynamically generated, so it is not practical to hard code them like the docs show.
<ItemGroup>
<ResourceFiles Include="<path_to>/af/af.res;<path_to>/af/feature1.af.res;<path_to>/af/feature2.af.res">
<Culture>af<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-NA/af_NA.res;<path_to>/af-NA/feature1.af_NA.res;<path_to>/af-NA/feature2.af_NA.res">
<Culture>af-NA<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-ZA/af_ZA.res;<path_to>/af-ZA/feature1.af_ZA.res;<path_to>/af-ZA/feature2.af_ZA.res">
<Culture>af-ZA<Culture>
</ResourceFiles>
</ItemGroup>
The above structure can be used to execute the AL task multiple times for each group of files. As you can see, my files are arranged in folders that are named the same as the culture in .NET.
My question is, how do I build this structure dynamically based on the 750+ locale folders, many which contain multiple files?
What I Tried
I was able to get the grouping to function. However, for some odd reason the list of files is being evaluated as a String rather than ITaskItem[] like it should be. This is the structure that does the correct grouping. It is based on this Gist (although I am not sure whether I am misunderstanding how to use the last bit because the example is incomplete).
<PropertyGroup>
<SatelliteAssemblyTargetFramework>netstandard2.0</SatelliteAssemblyTargetFramework>
<TemplateAssemblyFilePath>$(MSBuildProjectDirectory)/bin/$(Configuration)/$(TargetFramework)/$(AssemblyName).dll</TemplateAssemblyFilePath>
<ICU4JResourcesDirectory>$(SolutionDir)_artifacts/icu4j-transformed</ICU4JResourcesDirectory>
<ICU4NSatelliteAssemblyOutputDir>$(SolutionDir)_artifacts/SatelliteAssemblies</ICU4NSatelliteAssemblyOutputDir>
<PropertyGroup>
<Target
Name="GenerateOurSatelliteAssemblies"
DependsOnTargets="ExecICU4JResourceConverter"
AfterTargets="AfterBuild"
Condition=" '$(TargetFramework)' == '$(SatelliteAssemblyTargetFramework)' ">
<ItemGroup>
<EmbeddedResources Include="$(ICU4JResourcesDirectory)/*.*" />
<EmbeddedResourcesPaths Include="$([System.IO.Directory]::GetDirectories('$(ICU4JResourcesDirectory)'))" />
<!-- This groups each locale together along with its nested files and root path -->
<FolderInLocale Include="#(EmbeddedResourcesPaths)">
<Culture>$([System.IO.Path]::GetFileName('%(Identity)'))</Culture>
<Files>$([System.IO.Directory]::GetFiles('%(EmbeddedResourcesPaths.Identity)'))</Files>
</FolderInLocale>
</ItemGroup>
<!-- EmbedResources accepts ITaskItem[], but the result
of this transform is a ; delimited string -->
<AL EmbedResources="#(FolderInLocale->'%(Files)')"
Culture="%(FolderInLocale.Culture)"
TargetType="library"
TemplateFile="$(TemplateAssemblyFilePath)"
KeyFile="$(AssemblyOriginatorKeyFile)"
OutputAssembly="$(ICU4NSatelliteAssemblyOutputDir)/%(FolderInLocale.Culture)/$(AssemblyName).resources.dll" />
</Target>
I have attempted numerous ways to replace the semicolon characters with %3B and to split on semicolon (i.e. #(FolderInLocale->'%(Files.Split(';'))'), but in all cases, the transform fails to evaluate correctly.
I have also consulted the docs for MSBuild well-known item metadata to see if there is another way of grouping by folder. Unfortunately, there is no %(FolderName) metadata, which would solve my issue completely. While I was able to get it to group by folder using the below XML, it immediately flattened when trying to get the name of the top level folder, which is where the name of the culture is.
I am using GetFileName() to get the name of the top level folder after stripping the file name from it. But please do tell if there is a better way.
<ItemGroup>
<EmbeddedResourcesLocalizedFiles Include="$(ICU4JResourcesDirectory)/*/*.*"/>
<EmbeddedResourcesLocalized Include="#(EmbeddedResourcesLocalizedFiles)">
<Culture>%(RootDir)%(Directory)</Culture>
</EmbeddedResourcesLocalized>
<!-- Calling GetFileName() like this rolls the files into a single group,
but prior to this call it is grouped correctly. -->
<EmbeddedResourcesLocalized2 Include="#(EmbeddedResourcesLocalized)">
<Culture>$([System.IO.Path]::GetFileName('%(EmbeddedResourcesLocalized.Culture)'))</Culture>
</EmbeddedResourcesLocalized2>
</ItemGroup>

MSBuild: Filtering Items based on a part of the file path

I have a set of files I need to copy that are underneath a folder that contains the version of the package that deposited those files:
<ItemGroup>
<MyFiles Include="$(MyPackages)\foo.x64*\binaries\*.*"/>
</ItemGroup>
foo.x64* can resolve to more than one folder like foo.x64.17.5.50 and foo.x64.17.6.2 where the suffix represents the version of package. I want to filter my items based on that version but I can't seem to find a way to extract any part of the item file path as metadata on my item to then use it in batching or conditionals.
There are a few features of msbuild that can be used here. those are:
Defining custom item metadata. let's call themFooArch and FooVersion
MSBuild allows calling some .net APIs including regular expressions. So we can use RegEx.Match() here to extract the information you need to know.
When doing numeric comparison on strings, MSBuild parses them into Versions. This will allow you to do range conditions like '%(FooItems.FooVersion)' < '17.6.2'.
What's not possible out of the box is determining the highest version of a list of items. To achieve this, you'll need to write a custom msbuild task.
A more concrete example: To add the required metadata on the items and copy the right version and architecture, you can do the following:
<PropertyGroup>
<FooRegEx>[\\/]foo\.(?<arch>x(\d*))\.(?<version>((\d+)(\.\d+)+))</FooRegEx>
<FooVersionToUse>17.5.50</FooVersionToUse>
</PropertyGroup>
<ItemGroup>
<FooFiles Include="mypkgs\foo*\binaries\**\*">
<FooArch>$([System.Text.RegularExpressions.Regex]::Match(%(Identity), $(FooRegEx)).get_Groups().get_Item("arch"))</FooArch>
<FooVersion>$([System.Text.RegularExpressions.Regex]::Match(%(Identity), $(FooRegEx)).get_Groups().get_Item("version"))</FooVersion>
</FooFiles>
</ItemGroup>
<Target Name="IncludeFoo" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<FooArchToUse Condition="'$(FooArchToUse)' == '' and '$(Platform)' != 'AnyCPU'">$(Platform)</FooArchToUse>
<FooArchToUse Condition="'$(FooArchToUse)' == ''">x64</FooArchToUse>
</PropertyGroup>
<ItemGroup>
<Content Include="#(FooFiles)"
Link="foo\%(Filename)%(Extension)"
CopyToOutputDirectory="PreserveNewest"
Condition="'%(FooFiles.FooVersion)' == '$(FooVersionToUse)' and '%(FooFiles.FooArch)' == '$(FooArchToUse)'"/>
</ItemGroup>
</Target>
This will copy the items of the selected version and platform built for (defaulting to x64) into a foo subfolder of the output directory. (Tested on MSBuild 15.1). I've uploaded the example project to GitHub here.
You can you use the ItemGroup's Condition Attribute to filter out elements based on some boolean expression. You can use regex for instance as per this answer.

MSBuild exclude syntax not working

I have a test file in MSBuild to create a ZIP. I need exclude certain folders. I have the following working.
<PropertyGroup>
<TestZipPath>C:\path\to\my\folder\</TestZipPath>
<ExcludeList>$(TestZipPath)\**\_svn\**;$(TestZipPath)\**\.svn\**;$(TestZipPath)\**\obj\**;$(TestZipPath)\**\*.config</ExcludeList>
</PropertyGroup>
<ItemGroup>
<ZipFiles Include="$(TestZipPath)\**\*.*" Exclude="$(ExcludeList)" />
</ItemGroup>
<Message Text="%(ZipFiles.FullPath)"/>
That seems hideously verbose to me. Ideally I would want the ExcludeList to be formatted like this:
<ExcludeList>**\_svn\**;**\.svn\**;**\obj\**;**\*.config</ExcludeList>
But it doesn't seem to work. Why do I need to include $(TestZipPath) before every exclude pattern? Is ** not intended to be used at the beginning of a path? Is there a better way to do this?
I figured out the problem. The issue is that I am trying to include files that are not relative to the msbuild file that I'm executing. MSBuild assumes that file paths are relative to this location and gives you no way to change that. Because of this, all of my paths have to be absolute and can't be relative.
Try to add '.\' before every include pattern. Like this:
'.\**\obj\**'

Get list of dlls in a directory with MSBuild

The following gives me only 1 file, the exe:
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.*" Condition="'%(Extension)'=='.dll'"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
If I remove the Condition attribute, AssembliesToMerge contains all the files in the directory--dlls and otherwise. What am I doing wrong?
I am testing this via the ILMerge MSBuildCommunityExtensions Task. If there is a way to directly print the items in the ItemGroup, then that might help to ensure it's an issue with the Condition attribute.
Just use a wildcard in Include to filter dll files (Items wildcard)
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.dll"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
I think it doesn't work using the Condition attribute because Item metadatas aren't set yet during creation, so %(Extension) is empty.

Error using MSBuild tokens in PropertyGroup

I am trying to setup some properties that I use multiple times in my MSBuild script. I have the following property section:
<PropertyGroup>
<BuildDependsOn>$(BuildDependsOn); MyAfterBuild </BuildDependsOn>
<SubstitutionsFilePath>$(ProjectDir)app.config.substitutions.xml </SubstitutionsFilePath>
<AppConfig>$(TargetPath).config</AppConfig>
<HostConfig>$(TargetDir)$(TargetName).vshost.exe.config</HostConfig>
</PropertyGroup>
When I run this I get the following error:
The expression "#(TargetPath).config" cannot be used in this context. Item lists cannot be concatenated with other strings where an item list is expected. Use a semicolon to separate multiple item lists.
I don't understand this error, as the use of the $(BuildDependsOn) and $(ProjectDir) work fine. And I know the $(TargetXXX) values generate properly as when I put them directly into the Tasks section below, they work fine.
The reason for this problem is that TargetDir is defined as an item list, not a property; presumably to cater to the scenario where your outputs are distributed amongst several output directories?
I came up against this same problem and managed to work around it by using the $(OutDir) property instead of $(TargetDir).
(The OutDir property is defined in Microsoft.Common.Targets (lines 100-102) as a normalised version of the OutputPath defined in your project file.)
First try running your build with the /v:diag option, which will output a lot more information and give you a clue as to what part of the build is failing.
A clue might be in the Microsoft.Common.targets file (located in %SystemRoot%\Microsoft.NET\Framework\v2.0.50727) in the PrepareForBuild target:
<!--
These CreateProperty calls are required because TargetDir and TargetPath are defined
to contain an item list. We want that item list to be expanded so that it can be used
as a regular property value and not as an item-list-with-transform.
-->
<CreateProperty Value="$(TargetDir)">
<Output TaskParameter="Value" PropertyName="TargetDir" />
</CreateProperty>
<CreateProperty Value="$(TargetPath)">
<Output TaskParameter="Value" PropertyName="TargetPath" />
</CreateProperty>
To me this looks like a bug, you can report it at https://connect.microsoft.com/feedback/Search.aspx?SiteID=210.