When creating MSBuild Items from files using wildcards, such as,
<ItemGroup>
<Scripts>$(ScriptsDirectory)\**\*.sql</Scripts>
</ItemGroup>
the files seem to always be sorted alphabetically by path.
However, is this sort order guaranteed - can I rely on it?
Yes, MSBuild will always sort alphabetically by path, in order to guarantee repeatability. (This is covered in my new book, "MSBuild Trickery")
I needed to sort an ItemGroup of files based on the actual filename, not the full path. So I wrote an MSBuild task to do this, hope it helps...
MSBuild sort itemgroup of files by filename
Some time back I came across an oddity transitioning from Visual Studio 2008 to 2010 with the ordering of an ItemGroup that was creating using a wildcard. File names that started with an underscore character '_' went from being first in the results to last.
At the time I resolved this using the MSBuild Extension Pack. This provides a TaskAction to explicitly sort an ItemGroup.
E.g.
<!-- Sort an ItemGroup alphabetically -->
<MSBuild.ExtensionPack.Framework.MsBuildHelper TaskAction="Sort" InputItems1="#(MainDataFiles)">
<Output TaskParameter="OutputItems" ItemName="sorted"/>
</MSBuild.ExtensionPack.Framework.MsBuildHelper>
<Message Text="Sorted Items: %(sorted.Identity)"/>
Related
I am attempting to link resource files that were organized by locale folders into their own .resources.dll assembly. There are more than 750 locales that are dynamically generated, so it is not practical to hard code them like the docs show.
<ItemGroup>
<ResourceFiles Include="<path_to>/af/af.res;<path_to>/af/feature1.af.res;<path_to>/af/feature2.af.res">
<Culture>af<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-NA/af_NA.res;<path_to>/af-NA/feature1.af_NA.res;<path_to>/af-NA/feature2.af_NA.res">
<Culture>af-NA<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-ZA/af_ZA.res;<path_to>/af-ZA/feature1.af_ZA.res;<path_to>/af-ZA/feature2.af_ZA.res">
<Culture>af-ZA<Culture>
</ResourceFiles>
</ItemGroup>
The above structure can be used to execute the AL task multiple times for each group of files. As you can see, my files are arranged in folders that are named the same as the culture in .NET.
My question is, how do I build this structure dynamically based on the 750+ locale folders, many which contain multiple files?
What I Tried
I was able to get the grouping to function. However, for some odd reason the list of files is being evaluated as a String rather than ITaskItem[] like it should be. This is the structure that does the correct grouping. It is based on this Gist (although I am not sure whether I am misunderstanding how to use the last bit because the example is incomplete).
<PropertyGroup>
<SatelliteAssemblyTargetFramework>netstandard2.0</SatelliteAssemblyTargetFramework>
<TemplateAssemblyFilePath>$(MSBuildProjectDirectory)/bin/$(Configuration)/$(TargetFramework)/$(AssemblyName).dll</TemplateAssemblyFilePath>
<ICU4JResourcesDirectory>$(SolutionDir)_artifacts/icu4j-transformed</ICU4JResourcesDirectory>
<ICU4NSatelliteAssemblyOutputDir>$(SolutionDir)_artifacts/SatelliteAssemblies</ICU4NSatelliteAssemblyOutputDir>
<PropertyGroup>
<Target
Name="GenerateOurSatelliteAssemblies"
DependsOnTargets="ExecICU4JResourceConverter"
AfterTargets="AfterBuild"
Condition=" '$(TargetFramework)' == '$(SatelliteAssemblyTargetFramework)' ">
<ItemGroup>
<EmbeddedResources Include="$(ICU4JResourcesDirectory)/*.*" />
<EmbeddedResourcesPaths Include="$([System.IO.Directory]::GetDirectories('$(ICU4JResourcesDirectory)'))" />
<!-- This groups each locale together along with its nested files and root path -->
<FolderInLocale Include="#(EmbeddedResourcesPaths)">
<Culture>$([System.IO.Path]::GetFileName('%(Identity)'))</Culture>
<Files>$([System.IO.Directory]::GetFiles('%(EmbeddedResourcesPaths.Identity)'))</Files>
</FolderInLocale>
</ItemGroup>
<!-- EmbedResources accepts ITaskItem[], but the result
of this transform is a ; delimited string -->
<AL EmbedResources="#(FolderInLocale->'%(Files)')"
Culture="%(FolderInLocale.Culture)"
TargetType="library"
TemplateFile="$(TemplateAssemblyFilePath)"
KeyFile="$(AssemblyOriginatorKeyFile)"
OutputAssembly="$(ICU4NSatelliteAssemblyOutputDir)/%(FolderInLocale.Culture)/$(AssemblyName).resources.dll" />
</Target>
I have attempted numerous ways to replace the semicolon characters with %3B and to split on semicolon (i.e. #(FolderInLocale->'%(Files.Split(';'))'), but in all cases, the transform fails to evaluate correctly.
I have also consulted the docs for MSBuild well-known item metadata to see if there is another way of grouping by folder. Unfortunately, there is no %(FolderName) metadata, which would solve my issue completely. While I was able to get it to group by folder using the below XML, it immediately flattened when trying to get the name of the top level folder, which is where the name of the culture is.
I am using GetFileName() to get the name of the top level folder after stripping the file name from it. But please do tell if there is a better way.
<ItemGroup>
<EmbeddedResourcesLocalizedFiles Include="$(ICU4JResourcesDirectory)/*/*.*"/>
<EmbeddedResourcesLocalized Include="#(EmbeddedResourcesLocalizedFiles)">
<Culture>%(RootDir)%(Directory)</Culture>
</EmbeddedResourcesLocalized>
<!-- Calling GetFileName() like this rolls the files into a single group,
but prior to this call it is grouped correctly. -->
<EmbeddedResourcesLocalized2 Include="#(EmbeddedResourcesLocalized)">
<Culture>$([System.IO.Path]::GetFileName('%(EmbeddedResourcesLocalized.Culture)'))</Culture>
</EmbeddedResourcesLocalized2>
</ItemGroup>
I have a set of files I need to copy that are underneath a folder that contains the version of the package that deposited those files:
<ItemGroup>
<MyFiles Include="$(MyPackages)\foo.x64*\binaries\*.*"/>
</ItemGroup>
foo.x64* can resolve to more than one folder like foo.x64.17.5.50 and foo.x64.17.6.2 where the suffix represents the version of package. I want to filter my items based on that version but I can't seem to find a way to extract any part of the item file path as metadata on my item to then use it in batching or conditionals.
There are a few features of msbuild that can be used here. those are:
Defining custom item metadata. let's call themFooArch and FooVersion
MSBuild allows calling some .net APIs including regular expressions. So we can use RegEx.Match() here to extract the information you need to know.
When doing numeric comparison on strings, MSBuild parses them into Versions. This will allow you to do range conditions like '%(FooItems.FooVersion)' < '17.6.2'.
What's not possible out of the box is determining the highest version of a list of items. To achieve this, you'll need to write a custom msbuild task.
A more concrete example: To add the required metadata on the items and copy the right version and architecture, you can do the following:
<PropertyGroup>
<FooRegEx>[\\/]foo\.(?<arch>x(\d*))\.(?<version>((\d+)(\.\d+)+))</FooRegEx>
<FooVersionToUse>17.5.50</FooVersionToUse>
</PropertyGroup>
<ItemGroup>
<FooFiles Include="mypkgs\foo*\binaries\**\*">
<FooArch>$([System.Text.RegularExpressions.Regex]::Match(%(Identity), $(FooRegEx)).get_Groups().get_Item("arch"))</FooArch>
<FooVersion>$([System.Text.RegularExpressions.Regex]::Match(%(Identity), $(FooRegEx)).get_Groups().get_Item("version"))</FooVersion>
</FooFiles>
</ItemGroup>
<Target Name="IncludeFoo" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<FooArchToUse Condition="'$(FooArchToUse)' == '' and '$(Platform)' != 'AnyCPU'">$(Platform)</FooArchToUse>
<FooArchToUse Condition="'$(FooArchToUse)' == ''">x64</FooArchToUse>
</PropertyGroup>
<ItemGroup>
<Content Include="#(FooFiles)"
Link="foo\%(Filename)%(Extension)"
CopyToOutputDirectory="PreserveNewest"
Condition="'%(FooFiles.FooVersion)' == '$(FooVersionToUse)' and '%(FooFiles.FooArch)' == '$(FooArchToUse)'"/>
</ItemGroup>
</Target>
This will copy the items of the selected version and platform built for (defaulting to x64) into a foo subfolder of the output directory. (Tested on MSBuild 15.1). I've uploaded the example project to GitHub here.
You can you use the ItemGroup's Condition Attribute to filter out elements based on some boolean expression. You can use regex for instance as per this answer.
I have a bunch of files x.txt in various directories throughout my project. As part of my build step, I would like to collect these and place them in a single folder (without needing to declare each one). I can detect them by using:
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
</ItemGroup>
However, if I copy these to a single folder - they all overwrite each other. I have tried using a transform where I append a GUID to each file name, but the GUID is only created once, and then re-used for each transform (thus they overwrite each over). Is there a way of generating unique names in MSBuild when copying an ItemGroup with identically named files? The end naming scheme is not important, as long as all the files end up in that folder.
The transform works but you have to 'force' it to generate new data on each iteration. It took me a while to figure that out and it makes sense now but I couldn't find any documentation explaining this. But it works simply by referencing other existing metadata: msbuild sees that has to be evaluated on every iteration so it will happily evaluate anything part of the new metadata. Example, simply using %(FileName):
<Target Name="CreateUniqueNames">
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
<MyFiles>
<Dest>%(Filename)$([System.Guid]::NewGuid())%(FileName)</Dest>
</MyFiles>
</ItemGroup>
<Message Text="%(MyFiles.Identity) -> %(MyFiles.Dest)"/>
</Target>
Alternatively you can make use of the unique metadata you already have namely RecursiveDir:
<Target Name="CreateUniqueNames">
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
<MyFiles>
<Dest>x_$([System.String]::Copy('%(RecursiveDir)').Replace('\', '_')).txt</Dest>
</MyFiles>
</ItemGroup>
<Message Text="%(MyFiles.Identity) -> %(MyFiles.Dest)"/>
</Target>
Below is a portion of a MSBuild file that I'm working on:
<ItemGroup>
<Tests Include="$(SolutionDir)\**\bin\$(TestPlatform)\$(Configuration)\*.Tests.dll" />
</ItemGroup>
<PropertyGroup>
<TestProperties>/testcontainer:%(Tests.FullPath)</TestProperties>
</PropertyGroup>
I want to have a property that holds a command line switch. However, when I try to use $(TestProperties) in an Exec Command string, %(Tests.FullPath) is never resolved to the absolute path of the Tests item. Instead, it's always processed literally, as "%(Tests.FullPath)".
Am I doing something wrong or is this a standard MSBuild behavior? If the latter, is there a way for me to workaround this?
Thanks
P.S. - I realize I probably don't need to access the FullPath property since my Include value is an absolute path. However, I'd still like to understand the issue, along with how to handle it.
You have a syntax error. Item lists are referenced via the # character and item meta data is referenced via %. Reference the MSBuild Special Character Reference for details. To access the well known item metadata, you need to apply a transform inside the Property itself.
<ItemGroup>
<Tests Include="MyFile.txt" />
</ItemGroup>
<PropertyGroup>
<TestProperties>/testcontainer:#(Tests->'%(FullPath)')</TestProperties>
</PropertyGroup>
You can find more help here
The following gives me only 1 file, the exe:
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.*" Condition="'%(Extension)'=='.dll'"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
If I remove the Condition attribute, AssembliesToMerge contains all the files in the directory--dlls and otherwise. What am I doing wrong?
I am testing this via the ILMerge MSBuildCommunityExtensions Task. If there is a way to directly print the items in the ItemGroup, then that might help to ensure it's an issue with the Condition attribute.
Just use a wildcard in Include to filter dll files (Items wildcard)
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.dll"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
I think it doesn't work using the Condition attribute because Item metadatas aren't set yet during creation, so %(Extension) is empty.