How does MsBuild match wildcards in Exclude attribute on items? - msbuild

What is the exact behavior of the Exclude attribute with wildcards on items in item collection? In some cases it works as I would expect, and in other cases it fails to exclude apparently matching files.
Suppose I have an src/ subdirectory, containing a few subdirectories on its own. Let's call them src/base and src/util (really there are quite a few, 20+). My intent is to compile all files from selected subdirectories of src/, except files ending in -test.cc. Suppose, without loss of generality, that there exists a file src/base/io-test.cpp.
In the two cases below path (sans filename) on Include and Exclude attributes is same, and the file src/base/io-test.cpp is excluded as expected: The first, as anticipated, includes non-test files only from src/base
<ItemGroup>
<ClCompile Include="src/base/*.cc" Exclude="src/base/*-test.cc" />
</ItemGroup>
and the second all non-test files from all subdirectories of src/:
<ItemGroup>
<ClCompile Include="src/*/*.cc" Exclude="src/*/*-test.cc" />
</ItemGroup>
But the following does not exclude files seemingly matching the Exclude pattern:
<ItemGroup>
<ClCompile Include="src/base/*.cc" Exclude="src/*/*-test.cc" />
</ItemGroup>
I would expect the src/base/io-test.cpp file excluded, as its name clearly matches the pattern (and actually excluded by in in the second case), but the file is included in the collection.
MSDN is somewhat vague on the Exclude behavior. Strict reading might even suggest that wildcards are not supported in it at all, but I do not believe this is the case.
Since my intention is to include files from a few listed directories (akin to Include="src/base/*.cc;src/util/*.cc") but exclude all test files, the last definition is the one that I am trying to get working.

Related

How do I group files by the top folder they are in and have a task act on those files?

I am attempting to link resource files that were organized by locale folders into their own .resources.dll assembly. There are more than 750 locales that are dynamically generated, so it is not practical to hard code them like the docs show.
<ItemGroup>
<ResourceFiles Include="<path_to>/af/af.res;<path_to>/af/feature1.af.res;<path_to>/af/feature2.af.res">
<Culture>af<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-NA/af_NA.res;<path_to>/af-NA/feature1.af_NA.res;<path_to>/af-NA/feature2.af_NA.res">
<Culture>af-NA<Culture>
</ResourceFiles>
<ResourceFiles Include="<path_to>/af-ZA/af_ZA.res;<path_to>/af-ZA/feature1.af_ZA.res;<path_to>/af-ZA/feature2.af_ZA.res">
<Culture>af-ZA<Culture>
</ResourceFiles>
</ItemGroup>
The above structure can be used to execute the AL task multiple times for each group of files. As you can see, my files are arranged in folders that are named the same as the culture in .NET.
My question is, how do I build this structure dynamically based on the 750+ locale folders, many which contain multiple files?
What I Tried
I was able to get the grouping to function. However, for some odd reason the list of files is being evaluated as a String rather than ITaskItem[] like it should be. This is the structure that does the correct grouping. It is based on this Gist (although I am not sure whether I am misunderstanding how to use the last bit because the example is incomplete).
<PropertyGroup>
<SatelliteAssemblyTargetFramework>netstandard2.0</SatelliteAssemblyTargetFramework>
<TemplateAssemblyFilePath>$(MSBuildProjectDirectory)/bin/$(Configuration)/$(TargetFramework)/$(AssemblyName).dll</TemplateAssemblyFilePath>
<ICU4JResourcesDirectory>$(SolutionDir)_artifacts/icu4j-transformed</ICU4JResourcesDirectory>
<ICU4NSatelliteAssemblyOutputDir>$(SolutionDir)_artifacts/SatelliteAssemblies</ICU4NSatelliteAssemblyOutputDir>
<PropertyGroup>
<Target
Name="GenerateOurSatelliteAssemblies"
DependsOnTargets="ExecICU4JResourceConverter"
AfterTargets="AfterBuild"
Condition=" '$(TargetFramework)' == '$(SatelliteAssemblyTargetFramework)' ">
<ItemGroup>
<EmbeddedResources Include="$(ICU4JResourcesDirectory)/*.*" />
<EmbeddedResourcesPaths Include="$([System.IO.Directory]::GetDirectories('$(ICU4JResourcesDirectory)'))" />
<!-- This groups each locale together along with its nested files and root path -->
<FolderInLocale Include="#(EmbeddedResourcesPaths)">
<Culture>$([System.IO.Path]::GetFileName('%(Identity)'))</Culture>
<Files>$([System.IO.Directory]::GetFiles('%(EmbeddedResourcesPaths.Identity)'))</Files>
</FolderInLocale>
</ItemGroup>
<!-- EmbedResources accepts ITaskItem[], but the result
of this transform is a ; delimited string -->
<AL EmbedResources="#(FolderInLocale->'%(Files)')"
Culture="%(FolderInLocale.Culture)"
TargetType="library"
TemplateFile="$(TemplateAssemblyFilePath)"
KeyFile="$(AssemblyOriginatorKeyFile)"
OutputAssembly="$(ICU4NSatelliteAssemblyOutputDir)/%(FolderInLocale.Culture)/$(AssemblyName).resources.dll" />
</Target>
I have attempted numerous ways to replace the semicolon characters with %3B and to split on semicolon (i.e. #(FolderInLocale->'%(Files.Split(';'))'), but in all cases, the transform fails to evaluate correctly.
I have also consulted the docs for MSBuild well-known item metadata to see if there is another way of grouping by folder. Unfortunately, there is no %(FolderName) metadata, which would solve my issue completely. While I was able to get it to group by folder using the below XML, it immediately flattened when trying to get the name of the top level folder, which is where the name of the culture is.
I am using GetFileName() to get the name of the top level folder after stripping the file name from it. But please do tell if there is a better way.
<ItemGroup>
<EmbeddedResourcesLocalizedFiles Include="$(ICU4JResourcesDirectory)/*/*.*"/>
<EmbeddedResourcesLocalized Include="#(EmbeddedResourcesLocalizedFiles)">
<Culture>%(RootDir)%(Directory)</Culture>
</EmbeddedResourcesLocalized>
<!-- Calling GetFileName() like this rolls the files into a single group,
but prior to this call it is grouped correctly. -->
<EmbeddedResourcesLocalized2 Include="#(EmbeddedResourcesLocalized)">
<Culture>$([System.IO.Path]::GetFileName('%(EmbeddedResourcesLocalized.Culture)'))</Culture>
</EmbeddedResourcesLocalized2>
</ItemGroup>

MSBuild task to Copy contents of a specific folder by using a wild card in the path

My Folder structure is as follows
Module
-->Asia - 1.0.0.12
------>Deployment
------>Install.exe
------>version.xml
-->Africa - 1.0.3.4
------>Deployment
------>Install.exe
------>version.xml
-->Europe - 2.0.1.2
------>Deployment
------>Install.exe
------>version.xml
I want to copy the 'deployment' folder (and subfolders) under each region to my output directory. The region numbers will change so i cannot hardcode them in my Include statement. The command i am using to copy for Asia region is
<ItemGroup>
<GetFiles Include="$(MSBuildProjectDirectory)\Module\Asia*\Deployment\**\*">
<Destination>D:\Region\Asia</Destination>
</GetFiles >
</ItemGroup>
<Copy
SourceFiles="%(GetFiles.Identity)"
DestinationFolder="%(GetFiles.Destination)\%(RecursiveDir)"
/>
Instead of the Deployment folder and its subdirectories getting copied under the Destination, i am getting the folder structure as
D:\Region\Asia\Asia - 1.0.0.12\Deployment
what i want is
D:\Region\Asia\Deployment
Can this be achieved? Thanks
MSBuild task to Copy contents of a specific folder by using a wild
card in the path
If you use wildcard in msbuild and do a copy task, the MSBuild will always copy the path from the first address where the wildcard is used.
For your situation, since you just use wildcard under Asia*, so it will keep Asia - 1.0.0.12\Deployment folder structure under D:\Region\Asia.
As a suggestion, to get what you want, you need to clearly indicate the name of the Asia folder, even if it is much more complicated, you need to specify one by one.
Use like this:
<Target Name="xxx" BeforeTargets="Build">
<ItemGroup>
<GetFiles Include="$(MSBuildProjectDirectory)\Module\Asia - 1.0.0.12\Deployment\**\*">
<Destination>D:\Region\Asia</Destination>
</GetFiles>
</ItemGroup>
<Copy SourceFiles="%(GetFiles.Identity)"
DestinationFolder="%(GetFiles.Destination)\Deployment\%(RecursiveDir)"/>
</Target>

How to use MSBuild transform when ItemGroup files all have identical names?

I have a bunch of files x.txt in various directories throughout my project. As part of my build step, I would like to collect these and place them in a single folder (without needing to declare each one). I can detect them by using:
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
</ItemGroup>
However, if I copy these to a single folder - they all overwrite each other. I have tried using a transform where I append a GUID to each file name, but the GUID is only created once, and then re-used for each transform (thus they overwrite each over). Is there a way of generating unique names in MSBuild when copying an ItemGroup with identically named files? The end naming scheme is not important, as long as all the files end up in that folder.
The transform works but you have to 'force' it to generate new data on each iteration. It took me a while to figure that out and it makes sense now but I couldn't find any documentation explaining this. But it works simply by referencing other existing metadata: msbuild sees that has to be evaluated on every iteration so it will happily evaluate anything part of the new metadata. Example, simply using %(FileName):
<Target Name="CreateUniqueNames">
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
<MyFiles>
<Dest>%(Filename)$([System.Guid]::NewGuid())%(FileName)</Dest>
</MyFiles>
</ItemGroup>
<Message Text="%(MyFiles.Identity) -> %(MyFiles.Dest)"/>
</Target>
Alternatively you can make use of the unique metadata you already have namely RecursiveDir:
<Target Name="CreateUniqueNames">
<ItemGroup>
<MyFiles Include="$(SRCROOT)\**\x.txt"/>
<MyFiles>
<Dest>x_$([System.String]::Copy('%(RecursiveDir)').Replace('\', '_')).txt</Dest>
</MyFiles>
</ItemGroup>
<Message Text="%(MyFiles.Identity) -> %(MyFiles.Dest)"/>
</Target>

MSBuild exclude syntax not working

I have a test file in MSBuild to create a ZIP. I need exclude certain folders. I have the following working.
<PropertyGroup>
<TestZipPath>C:\path\to\my\folder\</TestZipPath>
<ExcludeList>$(TestZipPath)\**\_svn\**;$(TestZipPath)\**\.svn\**;$(TestZipPath)\**\obj\**;$(TestZipPath)\**\*.config</ExcludeList>
</PropertyGroup>
<ItemGroup>
<ZipFiles Include="$(TestZipPath)\**\*.*" Exclude="$(ExcludeList)" />
</ItemGroup>
<Message Text="%(ZipFiles.FullPath)"/>
That seems hideously verbose to me. Ideally I would want the ExcludeList to be formatted like this:
<ExcludeList>**\_svn\**;**\.svn\**;**\obj\**;**\*.config</ExcludeList>
But it doesn't seem to work. Why do I need to include $(TestZipPath) before every exclude pattern? Is ** not intended to be used at the beginning of a path? Is there a better way to do this?
I figured out the problem. The issue is that I am trying to include files that are not relative to the msbuild file that I'm executing. MSBuild assumes that file paths are relative to this location and gives you no way to change that. Because of this, all of my paths have to be absolute and can't be relative.
Try to add '.\' before every include pattern. Like this:
'.\**\obj\**'

Get list of dlls in a directory with MSBuild

The following gives me only 1 file, the exe:
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.*" Condition="'%(Extension)'=='.dll'"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
If I remove the Condition attribute, AssembliesToMerge contains all the files in the directory--dlls and otherwise. What am I doing wrong?
I am testing this via the ILMerge MSBuildCommunityExtensions Task. If there is a way to directly print the items in the ItemGroup, then that might help to ensure it's an issue with the Condition attribute.
Just use a wildcard in Include to filter dll files (Items wildcard)
<ItemGroup>
<AssembliesToMerge Include="$(MSBuildProjectDirectory)\App\bin\Release\*.dll"/>
<AssembliesTomerge Include="$(MSbuildProjectDirectory)\App\bin\Release\App.exe"/>
</ItemGroup>
I think it doesn't work using the Condition attribute because Item metadatas aren't set yet during creation, so %(Extension) is empty.