MSBuild: Copy multiple directories in a generic way - msbuild

Question

I have a Visual Studio solution with multiple web projects (e.g. WebappA, WebappB, WebappC). When TFS builds the solution it puts the build results in a _PublishedWebsites folder. The folder structure may look like this:
$(OutDir)
|
+-- _PublishedWebsites
|
+-- WebappA
|
+-- WebappA_Package
|
+-- WebappB
|
+-- WebappB_Package
|
+-- WebappC
|
+-- WebappC_Package
I want to build a deployment package for our operations department in terms of a zip file. Therefore I let TFS run an MSBuild script which copies the _Package folders into a custom directory structure which is zipped in a subsequent step.
$(PackageDirectory)
|
+-- Web
|
+-- WebappA
|
+-- WebappB
|
+-- WebappB
I was able to create a bunch of MSBuild targets which do the copy operations. But I'm unhappy with my solution. I am referencing each webapp in an explicit way that's why I ended up with much repetitive code. To make matters worse each time a new webapp is added I have to extent the build script.
<Target Name="Pack" DependsOnTargets="Pack-WebappA;Pack-WebappB;Pack-WebappC" />
<Target Name="Pack-WebappA">
<ItemGroup>
<WebAppFile Include="$(OutDir)_PublishedWebsites\WebappA_Package\*.*" />
</ItemGroup>
<Copy SourceFiles="#(WebAppFile)" DestinationFolder="$(PackageDirectory)Web\WebappA\" />
</Target>
<Target Name="Pack-WebappB">
<ItemGroup>
<WebAppFile Include="$(OutDir)_PublishedWebsites\WebappB_Package\*.*" />
</ItemGroup>
<Copy SourceFiles="#(WebAppFile)" DestinationFolder="$(PackageDirectory)Web\WebappB\" />
</Target>
<Target Name="Pack-WebappC">
<ItemGroup>
<WebAppFile Include="$(OutDir)_PublishedWebsites\WebappC_Package\*.*" />
</ItemGroup>
<Copy SourceFiles="#(WebAppFile)" DestinationFolder="$(PackageDirectory)Web\WebappC\" />
</Target>
I'm searching for a solution which does the whole thing in a generic way without to referencing the concrete webapps. In essence all what MSBuild should do is to look into the _PublishedWebsites folder and copy each subfolder with a _Package suffix to another folder and remove the suffix. This sounds pretty easy but I was not able to come up with a working solution. I've tried it with batching without success.

Answer 1

You're pretty much right in that you're unhappy with the current solution: automating such things is a must. I agree MsBuild doesn't always make it straightforward though; batching is the right way but you have to add some filtering/manipulating of the items. Using property functions this isn't all that hard though:
In essence all what MSBuild should do is to look into the
_PublishedWebsites folder and copy each subfolder with a _Package suffix to another folder and remove the suffix.
We'll translate this to:
list all directories in _PublishedWebsites
filter the list and include only those ending in _Package
list all files in those directories and set destination for them to a subdirectory of PackageDirectory with the suffix removed
copy each file to corresponding directory
Step 3 is actually two things (list+specify dir) because that is the typical msbuild way of doing things. There are other ways to do this, but this one seems appropriate here.
<Target Name="BatchIt">
<PropertyGroup>
<SourceDir>$(OutDir)_PublishedWebsites\</SourceDir>
<DestDir>$(PackageDirectory)Web\</DestDir>
<PackageString>_Package</PackageString>
</PropertyGroup>
<ItemGroup>
<!-- step 1 -->
<Dirs Include="$([System.IO.Directory]::GetDirectories( `$(SourceDir)`, `*`, System.IO.SearchOption.AllDirectories ) )"/>
<!-- step 2 -->
<PackageDirs Include="%(Dirs.Identity)" Condition="$([System.Text.RegularExpressions.Regex]::IsMatch( %(Filename), '.*$(PackageString)' ) )"/>
<!-- step 3 -->
<PackageFiles Include="%(PackageDirs.Identity)\*.*">
<DestDir>$(PackageDirectory)$([System.IO.Path]::GetFilename( %(PackageDirs.Identity) ).Replace( $(PackageString), '' ) )</DestDir>
</PackageFiles>
</ItemGroup>
<!-- step 4 -->
<Copy SourceFiles="%(PackageFiles.Identity)" DestinationFolder="%(PackageFiles.DestDir)" />
</Target>
edit A more performant and maybe more logical way is to specify the DestDir directly when building the PackageDir list:
<ItemGroup>
<Dirs Include="$([System.IO.Directory]::GetDirectories( `$(SourceDir)`, `*`, System.IO.SearchOption.AllDirectories))"/>
<PackageDirs Include="%(Dirs.Identity)" Condition="$([System.Text.RegularExpressions.Regex]::IsMatch( %(Filename), '.*$(PackageString)' ))">
<DestDir>$(DestDir)$([System.IO.Path]::GetFilename( %(Dirs.Identity) ).Replace( $(PackageString), '' ))</DestDir>
</PackageDirs>
<PackageFiles Include="%(PackageDirs.Identity)\*.*">
<DestDir>%(PackageDirs.DestDir)</DestDir>
</PackageFiles>
</ItemGroup>

Answer 2

stijn came up with a excellent answer. It worked for me almost. I changed only one or two things. This is the code which does the trick, at least in my case.
<Target Name="PackWeb">
<PropertyGroup>
<SourceDir>$(OutDir.TrimEnd('\'))\_PublishedWebsites\</SourceDir>
<PackDir>$(PackageDirectory.TrimEnd('\'))\Web\</PackDir>
<PackageString>_Package</PackageString>
</PropertyGroup>
<ItemGroup>
<Dirs Include="$([System.IO.Directory]::GetDirectories( `$(SourceDir)`, `*` ) )"/>
<PackageDirs Include="%(Dirs.Identity)" Condition="$([System.Text.RegularExpressions.Regex]::IsMatch( %(FullPath), '.*$(PackageString)' ) )"/>
<PackageFiles Include="%(PackageDirs.Identity)\*.*">
<DestDir>$(PackDir)$([System.IO.Path]::GetFilename( %(PackageDirs.Identity) ).Replace( $(PackageString), '' ) )</DestDir>
</PackageFiles>
</ItemGroup>
<Copy SourceFiles="%(PackageFiles.Identity)" DestinationFolder="%(PackageFiles.DestDir)" />
</Target>