How can i get MSBuild target batching to work - msbuild

I'm trying to create a build script for our code deployment to multiple environments. The code is as follows:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<PropertyGroup>
<TargetEnv>Production</TargetEnv>
</PropertyGroup>
<ItemGroup Condition="'$(TargetEnv)' == 'Integration'">
<Server Include="int1">
<ip>172.0.0.1</ip>
</Server>
</ItemGroup>
<ItemGroup Condition="'$(TargetEnv)' == 'Production'">
<Server Include="prod1">
<ip>172.0.2.1</ip>
</Server>
<Server Include="prod2">
<ip>172.0.2.2</ip>
</Server>
</ItemGroup>
<Target Name="Deploy">
<CallTarget Targets="DeployIntegration" />
<CallTarget Targets="DeployServers" />
</Target>
<Target Name="DeployIntegration" Condition="'$(TargetEnv)' == 'Integration'" Outputs="%(Server.Identity)">
<Message Text="= specific int server thing need access to variable %(Server.Identity) =" Importance="high" />
</Target>
<Target Name="DeployServers" Condition="'$(TargetEnv)' != 'Integration'" Outputs="%(Server.Identity)">
<Message Text="= specific prod thing here need access to variable %(Server.Identity) =" Importance="high" />
</Target>
<Target Name="RemoveServerFromLoadBalancer" AfterTargets="DeployServers" Condition="'$(TargetEnv)' != 'Integration'">
<Message Text="= removing %(Server.Identity) from load balancer =" Importance="high" />
</Target>
<Target Name="IgnoreRemoveServerFromLoadBalancer" AfterTargets="DeployServers" Condition="'$(TargetEnv)' == 'Integration'">
<Message Text="= ignore removing %(Server.Identity) from load balancer =" Importance="high" />
</Target>
<Target Name="CopyFilesAndCreateFolderLinks" AfterTargets="RemoveServerFromLoadBalancer;IgnoreRemoveServerFromLoadBalancer">
<Message Text=" = creating and copying files %(Server.Identity) =" Importance="high" />
</Target>
<Target Name="SetWebFarmServerName" AfterTargets="UpdateWebConfig" Condition="'$(TargetEnv)' != 'Integration'">
<Message Text=" = app setting CMSWebFarmServerName set to %(Server.Identity) =" Importance="high" />
</Target>
<Target Name="DisableWebFarmForIntegration" AfterTargets="UpdateWebConfig" Condition="'$(TargetEnv)' == 'Integration'">
<Message Text=" = Disabled webfarm setting for Integration - %(Server.Identity) =" Importance="high" />
</Target>
<Target Name="AddBackToLoadBalancer" AfterTargets="DisableWebFarmForIntegration" Condition="'$(TargetEnv)' != 'Integration'">
<Message Text=" = Putting server %(Server.Identity) back on load balancer =" Importance="high" />
</Target>
</Project>
This code is in an xml(saved in the 11.0 folder) file and i run it using the msbuild command:
C:\Program Files (x86)\Microsoft Visual Studio 11.0>msbuild buildtest.xml /t:Deploy
This code returns this when i run the build task for production:
DeployServers:
= specific prod thing here need access to variable prod1 =
DeployServers:
= specific prod thing here need access to variable prod2 =
RemoveServerFromLoadBalancer:
= removing prod1 from load balancer =
= removing prod2 from load balancer =
CopyFilesAndCreateFolderLinks:
= creating and copying files prod1 =
= creating and copying files prod2 =
I basically want to make sure that, if i'm on integration, it doesn't run specific targets such as load-balancer related tasks as there is only one machine for it. I'm thinking, the returned value should look like this:
DeployServers:
= specific prod thing here need access to variable prod1 =
RemoveServerFromLoadBalancer:
= removing prod1 from load balancer =
CopyFilesAndCreateFolderLinks:
= creating and copying files prod1 =
DeployServers:
= specific prod thing here need access to variable prod2 =
RemoveServerFromLoadBalancer:
= removing prod2 from load balancer =
CopyFilesAndCreateFolderLinks:
= creating and copying files prod2 =
Sorry for the long post, this msbuild stuff is a little tricky. I appreciate your input.

The batching MSBuild performs here is correct, think about it: you ask it to Message for text %(Server.Identity) so it's going to do that for as many servers as it knows and there's no reason it's gonna wait for oter targets in between. So to get what you want you'll have to make it perform all required tasks once per server. Furthermore your general structure is a bit too complicated. The conditions on the targets are unmanageable: just the fact that you repeat the same condition x times is already a sign something's wrong as it, simply said, violates the DIY principle. Also what if you add another TargetEnv? And then another one? Yep you figured it: won't be nice :] Second possible future pitfall is the use of AfterTargets: it's nice when you only have a couple, but after a while you keep adding targets and you'll have no idea what the order is, you'll basically have to go through the entire file to grasp what's going on. Also what if you add more targets that are common for each TargetEnv? Or if you add another TargetEnv. Again won't be nice as you'll have to fix that in multiple places.
Now because you sort of mixed these two complications and threw batching on top of it, things got pretty unclear. Back to the start and think about what you really need: if TargetEnv is A you want to do X and Y and Z, if TargetEnv is B you want to do and Q and Z. That's it. You can consider that as two seperate responsabilities: selecting something based on a condition, and maintaining lists of actions per condition. So let's express this in an msbuild way.
Here's the condition part, now in the new Deploy target. The rest of the targets are moved to another file. Deploy will call a target (which one depends on the condition) in the other msbuild file called deploy.targets, in the same directory as the current file. Because the batching is now on a higher level, it will automatically be executed the way you want: once per server. Note the selected server is passed as a property to the other file. There are other ways to do this, but just like with code it's nice to have a couple of smaller files instead of one big do-it-all one.
<Target Name="Deploy">
<PropertyGroup>
<TargetsFile>$(MsBuildThisFileDirectory)deploy.targets</TargetsFile>
<TargetToCall Condition="$(TargetEnv)=='Production'">DeployServers</TargetToCall>
<TargetToCall Condition="$(TargetEnv)=='Integration'">DeployIntegration</TargetToCall>
</PropertyGroup>
<MSBuild Projects="$(TargetsFile)" Targets="$(TargetToCall)" Properties="Server=%(Server.Identity)" />
</Target>
And here's the new file which has all the targets, and two 'master' targets that now specify exactly which other targets they want to have called, no more need for conditions, no more AfterTargets.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<PropertyGroup>
<CommonTargets>CopyFilesAndCreateFolderLinks</CommonTargets>
</PropertyGroup>
<Target Name="DeployIntegration">
<Message Text="= specific int server thing need access to variable $(Server) =" Importance="high" />
<CallTarget Targets="IgnoreRemoveServerFromLoadBalancer;$(CommonTargets)"/>
</Target>
<Target Name="DeployServers">
<Message Text="= specific prod thing here need access to variable $(Server) =" Importance="high" />
<CallTarget Targets="RemoveServerFromLoadBalancer;AnotherTargetJustForDeploy;$(CommonTargets)"/>
</Target>
<Target Name="RemoveServerFromLoadBalancer">
<Message Text="= removing $(Server) from load balancer =" Importance="high" />
</Target>
<Target Name="AnotherTargetJustForDeploy">
<Message Text="= AnotherTargetJustForDeploy for $(Server) =" Importance="high" />
</Target>
<Target Name="IgnoreRemoveServerFromLoadBalancer">
<Message Text="= ignore removing $(Server) from load balancer =" Importance="high" />
</Target>
<Target Name="CopyFilesAndCreateFolderLinks">
<Message Text=" = creating and copying files $(Server) =" Importance="high" />
</Target>
</Project>

Related

Target with AfterTargets="Publish" executes in unpublishable project

I have a project in my solution that I wanna publish separately from the rest of the solution. So the way to skip it is by setting the IsPublishable property to false, which works like a charm. It seems though that no matter the publishable status of the project, targets set to run after the publish target (AfterTargets="Publish") are still executed when I try to publish the entire solution.
Is this intended? Is there any way to prevent this? I am using VS 2022 preview.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net5.0</TargetFramework>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<LangVersion>9.0</LangVersion>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<BaseOutputPath>..\Build</BaseOutputPath>
<IsPublishable>false</IsPublishable>
</PropertyGroup>
<ItemGroup>
<SomeFiles Include="$(SolutionDir)SomeFiles\**\*.txt" />
</ItemGroup>
<Target Name="CopyCustomContentBuild" AfterTargets="AfterBuild">
<Copy SourceFiles="#(SomeFiles)" DestinationFolder="$(TargetDir)SomeFiles" />
<Message Text="Files copied successfully." Importance="high" />
</Target>
<Target Name="CopyCustomContentPublish" AfterTargets="Publish">
<Copy SourceFiles="#(SomeFiles)" DestinationFolder="$(PublishDir)SomeFiles" />
<Message Text="Files copied successfully to publish dir." Importance="high" />
</Target>
</Project>
That is the intended behaviour. When you set IsPublishable to false MsBuild still logs when a Publish target is supposed to run and continues onto your AfterTargets="Publish" target.
You'll have to set a condition on your actions inside the target to make sure they do not get executed when IsPublishable is false.
<Target Name="CopyCustomContentPublish" AfterTargets="Publish">
<Copy SourceFiles="#(SomeFiles)" DestinationFolder="$(PublishDir)SomeFiles" Condition=" '$(IsPublishable)' == 'true' " />
<Message Text="Files copied successfully to publish dir." Importance="high" Condition=" '$(IsPublishable)' == 'true' " />
</Target>

MSBuild - can I compile all solutions in child directories?

Is there a way in MSBuild to compile all solutions in folders, and sub-folders, and sub... under a specified parent?
We have a bunch of sample programs we ship with our library. I want to add to the build process that we know they all compile.
You can create own targets for restore and build operations. For example:
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" InitialTargets="EntryRestore" DefaultTargets="EntryMSBuild">
<ItemGroup>
<SolutionFile Include="./**/*.sln"/>
</ItemGroup>
<Target Name="EntryRestore">
<Message Text="-----Entry-----" Importance="high"/>
<Message Text=" Restore " Importance="high"/>
<Message Text="-----Entry-----" Importance="high"/>
<MSBuild Projects="%(SolutionFile.Identity)" Targets="restore"/>
</Target>
<Target Name="EntryMSBuild">
<Message Text="-----Entry-----" Importance="high" />
<Message Text=" Build " Importance="high" />
<Message Text="-----Entry-----" Importance="high" />
<MSBuild Projects="%(SolutionFile.Identity)" Targets="build" />
</Target>
</Project>
Item SolutionFile will contains paths for all .sln files that located in current directory and its subdirectories. You also may define path in CLI and perform searching relative it.
<ItemGroup>
<SolutionFile Include="$(MyDir)/**/*.sln"/>
</ItemGroup>
MSBuild task launches MSBuild for performing specified targets. In our cas it are restore and build. It task used 'batching' that allow iterate over items by metadata.
&"D:\Visual Studio\MSBuild\15.0\Bin\msbuild.exe" .\Make.targets
MSBuild targets | Item element | Item metadata in task batching | MSBuild task

Current build number not being considered during tfsbuild

Something is not right. i am trying to build and deploy thru the below code snippet, but it so happens that the current build doesnt get deployed, whereas if i give a build number older than a current build, that gets deployed. I am puzzled what is wrong ... Can you please help me ...
I am not sure why the current buildnumber is not being considered ...
<PropertyGroup>
<deployappsvr>\\vdev\$(HostedFolder);\\vdev2\$(HostedFolder)</deployappsvr>
<prjbin>Release\_PublishedWebsites\RE.Service</prjbin>
</PropertyGroup>
<Target Name ="AfterEndToEndIteration" Condition=" '$(IsDesktopBuild)'!='true' ">
<!-- Starting deployment to servers -->
<Message Text="Starting deployment to servers" />
<CallTarget Targets="DeployBatching" />
<Message Text="finished deploying to servers" />
<!-- Unmap TFS mapping -->
<Exec Command="tf workfold /unmap $(tfsmap) /workspace:$(WorkspaceName) /collection:http://tfsapp:8080/tfs"/>
</Target>
<ItemGroup>
<SrcToCopy Include="$(DropLocation)\$(BuildNumber)\$(prjbin)\**\*"/>
<DestToCopy Include="$(deployappsvr)"/>
</ItemGroup>
<Target Name="DeployBatching" Outputs="%(DestToCopy.FullPath)">
<PropertyGroup>
<DestToCopy>%(DestToCopy.FullPath)</DestToCopy>
</PropertyGroup>
<RemoveDir Directories="#(DestToCopy)"/>
<MakeDir Directories="#(DestToCopy)"/>
<Message Text="111 #(SrcToCopy) 222 $(prjbin) 333 "/>
<Message Text="444 Copying source files #(SrcToCopy->'$(DestToCopy)\%(RecursiveDir)\%(Filename)%(Extension)') "/>
<Copy
SourceFiles="#(SrcToCopy)"
DestinationFiles="#(SrcToCopy->'$(DestToCopy)\%(RecursiveDir)\%(Filename)%(Extension)')"/>
<Message Text="Finished Copying source files"/>
<Exec Command="powershell Invoke-Command -computername vdev -scriptblock {md c:\buildtestfolder} > c:\power\pwrcmd.log 2>&1"/>
</Target>
I tired deploying from the build server i.e deploying build artifacts from the server copy instead of dropzone, that seem to be working but deploying from dropzone doesn't seem to be working still.

How to publish additional files using msbuild file and TeamCity?

I'm using a msbuild file, TeamCity and Web Deploy to deploy my siteand everything works just fine, for the files included in the Visual Studio csproj file. In addition to these files I want to publish a couple of more files such as license files etc depending on environment.
This is my build file DeployToTest.proj:
<Project DefaultTargets="Deploy" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<ItemGroup>
<LicenseSourceFiles Include="License.config"/>
<RobotSourceFile Include="robots.txt" />
</ItemGroup>
<Target Name="Build">
<Message Text="Starting build" />
<MSBuild Projects="..\..\WebApp.sln" Properties="Configuration=Test" ContinueOnError="false" />
<Message Text="##teamcity[buildNumber '$(FullVersion)']"/>
<Message Text="Build successful" />
</Target>
<Target Name="Deploy" DependsOnTargets="Build">
<Copy SourceFiles="#(LicenseSourceFiles)" DestinationFolder="..\..\wwroot"></Copy>
<Copy SourceFiles="#(RobotSourceFile)" DestinationFolder="..\..\wwwroot"></Copy>
<Message Text="Started deploying to test" />
<Exec Command="C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe ..\..\wwwroot\WebApp.csproj /property:Configuration=Test /t:MsDeployPublish /p:MsDeployServiceUrl=99.99.99.99;DeployIisAppPath=MySite;username=user;password=pass;allowuntrustedcertificate=true" />
<Message Text="Finished deploying to test" />
</Target>
</Project>
As you can see I tried to copy the license.config and robots.txt without any luck.
This .proj file is selected as the 'Build file path' in TeamCity.
Any suggestions on how I can accomplish this?
To solve this problem it may be worth executing the build script with the verbosity set to the 'detailed' or 'diagnostic' level. That should tell you exactly why the copy step fails.
However one of the most likely problems could be the fact that the script is using relative file paths, which depend on the working directory being set to the correct value. For build scripts I prefer use absolute paths to prevent any file path problems.
To get the absolute path you can use the MSBuildProjectDirectory property. The value of this property points to the path of the directory containing the currently executing MsBuild script. With that you can change your MsBuild script like this:
<Project DefaultTargets="Deploy" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">
<PropertyGroup>
<BaseDir>$(MSBuildProjectDirectory)</BaseDir>
</PropertyGroup>
<ItemGroup>
<LicenseSourceFiles Include="$(BaseDir)\License.config"/>
<RobotSourceFile Include="$(BaseDir)\robots.txt" />
</ItemGroup>
<Target Name="Build">
<Message Text="Starting build" />
<MSBuild Projects="$(BaseDir)\..\..\WebApp.sln" Properties="Configuration=Test" ContinueOnError="false" />
<Message Text="##teamcity[buildNumber '$(FullVersion)']"/>
<Message Text="Build successful" />
</Target>
<Target Name="Deploy" DependsOnTargets="Build">
<Copy SourceFiles="#(LicenseSourceFiles)" DestinationFolder="$(BaseDir)\..\..\wwroot"></Copy>
<Copy SourceFiles="#(RobotSourceFile)" DestinationFolder="$(BaseDir)\..\..\wwwroot"></Copy>
<Message Text="Started deploying to test" />
<Exec Command="C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe ..\..\wwwroot\WebApp.csproj /property:Configuration=Test /t:MsDeployPublish /p:MsDeployServiceUrl=99.99.99.99;DeployIisAppPath=MySite;username=user;password=pass;allowuntrustedcertificate=true" />
<Message Text="Finished deploying to test" />
</Target>
</Project>
Now this should fix the problem if there is indeed a problem with the relative file paths.
Solution was to change settings for the web project in Visual Studio. Under Package/Publish Web i set 'Items to deploy' to 'All files in this project folder'. I then added a filter to remove all .cs files and other unwanted files.

Incremental Build of Nuget Packages

I want to execute an msbuild project which uses batching to determine that one or more csproj projects have been freshly-built, and therefore require fresh nuget packaging. The script I've made so far seems like a reasonable start, but it the incremental-build mechanism isn't working. The MainBuild target executes every time, no matter what.
Here is what I have:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="MainBuild" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)'=='' ">Debug</Configuration>
<Content>content\plugins\</Content>
</PropertyGroup>
<ItemGroup>
<Nuspec Include="$(MSBuildProjectDirectory)\plugins\*\*.nuspec" />
</ItemGroup>
<Target Name="MainBuild"
Inputs="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll"
Outputs="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" >
<ItemGroup>
<Inputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll" />
<Outputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" />
</ItemGroup>
<Message Text="INPUTS: %(Inputs.FullPath)" />
<Message Text="OUTPUTS: #(Outputs->'%(FullPath)')" />
<Copy SourceFiles="#(Inputs)" DestinationFiles="#(Outputs->'%(FullPath)')" />
</Target>
</Project>
The Copy task is just a debugging placeholder for calling-out to nuget and creating a new package.
The idea is that if any files in the bin\Debug directory are newer than the corresponding .nuspec file (found two folders above bin\Debug), then the MainBuild target should execute.
Any ideas?
p.s. The Inputs and Outputs attributes of the Target presumably each create an item. I think it strange that the items created can't be referenced inside the target. In the above example, I had to make a target-interna dynamic ItemGroup to re-create the items, just so that I could access them. Is there a way around that?
I read this in the MSBuild Batching documentation
If a task inside of a target uses batching, MSBuild needs to determine
if the inputs and outputs for each batch of items is up-to-date.
Otherwise, the target is executed every time it is hit.
Which may be the cuprit. Try changing your copy target to use batching instead of an ite transform (I don't think using item metadata in an item group satisfies the above requirement).
<Target Name="MainBuild"
Inputs="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll"
Outputs="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" >
<ItemGroup>
<Inputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)bin\$(Configuration)\*.dll" />
<Outputs Include="%(Nuspec.RootDir)%(Nuspec.Directory)%(FileName).pkg" />
</ItemGroup>
<Message Text="INPUTS: %(Inputs.FullPath)" />
<Message Text="OUTPUTS: #(Outputs->'%(FullPath)')" />
<Copy SourceFiles="#(Inputs)" DestinationFiles="%(Outputs.FullPath)" />
</Target>
It looks like the number of inputs may be different than the number of outputs (I suspect there is more than one .dll files in the output directory for each project), which will also cause the target to execute.