How to create a common repository of MSBuild .proj files? - msbuild

Since I started using MSBuild for our projects, I've created several .proj scripts that are shared by several projects in our repository. All these shared scripts reside in a single directory.
So far I've been referring to the shared scripts by using a relative path, something like this:
<MSBuild Projects="..\..\common\build\MyScriptA.proj" Properties="ABC=XYZ"/>
However, every project also imports a common .proj script like so:
<Import Project="..\..\common\build\CommonImports.proj"/>
which <Import>s several other things and defines some properties.
This morning I thought I could replace the relative path with a variable, perhaps $(CommonDir), which would be defined by importing the CommonImports.proj mentioned above. This would enable me to call the common tasks like this:
<MSBuild Projects="$(CommonDir)\MyScriptA.proj" Properties="ABC=XYZ"/>
However, I can't figure out a way to define this $(CommonDir) variable in such a way as to make it work in all other MSBuild scripts that import CommonImports.proj, regardless of their location.
This question offers several ways of creating a property containing an absolute path from a relative path, but none of those seem to work if all I do is <Import> the script defining the property.
Question 1: I'm fairly new to MSBuild; is there a better way of creating a "library" of reusable .proj scripts I could run via the <MSBuild> task? I am aware of $(MSBuildExtensionsPath), however I would like the common tasks to reside in my checkout so that our build machine would automatically get the latest versions of the common tasks whenever it performs a checkout.
Question 2: How do I define $(CommonDir) inside CommonImports.proj so as to make it contain the absolute path to the directory containing CommonImports.proj?

I wonder if you shouldn't put it in the MSBuild path:
For example, this project is consumed via:
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
and installs itself into:
{program files}\MSBuild\MSBuildCommunityTasks
So perhaps define your own specific sub-folder, and use from there?
<Import Project="$(MSBuildExtensionsPath)\romkyns\CommonImports.proj"/>
etc. Because the $MSBuildExtensionsPath variable is defined separately you shouldn't have as much difficulty with it. Maybe.

This isn't a detailed answer, but here's some notes that are generally very useful when people solve this problem. All require MSBuild 4.0 or later.
(1) The $(MSBuildThisFile) property and similar properties. It allows imported files to refer to files relative to themselves, rather than to the project they're imported in. That uncouples them from the project they're imported into.
(2) The "GetDirectoryNameOfFileAbove" function. See here. This wonderfully useful function makes it possible for projects to import files whose location they don't know. Put the shared files (or one single stub shared file that imports the others from somewhere) at the top of your source tree. Then use this function in projects below it to find that stub and import it and thus all your shared build process.
(3) Import tag allows wildcards. This makes it possible to cause a file to be imported -- in other words, extend a build process -- by simply dropping it in a particular location, and editing no existing files.

Here's what's worked best for us.
First, check in all reusable stuff into a /common/build/ directory in the VCS.
Then, add a /common/build/CommonImports.proj, looking something like this:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
ToolsVersion="4.0">
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<Import Project="ILMerge.proj"/>
<Import Project="Mage.proj"/>
<Import Project="InnoSetup.proj"/>
<UsingTask TaskName="RT.Tasks.AssemblyVersion" AssemblyFile="Tasks\Release\RT.Tasks.dll"/>
<UsingTask TaskName="RT.Tasks.WaitForProcessesToTerminate" AssemblyFile="Tasks\Release\RT.Tasks.dll"/>
<PropertyGroup>
<BuildSingleProj>$(Root)\common\build\BuildSingle.proj</BuildSingleProj>
</PropertyGroup>
</Project>
This imports a few projects and tasks that everything uses, and defines a property group that is shared by everything. Then, add this in every build script:
<PropertyGroup>
<Root>..\..</Root>
[... other global properties ...]
</PropertyGroup>
<Import Project="$(Root)\common\build\CommonImports.proj"/>

10+ years later, another strategy is to put a Directory.Build.props file in the root of the repository declaring a property using $(MSBuildThisFileDirectory) to get the path to the root of the repository (or other useful place). Then all projects in the repository can use paths relative to the root instead of to themselves.
I would probably only do this if I don't mind an extra file at the root of the repository, and if the repository primarily consists of MSBuildable projects.
Example:
/Directory.Build.props
/common/build/MyScriptA.proj
/common/build/CommonImports.proj
Directory.Build.props:
<Project>
<PropertyGroup>
<RepoRootDir>$(MSBuildThisFileDirectory)</RepoRootDir>
<CommonDir>$(RepoRootDir)common\build\</CommonDir>
</PropertyGroup>
</Project>
Note that if you use nested Directory.Build.props files, they would need to import upwards using something like this: <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" /> since it stops after finding the first match.

Related

How to dynamically load properties from a file in MSBuild

I'm scripting my build process, and I'd like to have a single MSBuild script that clones a repo, then includes properties in a file in that repo to drive the build. The only way I've found to include properties from another file is with the Import task, which can't reside inside a Target, so the file it's importing has to exist when MSBuild is initially invoked. Is there any way to run the Import after a target has run, or another way altogether to get properties out of a file in the middle of a build?
You have to call "msbuild" task to run another msbuild process for same project with specific params (path to .props file for example).
<Target Name="Default">
<MSBuild
Projects="$(MSBuildThisFileFullPath)"
Properties="ParamsPath='./ParamsPath/name.props"
Targets="DoSomethingTarget"/>
</Target>
<Import Project=$(ParamsPath) Condition="Exists('$(ParamsPath)')"/>
<Target Name="DoSomethingTarget">
<DoSomeThingTasks/>
</Target>
But Im sure that "the right tool for right the job". Maybe you should look at solutions such as FAKE, PSake, Cake?
I decided to make two separate calls to my script in the BAT script that invokes it. The first calls my Clone task, which creates the properties file. The second calls the tasks that require those properties. I made sure the <Import> task has a conditional requiring the file to exist. I'm still open to a cleaner approach, but this works.

Use $(SolutionName) in the MsBuild commandline parameters

In order to emulate the "PerProject" option in TFS 2013's XAML build in the new Build 2015 task based builds, I'd like to be able to pass the SolutionName to the msbuild commandline arguments without having to manually set it every time.
I'd like to do something like:
/p:OutputPath=$(Build.BinariesDirectory)\$(SolutionName)\
Where I'd like MsBuild to infer the $(SolutionName) parameter. But when passing this on the commandline, the new task runner will substitute the $(Build.BinariesDirectory) with the correct target path and leaves $(SolutionName) alone. Unfortunately MsBuild subsequently also leaves the property alone:
Copying file from "obj\Debug\TFSBuild.exe" to "bin\debug\$(SolutionName)\TFSBuild.exe".
TFSBuild -> b\$(SolutionName)\TFSBuild.exe
Copying file from "obj\Debug\TFSBuild.pdb" to "b\$(SolutionName)\TFSBuild.pdb".
I can't remember a way to pass a property to the commandline and have it do late-expansion... Any tips?
For those looking to emulate SingleFolder or AsConfigured, those are easy:
SingleFolder -> /p:OutputPath="$(Build.BinariesDirectory)"
Asconfigured -> don't pass OutputPath
PerProject -> /p:OutputPath="$(Build.BinariesDirectory)\HARDCODESOLUTIONNAME"
As I feared there doesn't seem to be a simple way to override a property from the commandline and "inject" the value of another property into it during the evaluation stage.
There are a few ways to get around it, but they're not ideal and certainly not universal for each language supported by MsBuild. A pity.
I've debugged the MsBuild targets files and found a solution to reproduce the old behaviour from the 2005/2008 era. Not entirely per solution, but it does redirect projects into a subfolder.
/p:GenerateProjectSpecificOutputFolder=true /p:OutDirWasSpecified=true
/p:OutputPath=$(Build.BinariesDirectory)
Normally, $(SolutionName) is defined when executing solution-level MSBuild pipelines, such as running dotnet restore in the root solution directory.
To make $(SolutionName) available for project-level MSBuild pipelines, add a Directory.Build.props file in the root of your solution with the following contents:
<Project>
<PropertyGroup>
<SolutionName Condition="'$(SolutionName)' == ''">
$([System.IO.Path]::GetFileNameWithoutExtension($([System.IO.Directory]::GetFiles("$(MSBuildThisFileDirectory)", "*.sln")[0])))
</SolutionName>
</PropertyGroup>
</Project>
Now $(SolutionName) will be defined even when executing project-level MSBuild pipelines.
This answer works best when there is exactly one solution file in the root of the solution directory. You'll need to massage the above a bit for other project structures.
Of course, you can also be lazy and specify the solution name directly, but this opens up the possibility of refactoring issues (need to remember to update this file if the solution name changes).
<Project>
<PropertyGroup>
<SolutionName Condition="'$(SolutionName)' == ''">
MySolutionName
</SolutionName>
</PropertyGroup>
</Project>
One solution is to mimic such 'late evaluation' yourself by altering OutputPath withing the projectfile. To do without manually changing each single project file you can use the CustomBeforeMicrosoftCSharpTargets extension point. Which is an fancy way of saying it is just a property which when found and pointing to an existing file, will lead that file to be imported somewhere before all the actual build logic. Here's the idea: create a file like paths.targets somewhere - either include it in source control or you can generate it on the fly as part of the build process. Contents:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<OutputPath Condition="'$(OutputPathBaseDir)'!=''">$(OutputPathBaseDir)\$(SolutionName)</OutputPath>
</PropertyGroup>
</Project>
So this just overrides OutputPath to some base dir + solutionname. Then if you build the solution like
msbuild my.sln /p:CustomBeforeMicrosoftCSharpTargets=paths.targets;
OutputPathBaseDir=$(Build.BinariesDirectory)
each project will import the paths.targets file and set output property to valueOfBinariesDirectory\my which I think is exactly what you are after.
You are right that TFS vNext build can't recognize $(SolutionName) in OutputPath, as $(SolutionName) doesn't list in the Predefined variables.
As an alternative, we may name the build definition with the solution name, then specify the MSBuild argument to: /p:OutputPath="$(Build.BinariesDirectory)\$(Build.DefinitionName)"in this way, we can get the output under the solution name.

MSBuild - Project-specific targets for solution does not work

I have a solution that has multiple projects in it, including a web application. I want MSBuild to execute "WebPublish" target against the web application project and "default target" for all other projects in the solution.
This MSDN article says that I can do it specifying the command line
msbuild SlnFolders.sln /t:NotInSlnfolder:Rebuild;NewFolder\InSolutionFolder:Clean
But I never could make it work - MSBuild return an error, something like "NotInSlnFolder:Rebuild" target does not exist. It does not matter what target to specify, Build, Rebuild or Clean - it does not work in any case.
How can I achieve my goal of specifying project-specific targets for a solution?
The MSDN documentation does not work. Or have I missed something?
NOTE: This workaround is not officially supported by Microsoft, so there is no guarantee that it will work forever.
Short Answer
In folder with the SLN file, create the file before.{YourSolution}.sln.targets, with the following content: (Replace what in curly brackets to whatever you need.)
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="{MyCompany_MyProduct_WebApp:WebPublish}">
<MSBuild
Condition="'%(ProjectReference.Identity)' == '{$(SolutionDir)MyCompany.MyProduct.WebApp\MyCompany.MyProduct.WebApp.csproj}'"
Projects="#(ProjectReference)"
Targets="{WebPublish}"
BuildInParallel="True"
ToolsVersion="4.0"
Properties="BuildingSolutionFile=true; CurrentSolutionConfigurationContents=$(CurrentSolutionConfigurationContents); SolutionDir=$(SolutionDir); SolutionExt=$(SolutionExt); SolutionFileName=$(SolutionFileName); SolutionName=$(SolutionName); SolutionPath=$(SolutionPath)"
SkipNonexistentProjects="%(ProjectReference.SkipNonexistentProjects)" />
</Target>
</Project>
After that you can execute the command line:
msbuild {YourSolution}.sln /t:{MyCompany_MyProduct_WebApp:WebPublish}
Long Answer
If you add environment variable MSBUILDEMITSOLUTION, setting its value to 1, MSBuild will not delete temporary files generated for the solution and projects.
This will allow you to find {YourSolution}.sln.metaproj and {YourSolution}.sln.metaproj.tmp files generated in the solution folder, which are just standard MSBuild project files.
For MSBuild 3.5, the generated file is {YourSolution}.sln.cache and is retained regardless of environment variables. Analyzing those files, you will understand low-level details of the process and to see the customization opportunities available.
After executing MSBuild with some project-specific target in the .Metaproj file you will find out that the list of project-specific targets is hardcoded and only standard targets are supported (Build, Rebuild, Clean, Compile, Publish; note: Publish and WebPublish are not the same). MSBuild 3.5 only generates Clean, Rebuild and Publish targets as well as a target with just the project's name that means "Build".
You also can see that NotInSlnfolder:Rebuild is just a name of an autogenerated target. In reality MSBuild does not parse it and does not care about project names and location. Also note that the autogenerated target names specify the project name with solution folders hierarchy if it's in one, e.g. SolFolder\SolSubfolder\ProjectName:Publish.
One more critically important thing you will find: The MSBuild Target Name does not support dots. All dots in project names are replaced with underscores. For example, for a project named MyCompany.MyProduct.Components you will have to specify in the command line:
/t:MyCompany_MyProduct_Components:Rebuild
That's why even standard project-specific target Build didn't work - my project name contained dots.
Analyzing file {YourSolution}.sln.metaproj.tmp, you will find out that at runtime it tries to import targets from file named before.{YourSolution}.sln.targets and after.{YourSolution}.sln.targets, if those files exist. This has a key to the workaround for this MSBuild limitation/bug.
You can open your solution file in text editor and check whether following line is exist or not if not then you can add
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" /> inside the <Project> tag.
Hope this help you.

Preventing MSBuild from building a project in a .sln without using Solution Configurations

I want to inhibit the building of certain projects within a solution from building (within a TeamCity Build Configuration in order to optimize the speed of my Commit Build feedback if you must know).
I'm aware of the Solution Configurations mechanism but don't want to have to force lots of .sln files to end up with every permutation of things I want to be able to switch off. I have Convention based rule where I want to say "If I'm doing the Commit Build, I dont want to do the final installer packaging". (And I don't want to break it out into a separate solution).
I'd prefer not to use a solution involving find and replace in the .sln file or in a .proj file created via [MsBuildEmitSolution][1]. I'm aware of questions here which cover the out of the box solution and this slightly related question.
I see MSBuild /v:diag is saying:
2>Target "Build" in file "Z.sln.metaproj" from project "Z.sln" (entry point):
Using "MSBuild" task from assembly "Microsoft.Build.Tasks.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "MSBuild"
Global Properties:
BuildingSolutionFile=true
CurrentSolutionConfigurationContents=<SolutionConfiguration>
<ProjectConfiguration Project="{C83D035D-169B-4023-9BEE-1790C9FE22AB}" AbsolutePath="X.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
<ProjectConfiguration Project="{15E7887D-F1DB-4D85-8454-E4EF5CBDE6D5}" AbsolutePath="Y.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
</SolutionConfiguration>
So the question is:
Is there a neat way of me getting to do an XPath replace or similar to have the effect of changing BuildProjectInSolution="True" to BuildProjectInSolution="False" for Project Y above
Failing that, is there a relatively simple edit I can do within a .ccproj (An Azure 1.4 Package) or a .csproj (a general project) file to cause the effects (including triggering of dependent projects) of the project being enabled within a commandline msbuild Z.sln solution build to be nullified?
Not sure it qualifies as neat, but you can set CustomAfterMicrosoftCommonTargets to import an msbuild file to over-ride the BuildDependsOn property, pointing it to your own custom build task. Basically, by setting CustomAfterMicrosoftCommonTargets you get msbuild to import an msbuild file containing the following:
<PropertyGroup>
<OldBuildDependsOn>$(BuildDependsOn)</OldBuildDependsOn>
<BuildDependsOn>MyBuild</BuildDependsOn>
</PropertyGroup>
<Target Name="OldBuild" DependsOnTargets="$(OldBuildDependsOn)" />
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="<IfIWantThis>" />
</Target>
Edit
You can use the following MyBuild target to Include/Exclude projects based on regular expressions passed in as IncludeInBuild and ExcludeFromBuild properties. (If you want complex regexes, you may fall foul of MSBuild special character escaping, but this works well enough for simple matching)
> msbuild /p:ExcludeFromBuild="Tests|Install|Azure"
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="('$(IncludeInBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(IncludeInBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='True') AND
('$(ExcludeFromBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(ExcludeFromBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='False')" />
</Target>
You could always pass the particular projects you want to build as parameters to the MSBuild.
The MSBuild command line would look like this:
MSBuild /t:<Project Name>:Rebuild;<Another Project Name>:Rebuild
In TeamCity, you would put <Project Name>:<Target Action> in the target field in the MSBuild runner.
I add a system parameter under Parameters
Name: system.ExcludeFromBuild
Kind: System property (system.)
Value: path to your csproj

How do I import the msbuildcommunitytasks project from another msbuild project with a relative file path?

Please go easy I am new to msbuild and msbuildtasks!
How can I set a property which represents a relative file path to a targets file which I want to import? I need relative references so it will work on all dev machines. But the target for import is trying to use the relative file path internally, which won't work as it is re-evaluated relative to the imported target!
Effectively I am trying to work around the documented behaviour of imported projects:
All relative paths in imported
projects are interpreted relative to
the directory of the imported project.
Therefore, if a project file is
imported into several project files in
different locations, the relative
paths in the imported project file
will be interpreted differently for
each imported project.
There was a similar question at Is it possible to use MSBuild Extension Pack without installation?. That question was how to do the same with the MSBuild Extension Pack, both of which are similar in this aspect. For the Extension Pack you have to declare the property ExtensionTasksPath,and for the Community tasks you have to declare a similar property named MSBuildCommunityTasksLib. So in your case it should look like:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<MSBuildCommunityTasksLib Condition="'$(MSBuildCommunityTasksLib)' == ''">E:\Data\Development\My Code\Community\MSBuild\CommunityTasks\</MSBuildCommunityTasksLib>
</PropertyGroup>
<Import Project="$(MSBuildCommunityTasksLib)MSBuild.Community.Tasks.Targets"/>
<Target Name="Demo">
<!-- Use the tasks here -->
</Target>
</Project>
Ok, I've found the answer. Essentially you have to set the property MSBuildCommunityTasksPath as a relative path back to the original containing directory.
For example, given a folder structure like this:
Root---project---Build---{My msbuild project}
|
|-Tools---MSBuildCommunityTasks---{Binaries and Targets}
Where :
{My msbuild project} is in Root\Project\Build\
{MSbuildCommunityTasks} is in Root\Project\Tools\MsBuildCommunityTasks
To get the targets project to reference its binaries via the property MSBuildCommunityTasksPath, it will find the tasks file like this:
<PropertyGroup>
<MSBuildCommunityTasksPath>..\MSBuildCommunityTasks\</MSBuildCommunityTasksPath> <!--Relative path back to yourself-->
</PropertyGroup>
Then you can import the targets file with another relative file reference :
<Import Project="..\..\Tools\MSBuildCommunityTasks\MsBuild.Community.Tasks.Targets"/>
#Sayed Ibrahim Hashimi
Talkin about MSBuild4
Just declaring the MSBuildCommunityTasksLib wont suffice cause if u check the MSBuild.Community.Tasks.Targets file the properties are declared as follows
<PropertyGroup>
<MSBuildCommunityTasksPath Condition="'$(MSBuildCommunityTasksPath)' == ''">$(MSBuildExtensionsPath)\MSBuildCommunityTasks</MSBuildCommunityTasksPath>
<MSBuildCommunityTasksLib>$(MSBuildCommunityTasksPath)\MSBuild.Community.Tasks.dll</MSBuildCommunityTasksLib>
</PropertyGroup>
So if U only over ride the MSBuildCommunityTasksLib it will again get over ridden in the MSBuild.Community.Tasks.Targets file as it is not conditional
So u HAVE TO ALSO OVERRIDE MSBuildCommunityTasksPath so that its proerty is NOT SET FROM MSBuildExtensionsPath but from ur custom path.
Correst me if I m wrong
This appears to be one answer:
http://social.msdn.microsoft.com/forums/en-US/msbuild/thread/feb782e3-72ae-4476-9011-617796f217b6
But this (if I understand it correctly) appears to be a ridiculous solution. To get the paths to work I need to change the imported project references? What would happen if I wanted to reference the imported project from third project in another folder?!?
I'm a noob at msbuild if I'm quite honest however I've just solved my own problem I had with this. I was turning one of the targets into its own project and it wasn't finding the paths for the msbuild community paths. If you look at your original project you may find something like this
<PropertyGroup>
<ExtensionTasksPath>./</ExtensionTasksPath>
<MSBuildCommunityTasksPath>./</MSBuildCommunityTasksPath>
</PropertyGroup>
<Import Project="MSBuildExtensionPack\MSBuild.ExtensionPack.tasks"/>
<Import Project="MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
Copy this code into your new project and it should work.
I just wanted to add, since i cannot comment (rep), that to do a path to your particular project you can use $(SolutionDir) on your property group like so:
$(SolutionDir)\My Code\Community\MSBuild\CommunityTasks\
This way its not tied down to a specific drive and can be based off of the location of the project relative to your solutions directory structure.
Also thanks for the answer above it helped me in my project with the addition above.