I'm trying to figure out when the different targets are run. But I'm a little confused when it comes to the AfterBuild target, it's comment is "Redefine this target in your project in order to run tasks just after Build". But when I look at what Build depends on I see:
<BuildDependsOn>
BeforeBuild;
CoreBuild;
AfterBuild
</BuildDependsOn>
Dos not this mean that the Build target runs after "AfterBuild" or am I missing something here? I'm new to Build so maybe I have missed something trivial.
You should do some further research in the same file (just do a search on BuildDependsOn in a text editor): you'll see the Build target itself is just a stub that looks something like this:
<Target
Name="Build"
DependsOnTargets="$(BuildDependsOn)"/>
So when one calls msbuild /t:Build, msbuild looks up the build target and sees it has a DependsOnTargets property whith the value BeforeBuild;CoreBuild;AfterBuild (note that is a list). Since DependsOnTargets is always executed before the target itself, all targets listed therein executed first in the order listed. Only then the Build target itself is executed (so yes, that effectively happens after AfterBuild). But the Build target itself actually doesn't do anything: compiling etc all happens in CoreBuild so by the time it's invoked everything is done already.
This might seem odd at first, but it's actually a very expandable way to make targets depends on each other and define the order in which they run. (there's DependsOn, but also BeforeTargets and AfterTargets) So suppose you want a target that for clarity effectively runs after Build, you can use the same principle:
<Target Name="MyTarget" AfterTargets="Build">
...
</Target>
Note this is actually the preferred way: in large projects it's not reliable to override AfterBuild since you don't know if somebody else also did it already, and overriding it in multiple places results in only the last one found to be called.
Related
I have a custom .targets file which I import into my C# MVC web application's project file. I've added custom targets to this like so:
<Target Name="CopyFiles" BeforeTargets="Build"></Target>
This works fine when building under Visual Studio, but when I use TeamCity to build it, the target never gets run, and I can't work out why.
If I change my target to use BeforeTargets="Compile" then it runs. Alternatively, if I add an additional target with the name Build to the .targets file
<Target Name="Build" />
then it will run, but doing so overrides the existing Build target and thus my application doesn't build. I can't quite make out the logic to this - it doesn't make sense. I'm using the Compile target for now, but if someone could explain why trying to execute it before the Build task doesn't work I'd really appreciate it.
'Build' is a special built-in target, so doesn't really work the same way as most other targets. It definitely can't be safely overridden.
The most relevant documentation is here: https://msdn.microsoft.com/en-us/library/ms366724.aspx
If you want something to run before build, the standard approach (as recommend by the comments in a newly-created .csproj file) is to override the BeforeBuild target (as documented above).
However, this isn't the most robust solution. As noted in the documentation above:
Overriding predefined targets is an easy way to extend the build process, but, because MSBuild evaluates the definition of targets sequentially, there is no way to prevent another project that imports your project from overriding the targets you already have overridden.
It's better (and only slightly more complex), to override the BuildDependsOn property and extend the default value of this property to include the target you want to run (this is also documented in the link above).
Another approach would be to leave BeforeBuild empty and use BeforeTargets="BeforeBuild", which feels a bit odd but is quite simple and will still work even if the BeforeBuild target gets overridden.
As to why BeforeTargets="Build" doesn't work, I can't find a reference for this in the documentation, but I think it's to do with its special nature. It doesn't work the same as ordinary targets and it's probably better not to think of it as a target at all.
I don't mind an occasional repetition of something when it's necessary, but in MSBuild I really don't know how to ever avoid repetition. It doesn't offer "functions" in the usual sense; a target can only ever get called once, even via CallTarget, and <Import> only works on Project level.
Here's a specific example I'm trying to de-"repetize":
<Target Name="Tgt1">
<PropertyGroup><Conf1>Twiddle</Conf1><Conf2>Thing</Conf2></PropertyGroup>
<PropertyGroup><xxxxxxxxxxExePath>$(xxxxxxxBuildRoot)\$(Conf1)Console-xxxxxxxxed</xxxxxxxxorExePath></PropertyGroup>
<MSBuild Projects="$(BuildSingleProj)" Targets="Build;Merge"
Properties="Configuration=$(Conf1)$(Conf2);Platform=$(Platform);CompiledFileName=$(CompiledFileName);ProjectName=$(ProjectName);SolutionFile=$(SolutionFile);Root=$(Root);Caller=$(MSBuildProjectFullPath)"/>
<MakeDir Directories="$(xxxxxxxxorExePath)" />
<WriteLinesToFile File="$(xxxxxxxxorExePath)\xxxxxxx.IsPortable.txt" />
<WriteLinesToFile File="$(xxxxxxxxorExePath)\xxxxxxx.Global.Settings.xml" Lines="#(xxxxxxxLicense)" Overwrite="true" />
<Exec Command='$(xxxxxxxxorExePath)\xxxxxxx.exe -a "$(xxxxxxxBuildRoot)\$(Conf1)$(Conf2)-Merged\xxxxxxx.exe" "$(xxxxxxxBuildRoot)\$(Conf1)$(Conf2)-xxxxxxxxed\xxxxxxx.exe"'/>
</Target>
I have four such targets, Tgt1, Tgt2, Tgt3, Tgt4. The only thing that differs between these four targets is the first line, the one that defines Conf1 and Conf2.
The only more or less workable de-duplication idea that I'm aware of is by moving the shared code to a new target and calling it via the MSBuild task. This, unfortunately, requires a loooooong string of properties to be manually passed in, and this task uses rather a few (I counted 11 properties and 1 item group).
An additional requirement is that I can invoke the script with an arbitrary subset of these targets, e.g. \t:Tgt2,Tgt3.
Is there any sensible alternative to just copy/pasting this chunk of code - that doesn't involve copying around huge lists of properties instead?
This is a perfect scenario to use Batching.
You'll need to create custom Items with the appropriate metadata and then create a single Target to reference the new Items.
You can wrap each Item in it's own target like so:
<Target Name="Tgt1">
<ItemGroup>
<BuildConfig Include="Tgt1">
<Conf1>Twiddle</Conf1>
<Conf2>Thing</Conf2>
</BuildConfig>
</ItemGroup>
</Target>
<Target Name="Tgt2">
<ItemGroup>
<BuildConfig Include="Tgt2">
<Conf1>Twaddle</Conf1>
<Conf2>Thing 1</Conf2>
</BuildConfig>
</ItemGroup>
</Target>
<Target Name="Tgt3">
<ItemGroup>
<BuildConfig Include="Tgt3">
<Conf1>Tulip</Conf1>
<Conf2>Thing 2</Conf2>
</BuildConfig>
</ItemGroup>
</Target>
You'll then need a core target to call that will perform all of the work like so:
<Target Name="CoreBuild" Outputs="%(BuildConfig.Identity)">
<Message Text="Name : %(BuildConfig.Identity)" />
<Message Text="Conf1 : %(BuildConfig.Conf1)" />
<Message Text="Conf2 : %(BuildConfig.Conf2)" />
</Target>
Adding Outputs="%(BuildConfig.Identity)" to the target will make sure you batch at the target level instead of at the task level.
You can execute this from msbuild with passing arbitrary combinations of the targets as long as the last target is your core target. For example executing this command MSBuild.exe test.msbulid /t:Tgt1,Tgt3,CoreBuild will give you the following output:
Name : Tgt1
Conf1 : Twiddle
Conf2 : Thing
Name : Tgt3
Conf1 : Tulip
Conf2 : Thing 2
DRY is not a tenet of MSBuild. With that being said its not good to repeat yourself in any case, when it is reasonably avoidable. The answer that Aaron gave regarding batching is a good one. This is one means to prevent duplication.
One thing that I would like to point out is that at a higher level it seems like you are thinking of MSBuild as a procedural language (i.e. having functions that you can call and what not). MSBuild is much more declarative than procedural though. If you are creating MSBuild scripts and you have the mindset 'Create function X so that I can call it at point Y', then you're entering a world of pain. Instead you should think of MSBuild as phases. For example; gather files, compile, publish, etc. When you think of it in this way then it makes total sense why targets are skipped after they have been executed once (which you've obviously observed during your trials).
Also after having been working with MSBuild for as long as I have I've figured out that it can really be a PITA to do things in a generic/uber-reusable way. It can be done, but I would reserve that type of effort for .targets files that you know for sure will be re-used many times. Now a days instead of going through that I am much more pragmatic and I land somewhere in between totally hacking scripts & doing things the way I used to do. I have a set of scripts that I re-use, but besides those I try and keep things simple. One big reason for this is that there are a lot of people how know the basics of MSBuild, but very few who have a very deep knowledge of it. Creating good generic scripts requires a deep knowledge of MSBuild, so when you leave a project the person who comes in behind you will have no idea what you were doing (perhaps good if you are a contractor? lol).
In any case I've got a bunch of resources on batching at: http://sedotech.com/Resources#Batching.
I read John Robbins' article TFS 2010 Build Number and Assembly File Versions: Completely In Sync with Only MSBuild 4.0, and I'm wondering about the best way to go about integrating this.
The download for the article has two files, one is a targets file and one is a proj file.
The targets file has a number of tasks to scrape out a build number based on the Tfs build number (the same one used for the builds) and write that number out to some location (call it BuildNumberFile) for consumption by other proj files.
The proj file is very simple. It just imports the aforementioned targets file, and then declares a target with name "All" while also declaring DefaultTargets on the Project element to be All as well.
<Project ToolsVersion="4.0" DefaultTargets="All" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<!-- The two required properties so the Wintellect.TFSBuildNumber tasks knows your major and minor values.-->
<TFSMajorBuildNumber>3</TFSMajorBuildNumber>
<TFSMinorBuildNumber>1</TFSMinorBuildNumber>
</PropertyGroup>
<Import Project="Wintellect.TFSBuildNumber.targets"/>
<!-- Just ask for the version information files you need. These are here to show all the diffent ones in
Wintellect.TFSBuildNumber.Targets. You can change the names -->
<Target Name="All"
DependsOnTargets="WriteSharedCSharpAssemblyVersionFile;
WriteSharedVBAssemblyVersionFile;
WriteSharedCPPCLIAssemblyVersionFile;
WriteSharedCPPAssemblyVersionFile;
WriteSharedWiXAssemblyVersionFile;
WriteSharedTextAssemblyVersionFile;"/>
</Project>
I have two questions about this:
I'm still learning MSBuild. If the name of the target isn't specified elsewhere in the targets, is the target executed? How do I ensure that this target is run?
Are the csproj files supposed to declare an Include item for the location where BuildNumberFile is, even though it doesn't exist until compiletime?
Do ItemGroups and Include have a DependsOnTargets or something that allows them make sure the file exists before they build?
Are the entire contents of the csproj file using this supposed to be wrapped in a target that expresses DependsOnTargets for BuildNumberFile?
Thanks!
I think I've got this figured out, but two people promoted my question so I'll answer it here:
You can ensure that a target is run by expressing a dependency on it from another target. Microsoft.Common.targets exposes two targets--BeforeBuild and AfterBuild--expressly for the purpose of being overridden for customizability. I found the easiest way to do this was <Target Name="BeforeBuild" DependsOnTargets="WriteSharedCSharpAssemblyVersionFile" /> where WriteSharedCSharpAssemblyVersionFile is the target declared in the download from the link in the original post. Also, if you're new to MSBuild, this BeforeBuild target must be declared after the Microsoft.CSharp.targets is imported, but the default csproj template guides you in doing this.
The WriteSharedCSharpAssemblyVersionFile target should indeed write the file to some central location, since when building a solution, all targets are executed only once. All projects should reference the file from that location even if it doesn't exist, since by the time compilation happens (or more importantly, by the time references are resolved), the BeforeBuild target will have run and the file will be in place.
In my structure, I have these versioning files in a folder directly underneath the branch root folder. Furthermore, since the file being built is generated, I have it build to the output directory. It seems a little strange to be referencing things from the output, but it preserves the invariant of having all build products in one place so that the output directory can be blown away as a means of performing a clean.
In MSBuild items constitute inputs into the system (usually files) so it's weird to think of them depending on targets. After some learning this question doesn't make a lot of sense. In any case, the answer is no.
The entire contents of the file should indeed not be all in one target--all that is required is to import the Wintellect.TFSBuildNumber.targets file at the beginning of your csproj file, and declare BeforeBuild's dependency on WriteSharedCSharpAssemblyVersionFile at the end.
Hope this helps!
Is it possible to take a list of projects and parallel build them only if they aren't up to date?
<Target Name="DependenciesLevel7" DependsOnTargets="DependenciesLevel6">
<Message Text="5 items to build" />
<MSBuild Projects="C:\Projects\ApplicationManager.csproj;C:\Projects\Metrics.csproj" Properties="$(CustomAllProperties)" BuildInParallel="true">
<Output TaskParameter="TargetOutputs" ItemName="built_DependenciesLevel7" />
</MSBuild>
This is an example of the format i'm building in, I was hoping to be able to parallel build only items that aren't up to date here? Perhaps internal msbuild task calls are automatically parallel? If so how would I set this up so that it's incremental based on the previous build task? (Target DependenciesLevel6) I believed for incremental building you have to use Inputs/Outputs in your target.
Question summary:
Is the list of projects passed to an msbuild task automatically incremental (building is skipped if the build is already up to date)?
If not, is it possible to do incremental on a list of projects while parallelizing?
Can you do between target incremental where each target is a parallel build?
what you are describing "parallel incremental building" is already built in (kind of) but what you really need is parallel partially building.
Let me first explain the concepts and then I'll circle back how it works in your scenario specifically. There are two things you need to know about; incremental building and partial building. I just wrote a blog post discussing this at http://sedodream.com/2010/09/23/MSBuildYouveHeardOfIncrementalBuildingButHaveYouHeardOfPartialBuilding.aspx but I'll paste the relevant parts here.
Incremental building is the concept that you should only build what is out of date. To support this MSBuild has the attributes, inputs and outputs on the Target element. With these attributes you can specify the files that go into a target (via inputs attribute), and the files that you are expecting to come out of a target (via outputs attribute). Once you do this MSBuild will compare the timestamp of the inputs to the outputs and if all outputs are up-to-date (i.e. the inputs are older) then the target will be skipped. Take a look at the very simple project file below.
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Files Include="src\01.txt;src\02.txt;src\03.txt;src\04.txt;src\05.txt;"/>
</ItemGroup>
<PropertyGroup>
<Dest>dest\</Dest>
</PropertyGroup>
<Target Name="CopyFiles"
Inputs="#(Files)"
Outputs="#(Files->'$(Dest)%(Filename)%(Extension)')">
<Message Text="CopyFiles" />
<Copy SourceFiles="#(Files)"
DestinationFiles="#(Files->'$(Dest)%(Filename)%(Extension)')"/>
</Target>
<Target Name="DeleteTwoFiles">
<Message Text="DeleteTwoFiles" />
<Delete Files="$(dest)01.txt;$(dest)02.txt"/>
</Target>
</Project>
In this project file we have two targets; CopyFiles and DeleteTwoFiles. Ignore DeleteTwoFiles for now. Also take a note that the directory where I’m executing this build has a folder, src, with the files listed in the Files item. On the CopyFiles target I have specified the inputs and outputs. The inputs is just #(Files), this are the files that the target is acting upon. The outputs contains the expression #(Files->'$(Dest)%(Filename)%(Extension)'). Which is the same expression from the Copy statement. If the Dest folder is empty and I execute the CopyFiles target the result is shown below.
So just as expected the files were copied over, so its all good. Now what happens if I execute it again? The output is shown below
So as you can see the target was skipped, the message statement “CopyFiles” was not executed nor was the copy as a result. So this, in a nutshell, is incremental building.
Now, with the dest folder containing all files, what do you think would happen I execute the command msbuild.exe PartialBuilding01.proj /t:DeleteTwoFiles;CopyFiles? This command will first delete two files from the output directory and then call the CopyFiles target again. Let’s see the result below.
When the CopyFiles target was executed you see that statement “Building target ‘CopyFiles’ partially, …”. When the time came to execute the target MSBuild examined the inputs and outputs, it determined that the files 01.txt & 02.txt were out of date (because they didn’t exist in the target) but 03.txt, 04.txt and 05.txt were up to date. So MSBuild feed the CopyFiles target a value for the Files item that only contained the 01.txt and 02.txt and let it do its thing.
Now this relates to your problem in many ways some not as direct as you might hope. Firstly MSBuild will incrementally build your project, so if your project is up to date then it will not be built again. The thing is though that in order for MSBuild to determine that your project is up to date it has to load the project run the default target (usually Build) and then the targets themselves will figure out that there is no work to do. This stuff itself takes time. So if you have a huge number of projects, or a huge number of files inside of a project then you can take matters into your own hands. What you need is a way to determine if your projects are up to date or not and correctly express that inside of your inputs and outputs attributes. Once you do this you should be able to skip building the projects which are up to date.
The core of the problem is how do you craft the inputs/outputs to be correct. If you can think of a way to do that then you will get what you want. How you craft this will depend on your scenario but I could see something like this:
After each project build drop a file to a known location that is specific to that project
Before you build a project scan its directory, find the newest file and then update the timestamp of the project file to be that value
Then you can place the project files as the Inputs values and the marker files as the Outputs
Then call your target
In this case you assume that all dependencies are fully contained in files under the directory of the project (which may not be true). I'm not saying this is the ideal solution, but a solution.
==============================================
Edit: Update based on questoins below.
You will want to put the projects into an item (though not required) like ProjectFiles and then use #(ProjectFiles) for inputs. For outputs that is what I was saying is the hard part. You have to figure out a way to know (or indicate to you via your own process) that the projects are up to date. There is nothing built in for this.
Concern fo incremental build vs. clean build. In a perfect world incremental & clean builds are the same. But sometimes that is not the case. For many projects it is. If you start adding a bunch of targets to your build process and you set them up to do incremental build, but you do not implement that properly then you may MSBuild may skip targets when they were indeed out of date. A good example of this would be when you create a target with Inputs set to a list of files and then the Outputs set to a list of created files. If you do not extend the clean process to delete those created files, then the next time you Rebuild (assuming you didn't change the files) the target will be skipped when it should have been cleaned on the previous Rebuild.
You're probably looking for IncrediBuild.
I'm not sure about the incremental part but I read that it checks inputs/ouputs.
I make parallel builds with MsBuild with a configuration similar to yours, don't forget to run msbuild with the /m option.
You can try on a simple project and check the logs to see if it's incremental or not.
We've got a MsBuild.proj file which contains the following section (simplified):
<Target Name="WEB" DependsOnTargets="CleanResults;UpdateAssemblyInfo;Services;Business">
<!-- Other build and release stuff -->
<MSBuild Projects="$(CreateInstallValuesScriptProjectFile)" Properties="DatabaseStructureLocation=$(DatabaseDirectory)\Sandbox\002.Structure" />
</Target>
Basically, the InstallValuesScript generates a .sql file in our databasedirectory, which will update the version of our application in the database. Fairly simple.
The build is called as such:
MSBuild msbuild.proj /m /t:WEB /p:Configuration=Release;DoRelease=true;DoSandBox=false;DoWix=false /fileLoggerParameters:LogFile=msbuild.log;Verbosity=normal;Encoding=UTF-8
However, what we're seeing is that the InstallValues section is called multiple times, and as a result this file is created a couple of times, and on different locations... Obviously when the build is compiled two or three times instead of only once, thats annoying but not really critical (just goes a bit slower), however for this Installvalues file, we really don't want multiple instances of it.
So what gives, can a target be called multiple times? Maybe caused by the compiling of a dependant assembly? Some light on this strange phenomenom would be highly appreciated.
In general, unless EDIT: even if you <CallTask>, MSBuild won't run a Target multiple times within a given top-level invocation of a build, even if you have duplicates in dependencies (the order matters, but once its done, its done).
Do you perhaps have some nesting of calls to project files which chain in some complex manner?
But The Truth is in the Logs [that you seem to be creating so assiduously!]