Msbuild extremely slow compilation time - msbuild

Org is using TFS and msbuild for a large .Net code base with multiple daily deployments. Our builds fly on dedicated build servers except when the build compiles. This is partly due to the repo architecture, but is there a way to speed up compilation and/or msbuild alternatives that might do a better job?

Related

NAnt or TFS build which is better?

There was a question about Msbuild and NAnt advantages and disadvantages. Now let's see which is better TFS Build(with msbuild) or NAnt. In my opinion NAnt because you can easily move the building environment in few seconds to another machine (depends on copying files), also it's easier to manage, much faster to debug and it's not integrated with Team Foundation Server, what do You think?
It's worth noting that "TFS Build" (actually named "Team Build") uses the MSBUILD engine that's part of the Windows SDK now. It's a free engine into which you can plug custom tasks, and there's a community of users who have done just that. See
http://www.codeplex.com/sdctasks
http://www.codeplex.com/MSBuildExtensionPack
among many others.
MSBUILD is very mature, having been part of .NET since version 2.0. It could hardly be any more widely-used, as it is the build engine that builds all C# and VB.NET projects from the Visual Studio IDE.
It also comes with an API, with which you can programmatically control builds, do custom logging, etc.
I have used both. We have moved from Cruise Control / Nant to TFS.
The big benefit and the reason why we moved is the integration and reporting possibilities.
Nant is easier to work with for simple projects. But if you have a large environment and you look at it as your software development process, not just a build server, then I find TFS much better.
Just use what make you the most productive.
I haven't used TFS, but have been using NAnt for some time, so take the following points for what you will:
NAnt is free and open source
NAnt is pluggable with custom tasks
NAnt can be used together with various other tools, usually free and open source
NAnt is mature and widely used
NAnt is portable, doesn't require installation
The question should be nant vs msbuild. You could use Teambuild to run nant if you wanted.
Sometimes a solution is easier in msbuild, sometimes nant...
I like both, but msbuild is alway on the box...

MSBuild vs. NAnt for a mixed 2005 and 2003 codebase

We are currently maintaining a huge code base with hundreds of *.proj and perhaps 50 solutions with interconnected dependencies.
All of our code base is either .net or legacy C++.
Are current build process is a nested command line script that does automated source control pulling and then builds each solution in turn by calling devenv.exe.
I would like to take advantage of some of the better build and deployment tools and I think I have settled between either NAnt or MSBuild.
Here is the rub :
About half of the code base is being built under visual studio 2003. The other half is 2005. It is not politically feasible to migrate it forward to the current IDE at this time as it will require re-testing and introduce risk.
My question is - Has anyone successfully use MSBuild to build legacy 2003 solutions? Can I easily create customs tasks that call our legacy solutions by invoking devenv.exe?
Or should I only be considering NAnt at this time?
I think a hybrid approach might be best here as well. Just use your MSBuild scripts for the VS2005 projects, and build NAnt scripts for the older VS2003 projects. If you need a build script to manage both, use NAnt to control the process, along with the NAnt MSBuild task.
The downside here is that you will have to maintain some knowledge of two build systems, but my bet is that you'd be migrating to MSBuild over the long-term anyway.

Anybody out there using MsBuild to do Installs?

I've noticed projects such as Msbuild Extension Pack and MsBuild Community Tasks give msbuild the power to install assemblies, sql, and setup IIS. These features seem to be oriented to doing installs and not builds.
So I was wondering how many people out there are using msbuild, perhaps in conjunction with Cruise Control.Net to do installs on staging environments?
I use MsBuild to build, and part of the build process runs Wix to create an installer(MSI) which is used to deploy to production.
I wrote up a little sample of templating configurations for different target environments with msbuild: http://blog.privosoft.com/2010/10/msbuild-environment-sandboxing.html
We use CC.NET & MSBuild to build and then also to publish to our dev and stage environments, however we do not have the push live on CruiseControl.NET, we run that MSBuild by hand. We just thought it would be way to tempting with a button to publish live ;) It took probably 2 or 3 revisions to get our MSBuild set up right. But now everything is in one file, and everything is based on Targets and Properties to do all the work. About 6 months ago, was what should be the last update and that was a multi-server push so we are ready for scaling up. We can now push any combination of parts to any combination of servers. So if we want 5 database servers, 3 contenet servers, and 2 web servers we have that ability. No need to use anything else. MSBuild can do it.
I created a deployment system where a central coordinator can:
- identify the right target server for a given component (e.g Windows service goes to a given server, web services go to another, etc.)
- perform a PsExec of a deployment MSBuild script on the target server
- the deployment MSBuild script is responsible for:
a) downloading the right component package (in my case a .zip)
b) backing up previous versions of the component
c) extracting the package to the right place
d) tailoring the installation steps to the type of component to deploy (e.g. needs to perform an Exec task of installutil.exe on a Windows service )
e) logging the result of the deployment
This system is built using a mix of:
- core MSBuild tasks
- [Tigris MSBuild community tasks][1]
- [MS SDC tasks][2]
- and custom tasks
The system allows us to perform consistent deployment of complex apps across partitioned environments (e.g. DEV, QA, UAT, etc) made of virtual servers.
I use MSBuild to build a fairly large client/server application. I use InstallShield 2008 to create a separate client and server install set.
By adding a custom target into the build process you can combine the creation of the installers into the build.
I would recommend that you create and test the build and the installer separately, before attempting to integrate the two.
I know this is an old question... but...
I am currently using MSBuild with MSBuild Extension Pack (http://msbuildextensionpack.codeplex.com) to do my entire deployment. The database portion is handled with the VS database command-line tool (vsdbcmd.exe - http://msdn.microsoft.com/en-us/library/dd193283.aspx). That Extension Pack is pretty amazing, and is letting me build web sites, app pools, Windows services, update config, and much more.
I've also put Team City agents on the test servers, so I can deploy as part of a build chain (introduced in version 7 of Team City). And running my deploy MSBuild script is super easy from Team City.
I used to use MSBuild, now I'm using PowerShell. MSBuild is a build language. It is painful to script in. There is a lot I wanted to do in it that were difficult and sometimes impossible.
Over the past year, I've created an PowerShell module somewhat equivalent to MSBuild Extension Pack called Carbon.
I strongly, strongly encourage everyone out there to learn and use PowerShell.

Advantages of using MSBuild or NAnt versus running DevEnv.exe from command-line

Can anyone explain what advantages there are to using a tool like MSBuild (or NAnt) to build a collection of projects versus running DevEnv.exe from the command-line?
A colleague I had worked with in the past had explained that (at least with older versions of Visual Studio) using DevEnv.exe was much slower than the other techniques, but I haven't read any evidence of that or if that is now a moot point now that starting with 2005, Visual Studio uses MSBuild under the hood.
I know one advantage of using MSBuild allows you to build your projects without requiring Visual Studio to be installed on the build machines, but I wasn't sure if there were others.
One reason is because there's much more to building a product than just compiling it. Tasks such as creating installs, updating version numbers, creating escrows, distributing the final packages, etc. can be much easier because of what these tools (and their extensions) provide.
While you could do all this with regular scripts, using NAnt or MSBuild give you a solid framework for doing all this. There's a lot of community support for both, including additional tasks that can be downloaded (such as the MSBuild Community Tasks Project). Plus, there's support for them in numerous third party and open source products.
If you're just interested in compiling (and not the entire build process), you may find one time saving benefit of MSBuild is the support for building with multiple processors.
The obvious answer from my team is that not everbody has visual studio installed, in particular we do not install Visual Studio onto our build/CI servers.
The prime reason for using an external build tool like NAnt or MsBuild is the ability to automate your build process and thus provide continous feedback on the status of your system. Also they can be used for loads of things besides a "pure" build and that's where you really start to get value from them, it's an extremly valuable thing to be able to build and test your application with a single command.
You can also start adding stuff like collection of metrics, packinging of release binaries and all sorts of nifty stuff like that.
As far as C# goes, devenv.exe 2005 runs the compiler in-proc, which may cause out of memory exceptions for sizable solutions. Msbuild resorts to launching csc.exe process for each project. Projects that don't compile with devenv /build work fine with msbuild. Hope you like this reason.
We are experimenting with switching from DevEnv to a tool (Visual Build Pro) that uses MsBuild under the hood and we got a "Reference required to assembly 'System.Drawing..." error for a project which doesn't need it and which builds fine in Visual Studio.
We have a large system consisting of C#, managed C++, and plain old unmanaged C++ assemblies/dlls. There is C++ code that depends on managed C++ code that depends of C# code that depends of managed C++ code that depends on plain old C++ code (whew!). When we were setting up our automated build environment a few years ago we discovered that MSBuild.exe didn't properly handle all of the dependencies that we have.
Working with Microsoft we were able to solve some of the issues but not all of them. If my memory serves me, we never could get the C# assemblies that depended on managed C++ dlls to build. So we ended up making a custom build script that called devenv.exe from the command line and it worked just fine.
Of course, that was with VS2005, it might be fixed now, but the script is still working so we haven't revisited the issue.

Should I switch from nant to msbuild?

I currently use nant, ccnet (cruise control), svn, mbunit. I use msbuild to do my sln build just because it was simpler to shell out.
Are there any merits to switching my whole build script to MSBuild? I need to be able to run tests, watir style tests, xcopy deploy. Is this easier?
Update: Any compelling features that would cause me to shift from nant to msbuild?
I like MSBuild. One reason is that .csproj files are msbuild files, and building in VS is just like building at the command line. Another reason is the good support from TeamCity which is the CI server I've been using. If you start using MSBuild, and you want to do more custom things in your build process, get the MSBuild Community Tasks. They give you a bunch of nice extra tasks. I haven't used NAnt for several years now, and I haven't regretted it.
Also, as Ruben mentions, there are the SDC Tasks tasks on CodePlex.
For even more fun, there is the MSBuild Extension Pack on CodePlex, which includes a twitter task.
My advice is just the opposite - Avoid MSBuild like the plague. NANT is far far easier to set up your build to do automatic testing, deploy to multiple production environments, integrate with cruisecontrol for an entry environment, integrate with source control. We've gone through so much pain with TFS/MSBuild (Using TFSDeployer, custom powershell scripts, etc) to get it to do what we were able to do with NANT out of the box. Don't waste your time.
The most compelling reason to use MSBuild (at least in .NET 3.5 and beyond) - the build engine can build concurrently.
This means a huge speed up in your builds in you have multiple cores/processors.
Previous to 3.5, MSBuild didnt do parallel builds.
I feel that MSBuild and Nant are fairly comparable. If you are using one of these, I generally wouldn't switch between them unless there was a compelling feature that was missing in the product you had selected.
I personally use MSBuild for any new project, but your mileage may vary.
Hope that helps!
Edit: #ChanChan - #Jon mentions that Nant doesn't build .NET 3.5 applications. This may be enough of a reason to either change, or at least use them in parallel. As I've moved more towards MSBuild, I am probably not the most informed person to highlight any other showstoppers with either technology.
Edit: It appears Nant now builds .NET 3.5 Applications.
NAnt has been around longer, and is a considerably more mature product, and also IMO easier to use. There is a lot of community know-how out there to tap into, and it is also cross-platform, should you be interested in building apps that can run under Mono as well as .NET and Silverlight. Out of the box, it does a whole lot more than MSBuild does. Oh yes, and you can call MSBuild from NAnt (OK, from NAntContrib) :-)
On the negative side, NAnt and its sister project NAntContrib do seem to have stagnated, with the most recent update being late 2007.
The main advantages that I see of MSBuild is that it ships with the .NET Framework, so it's one less product to install; and there is more active development going on (albeit in places to catch up with the older NAnt).
Personally, I find its syntax a little more difficult to pick up, but then I'm sure continued exposure to ti would make things easier.
Conclusion? If you're working with existing NAnt scripts, stick with them, it's not worth the hassle of porting. If you're starting a new project, and you're feeling adventurous, then give MSBuild a go.
We also switched from nant to msbuild. If Your build is pretty standard, then You won't have much problems setting it up, but if You have a lot of specific build tasks, You will have to write custom ms build tasks, as there are way less custom tasks for msbuild.
If you want to display reasonable build results, You will have to mess with custom loggers etc. The whole team build is not as ripe as nant is.
But the real benefit is integration with TFS source control and reporting services. If You are not using TFS as Your source control system, it's not worth it.
Don't switch unless you have a very convincing reason (at least).
NAnt is open source and if it weren't I wouldn't be able to customize our build system, MSBuild is not.
NAnt can easily run MSBuild, I'm not sure about the other way around.
MSBuild scripts are already written for you if you use VS2005 or newer (the project files are MSBuild files.)
If you use NAnt, and you use VS to edit project files, settings and configurations, you'll have to write a converter/sync tool to update your NAnt files from the VS Project files.
#Brad Leach
I generally wouldn't switch between them unless there was a compelling feature that was missing
what are the compelling reasons to use msbuild? are there cons?
So far I'm getting a pretty good, "no don't bother" from your answer.
I think they're relatively comparable both in features and ease of use. Just from being C# based I find msbuild easier to work with than nants, though that's hardly a compelling reason to switch.
What exactly is nant not doing for you? Or are you just hoping there's some cool feature you may be missing out on? :)
One super-nice thing about C# is that if you have the .net framework, you have everything you need to run msbuild. This is fantastic when you are working on large teams / projects and have people/hardware turnover.
Personally I prefer SCons over both of them :)
The main reason I still use nAnt over msbuild for my automated builds is that I have more granular control on my builds. Due to msbuild using the csproj has it's build file, all the source in that project is compiled into one assembly. Which causes me to have a lot of projects in my solution for large projects where I am separating logic. Well with nant, I can arrange my build where I can compile what I want into multiple assemblies from one project.
I like this route, because it keeps me from having to many project files in my solution. I can have one project with folders splitting out the layers and then use nant to build each layer into it's own assembly.
However, I do use both nant and msbuild in conjunction for some build tasks, like building WPF applications. It is just a lot easier to compile a WPF application with the msbuild target within nant.
To end this and the point of my answer is that I like to use them side by side, but when I use msbuild in this configuration, it is usually for straight compiling, not performing any build automation tasks like copying files to a directory, generating the help documentation, or running my unit tests for example.
I'm actually still trying to figure this question out myself, but there is one big bonus for MSBuild here: using the same build file for local continuous integration by calling msbuild.exe directly, while also being able to use VSTS's server-side continuous integration with the same build file (albeit most likely different properties/settings).
i.e. as compared to TeamCity's support for MSBuild scripts, VSTS only supports MSBuild scripts! I've hacked around this in the past by exec'ing NAnt from MSBuild; I've seen others recommend this practice as well as the reverse, but it just seems kludgey to me, so I try not to do it if I can avoid it. So, when you're using "the full Microsoft stack" (VSTS and TFS), I'd suggest just sticking with MSBuild scripts.
Nant has more features out of the box, but MSBuild has a much better fundamental structure (item metadata rocks) which makes it much easier to build reusable MSBuild scripts.
MSBuild takes a while to understand, but once you do it's very nice.
Learning materials:
Inside the Microsoft Build Engine: Using MSBuild and Team Foundation Build
by Sayed Ibrahim Hashimi (Jan, 2009)
Deploying .NET Applications: Learning MSBuild and ClickOnce by Sayed
by Y. Hashimi (Sep, 2008)
I don't see any reason to switch. MsBuild itself locks you into the framework you are using. If you use NAnt, you can use it across many frameworks and shell out to msbuild to actually do the building task for you.
I am a fan of NAnt in this respect, because it decouples you from the framework a little bit.
I have created a framework that puts conventions into automated builds and I built it on NAnt. It's called UppercuT and it is the insanely easy to use Build Framework.
Automated Builds as easy as (1) solution name, (2) source control path, (3) company name for most projects!
http://code.google.com/p/uppercut/
Some good explanations here: UppercuT
MSBuild being integrated with Visual Studio gives programmers less friction to use the build system. It mainly comes down to them only having to go "Build Solution" and it all works, versus having to use Custom Build Steps and other such things, or, worse, forcing developers to build by launching some kind of external script.
Now, I mostly tend to prefer MSBuild over NAnt because it's simpler. Sure, NAnt has a lot more features, is more powerful, etc., but it can quickly get out of hand. If you and your build engineers have the discipline to keep the NAnt scripts simple, then it's all good. However, I've seen too many NAnt-based systems go south to a point where nobody understands what it's doing anymore, and there's no real way to debug it besides doing the equivalent of a good ol' printf. The moment you start using some if/else statement or for loop, that's where, IMHO, it starts smelling.
On the other hand, MSBuild has a solid foundation based on metadata and a less verbose syntax. Its simplicity (or lack of features... depending on how you see it) forces you to write logic in .NET code via new tasks, instead of writing logic in XML markup. This encourages re-usability and, above all things, lets you actually debug your build system in a real debugger.
The only problem with MSBuild is the not-so-occasional bug (especially in the first version) or obscure (although documented) behaviour. And, if that's the kind of thing that really bothers you, being tied to Microsoft.
I switched from NANT to MSBuild. The project is running in .Net 4.0.
My experience in Nant was good. The project kind of died. And when .Net 4.0 came along, it was time to re evaluate the build process.
Since Nant was last released MSBuild has come along ways. At this point, MSBuild is the way to go. It's easy to use, has many extensions. I rewrote my Nant scripts in a day and a half. The MSBuild script is 1/3 the size of the Nant scripts.
Much of the work in the Nant script was setting up the different environments. In MsBuild/.Net 4.0 it's built-in.
I use MSBuild alongside Nant, because the current version of Nant can't as yet compile .NET 3.5 applications (same was true when .NET 2.0 first came out).
The only reason I can see for using msbuild is if you would like to use a automated build server like cruise control. If you are not going to switch, then I would leave it alone.
I use Nant and I love it. I used MSBuild and hated it because of these:
Microsoft forces you to follow their own build procedure that is so intrinsic to their doings that I at least was not able to make it work (I had to compile NET1.1 so I had to mix Nant and MSbuild). I know you can create your own MSBuild file, but I thought it was complex to understand and maintain.
ItemTypes to do file operations are just too hard to follow. You can have Nant do the exact same things and much easier and direct (I had to create an ItemType list and then pass to the file operations).
In MsBuild you have to create your own task dll, in Nant you can do this or you can embed C# code within your script, so its much easier to advance and just build the whole project.
Nant works with Net1.1, MsBuild doesn't.
To install nant, I can even unzip and locate inside my own repository to run it. To install MsBuild is much harder since it depends on many things from Visual Studio, etc. (maybe I'm wrong here, but that seems to be the truth).
Well these are my opinions...