I have a solution with several projects in it. Let's say
project A depends on projects B and C
project B depends on project C
When I run my solution on the local machine VS builds each project once and it takes 1 minute. However, on our build machine it takes about 4 minutes to build and, as I can understand from the MSBuild logs it goes like this:
build A -> build B for A, build C for A
build B -> build C for B
So it builds some projects several times... How can I speed up the build process?
P.S. It's not a question of 3 extra minutes, I just wonder why is it so different from my local machine build?
I am not sure about your build order. Sometimes TeamBuild can look like it is building projects over and over but it is building for different configurations. Take a look and make sure you have not defined multiple FlavorsToBuild.
Also, if you don't want to do a fresh check out and rebuild every time, you can define this at the bottom of your TFSBuild file.
<PropertyGroup>
<IncrementalBuild>true</IncrementalBuild>
</PropertyGroup>-->
Put that right before the </Project> tag.
This sample seems to work for me. TestLib.Extra depends on TestLib. If I change something in TestLib, both projects will build. If I change only in TestLib.Extra, only that one will build, and if I don't change anything at all, they will just report Skipping target "CoreCompile" because all output files are up-to-date and so on.
<Target Name="Build">
<MSBuild Targets="Build" Projects="TestLib\TestLib.csproj" />
<MSBuild Targets="Build" Projects="TestLib.Extra\TestLib.Extra.csproj" />
</Target>
The trick is to use the "Build" target of the projects, rather than "Rebuild". The difference between these is essentially the same as the difference between the "Build" and "Rebuild" commands in the build menu in Visual Studio.
Edit
This works well also if the projects are included in a solution file, and you specify the solution to build instead:
<Target Name="Build">
<MSBuild Targets="Build" Projects="TestLib.sln" />
</Target>
Maybe like our's, your build server is a virtual machine (10x at least slower).
Also TFS (and maybe others), does a fresh checkout on build, so it will have to build all the projects regardless.
Are you using the /maxcpucount switch? There may be a difference in number of processors between your local machine and the build machine. This setting can also be different between your msbuild file and the visual studio setting which could also explain the difference you're seeing in build times.
I do this as follows. It is somewhat complicated custom build system but the basic idea is.
The dlls which are reused in many solutions are build to a known folder. This is achieved my using a msbuild project file that builds these common dlls.
When building other csproj files in a solution we copy the csproj files then use xslt manipulation to replace the project references with dll refernces for those common dlls.
The build scripts then build these changed csproj files using custom msbuild project files we maintain corresponding to each solutions. We don't build .sln files. These custom project files is a itemgroup of .csproj files in correct dependency order.
Maybe this can help you in achieving what you want.
Related
I am trying to extend our existing Wix Project with Heat to include all files from the output directory of another project in the installer. Somehow I`m missing something that should be very easy since I guess this is a very common task.
Here is what the solution looks like:
SomeDependencyA.csproj
SomeDependencyB.csproj (Depends on SomeDependencyA)
WindowsService.csproj (Depends on SomeDependencyB)
Installer.wixproj(Depends on WindowsService.csproj)
Wix should basically grep anything from WindowsService Output folder and build the msi package from it. The process should be:
Build all Dependencies
Build WindowsService project
Generate a Fragement using Heat
Build the Installer
And here is my question: Where/how do I have to include the call to Heat? Pre-Build Step and BeforeBuild Target does not work, since at this time the WindowsService and the Dependencies are not Build yet. AfterBuild wouldn´t help, since at that point the generated Fragment would not be cosidered anymore. I was looking for some build target that is calles AFTER all dependencies have been build but BEFORE the Installer Project itself has been build. Should be easy, but I couldn´t figure it out yet.
The BeforeBuild target will happen potentially before a dependant project has completed its build.
A solution I've used for this type of build ordering in the past is to add a DependsOnTargets for the BeforeBuild target;
For csproj:
<Target Name="BeforeBuild" DependsOnTargets="ResolveReferences">
For vcxproj:
<Target Name="BeforeBuild" BeforeTargets="PrepareForBuild">
This will force the project to wait until the dependant projects are completed before running the BeforeBuild target.
My organization has some huge builds that run on build servers, building lots and and lots of MSBUILD projects that are linked with ProjectReferences. We need to be able to build projects and configurations in parallel with msbuild /m.
My problem is that I have a project that is referenced from a large number of other projects, but the project itself is not reentrant. If more that two or more nodes try to build that project in parallel, it will fail.
How can I wrap this one Project, or it's Target, inside a critical section?
What I really need to do is something like this:
<Target>
<EnterCriticalSection ID=$(ProjectGuid) />
<Exec something />
<LeaveCriticalSection ID=$(ProjectGuid) />
</Target>
The idea being that if multiple MSBUILD nodes tried to build this project in parallel, only one of the nodes could do the execution, and the rest of the nodes would have to wait (or go do something else).
I suppose I could write custom MSBUILD tasks to do this, but isn't there some way of doing this built into the MSBUILD system?
=== EDIT 4/5/13. To clarify, the project is building a 3rd-party library with the build script provided for it. Completely rewriting their build script to make it reentrant -- by insuring that each build used a different set of folders for intermediate files, etc. -- is possible in theory, but not a practical solution. For one thing, all that work would would have to be redone on each new release of that library.
=== EDIT 4/6/13. On further reflection, I don't think it's even theoretically possible to insure a project is reentrant. Let me explain:
Suppose project XYZ is set up to use different temporary directories depending on platform and configuration in the usual way:
XYZ.proj:
<PropertyGroup>
<MyWorkingDir>tmp.$(Platform).$(Configuration)</MyWorkingDir>
</PropertyGroup>
Now suppose some other project GraphicsWindow references project XYZ, either through ProjectReferences or MSBuild tasks. And suppose the GraphicsWindow project can be built to use various graphics APIs. I.e., there's an OpenGL version, a DirectX 9 version, a DirectX 10, version...
So somewhere there's a .proj or .targets file containing a target to build all four versions:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=OpenGL</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D9</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D10</Properties>
</ProjectToBuild>
<ProjectToBuild Include="GraphicsWindow.proj">
<Properties>GraphicsApi=D3D11</Properties>
</ProjectToBuild>
</ItemGroup>
<Target Name="All">
<MSBuild Projects="#(ProjectToBuild)" BuildInParallel="true" />
</Target>
</Project>
or the equivalent using batching.
Now MSBuild will build the XYZ project 4 times with the same Platform|Configuration combination and the same working directory.
This will work fine as long as you build without the /m option and MSBuild runs a single thread. Depending on how the XYZ project is written, the 2nd, 3rd, and 4th builds might do nothing because the outputs are up-to-date, or it might do some redundant work, but the final result will be correct and the build will succeed.
But as soon as you start using parallel MSBuild, this build is broken! There is now a race condition where multiple threads can enter the XYZ project's target(s) concurrently, and start building using the same working directories, which will fail.
Irrespective of how you execute MSBuild, with multi-proc option /m or without, it is guaranteed to execute a project once for every configuration requested by the build. Here is a quote from MSDN:
When the Microsoft Build Engine encounters a project-to-project (P2P) reference while it is using parallel builds to build a project, it builds the reference only one time. If two projects have the same P2P reference, the reference is not rebuilt for each project. Instead, the build engine returns the same P2P reference to both projects that depend on it. Future requests in the session for the same target are provided the same P2P reference.
If you see the same project built more than once, it means it was referenced in two (or more) different configurations. By configuration here I mean a set of parameters that was passed to the project, e.g. project platform (x86, x64, AnyCPU, etc.,) flavor(debug/retail), localization language, any other parameters you might use.
More often than not, this is problem with mixture of project platforms. For example, you have project A built for x64, project B built for AnyCPU, and both A and B reference C. Now C has to be built twice -- for x64 and AnyCPU. If C correctly handles both platforms by cleanly separating outputs into separate directories, there is no problem. However if C treats x64 and AnyCPU as the same, it will fail randomly in multi-proc build.
Start by inspecting your solution configuration dialog. Make sure all projects have consistent set of platform/configuration parameters. If you have a need to build same project in different configuraions, make sure it puts output into separate locations.
I want to inhibit the building of certain projects within a solution from building (within a TeamCity Build Configuration in order to optimize the speed of my Commit Build feedback if you must know).
I'm aware of the Solution Configurations mechanism but don't want to have to force lots of .sln files to end up with every permutation of things I want to be able to switch off. I have Convention based rule where I want to say "If I'm doing the Commit Build, I dont want to do the final installer packaging". (And I don't want to break it out into a separate solution).
I'd prefer not to use a solution involving find and replace in the .sln file or in a .proj file created via [MsBuildEmitSolution][1]. I'm aware of questions here which cover the out of the box solution and this slightly related question.
I see MSBuild /v:diag is saying:
2>Target "Build" in file "Z.sln.metaproj" from project "Z.sln" (entry point):
Using "MSBuild" task from assembly "Microsoft.Build.Tasks.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "MSBuild"
Global Properties:
BuildingSolutionFile=true
CurrentSolutionConfigurationContents=<SolutionConfiguration>
<ProjectConfiguration Project="{C83D035D-169B-4023-9BEE-1790C9FE22AB}" AbsolutePath="X.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
<ProjectConfiguration Project="{15E7887D-F1DB-4D85-8454-E4EF5CBDE6D5}" AbsolutePath="Y.csproj" BuildProjectInSolution="True">Debug|AnyCPU</ProjectConfiguration>
</SolutionConfiguration>
So the question is:
Is there a neat way of me getting to do an XPath replace or similar to have the effect of changing BuildProjectInSolution="True" to BuildProjectInSolution="False" for Project Y above
Failing that, is there a relatively simple edit I can do within a .ccproj (An Azure 1.4 Package) or a .csproj (a general project) file to cause the effects (including triggering of dependent projects) of the project being enabled within a commandline msbuild Z.sln solution build to be nullified?
Not sure it qualifies as neat, but you can set CustomAfterMicrosoftCommonTargets to import an msbuild file to over-ride the BuildDependsOn property, pointing it to your own custom build task. Basically, by setting CustomAfterMicrosoftCommonTargets you get msbuild to import an msbuild file containing the following:
<PropertyGroup>
<OldBuildDependsOn>$(BuildDependsOn)</OldBuildDependsOn>
<BuildDependsOn>MyBuild</BuildDependsOn>
</PropertyGroup>
<Target Name="OldBuild" DependsOnTargets="$(OldBuildDependsOn)" />
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="<IfIWantThis>" />
</Target>
Edit
You can use the following MyBuild target to Include/Exclude projects based on regular expressions passed in as IncludeInBuild and ExcludeFromBuild properties. (If you want complex regexes, you may fall foul of MSBuild special character escaping, but this works well enough for simple matching)
> msbuild /p:ExcludeFromBuild="Tests|Install|Azure"
<Target Name="MyBuild">
<CallTarget Targets="OldBuild" Condition="('$(IncludeInBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(IncludeInBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='True') AND
('$(ExcludeFromBuild)'=='' OR
'$([System.Text.RegularExpressions.Regex]::IsMatch($(MSBuildProjectFullPath),
$(ExcludeFromBuild),
System.Text.RegularExpressions.RegexOptions.IgnoreCase))'=='False')" />
</Target>
You could always pass the particular projects you want to build as parameters to the MSBuild.
The MSBuild command line would look like this:
MSBuild /t:<Project Name>:Rebuild;<Another Project Name>:Rebuild
In TeamCity, you would put <Project Name>:<Target Action> in the target field in the MSBuild runner.
I add a system parameter under Parameters
Name: system.ExcludeFromBuild
Kind: System property (system.)
Value: path to your csproj
I need to collect into a single folder all test assemblies, with their dependencies, and configuration files. The process should preserve the directory structure from the output of each test project. We have a solution that requires manually attaching test projects to a master project, but our solution has far too many projects for this to be maintainable. These should be located automatically based on naming convention (x.UnitTest.csproj, y.IntegrationTest.csproj).
For background, we are working with a build system that passes artifacts (binaries, etc) between agents. We are compiling on one agent, and testing on other agents. The massive duplication of assemblies between test projects is slowing the build process down.
What I have done:
1) I have a csproj that references most of the test projects. This gets binaries and dependencies into one folder.
2) I am able to identify all files to copy using this
<CreateItem
Include="%(ProjectReference.RootDir)%(ProjectReference.Directory)$(OutDir)*.config">
<Output TaskParameter="Include" ItemName="TestConfigurationFiles"/>
</CreateItem>
<Copy
SourceFiles="#(TestConfigurationFiles)"
DestinationFolder="$(OutDir)">
</Copy>
I've attempted most obvious things, such as
MsBuild task: RebaseOutputs attribute, overriding the OutDir property. I can provide the msbuild task with a dynamically generated set of outputs, but can only build them in their default folder.
Hooking into the TargetOutputs of
msbuild task gives only the primary
output assembly (without
dependencies).
I experimented with "Copy Always" for
configuration files. This puts them
in the output directory of the
dependent project as "app.config" not
"dllname.config", and not in the
final project.
Solutions that could make this better might include
Provide an example of adding to the projectreference item array dynamically, before compilation.
Use msbuild TargetOutputs to create a list of all files in the folder (instead of just the primary output) and copy to a destination folder.
Today I'm using msbuild 3.5. Ideally the solution would work with msbuild 3.5. We are transitioning to .NET 4 / MsBuild 4 soon, so, if must be done in .Net 4, that is fine.
Have you considered flattening the folder structure when you export your artifacts?
Something like:
src/*.UnitTest*/bin/**/*.* -> /testlibs
src/*.IntegrationTest*/bin/**/*.* -> /testlibs
I have this custom build step form my project:
<Target Name="BeforeBuild">
<WriteLinesToFile Condition="" File="$(OutputPath)\env.config"
Lines="$(Configuration)" Overwrite="true">
</WriteLinesToFile>
</Target>
Basically it outputs the build configuration to a file.
This works fine when building in Visual Studio. Team Build decided to make things harder. The OutputPathfor Team Build is in a different spot (not bin\$(Configuration))
I know how to find out if the build is part of a team build (Condition=" '$(TeamBuildConstants)' == '_TEAM_BUILD_' ") but unless I want to hard code the path, I can't see a way to get the Team Build path.
Any ideas on how to find out (in the project, the the Team Build file) what the output location is?
You might need to use $(OutDir) when running in Team Build:
http://blogs.msdn.com/aaronhallberg/archive/2007/06/07/preserving-output-directory-structures-in-orcas-team-build.aspx
So you could use the _TEAM_BUILD_ condition and have two calls - one with OutputPath and one with OutDir.