Is there a mode, some switch or a programmatic way that I can ask MSBuild to display or output it's calculated dependencies for a given build file?
Some background -
I have a large project that requires splitting up to speed up the build time and want to remove the slow changing infrastructure code into it's own release area. Not all of the information is contained in the build file itself, as some sub-projects are referenced by their vcproj or csproj files.
I'd really like to see what MSBuild thinks needs doing (either worse-case [rebuild all] and perhaps for a make) without actually doing the rebuild.
The MSBuild Profiler project should be able to help you in seeing where time is being taken on the build. It doesn't directly show dependencies. With or without build dependencies, just profiling the builds can probably give some insight and help speed up the process.
I did just come across this application, but I have not used it myself yet, Dependency Visualizer that looks to have just added MSBuild-compatible project files. There have also been posts about doing this previously, but no code (see A, B).
Whilst I asked the original question quite a long time ago, I have moved on in jobs and surprisingly encountered the same need. In this case I was more successful in my pursuit of a tool and discovered Microsoft Build Sidekick which offers:
view
edit
build
debug
of Microsoft Visual Studio© 2005, 2008 and 2010 project files.
As well as debugging and logging features I haven't yet used, it has a diagramming mode where you can select the "Target" and it shows all of the dependent Targets and steps within them. Apparently this diagram can be viewed when stepping through the build process (debugging)!
Related
Latest:
This is definitely a bug in msbuild. Other than that there cannot be any other explanation. This could only be happening on Linux or possibly on a wider range.
So i decided to just build one single project with absolutely no dependencies on others in the solution.
Looking at the captured diagnostics, I see these lines which are very promising:
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
Input files: Annotations.cs;Auth.cs;AuthorizationConfig.cs;Backend.cs;Billing.cs;Code.cs;...
Output files: .obj/TheAgent.dll;.obj/TheAgent.pdb
Set Property: NoWarn=;1701;1702
15:23:27.396 1>Done building target "CoreCompile" in project "TheAgent.csproj".: (TargetId:40)
It looks like my dll and my pdb weren't built which is what I expected.
However, something must be happening before or after causing the timestamp to change (be that of this build time and not the last).
The timestamp of the dll is updated both in the intermediate object folder (.obj/) and also in the output folder.
Is there a known way of stopping msbuild right after its CoreCompile task?
Update:
I decided to search for is newer this time and found instances of these. I don't know how they have got to the solution/project files though:
Input file ".obj/Common.csproj.CoreCompileInputs.cache" is newer than output file ".obj/Common.pdb".
Further to the above, I came across this:
https://github.com/dotnet/project-system/issues/4736
Thinking that this was the issue, I upgraded to dotnet sdk version 2.2.402.
The end result is still the same :(
Original:
I need some pointers on how to troubleshoot this issue. I am using /t:build to build a solution file.
The resulting executable keeps getting refreshed each time.
First i thought the package restore was causing this. I have removed that step however it didn't make a difference.
Then I looked at this:
https://oz-code.com/blog/visual-studio-keeps-rebuilding-projects-no-good-reason/
I'm basically looking for some text in the diagnostics output which tells me if a target or a file is out of date and needs to be rebuilt. The above link talks about "project 'B' is not up to date". I don't have a not up to date in my msbuild output.
I already had two resources with CopyAlways which I changed to CopyIfNewer.
The above article also talks about circular dependencies. I am checking everything manually. And yes the references to dependent project are actually references to the project outputs (dll's /exe's). So Finding a circular dependency by just checking for that pattern seems a little odd.
There was one more problems in the dotnet platform and/or msbuild causing this to fail.
One of those was this https://github.com/dotnet/project-system/issues/4736
Installing SDK 3.0.100-preview7-012821 or better solved the problem
We have a VB.NET Windows Forms Solution developed using the 3.5 framework.
The solution has more than 30 projects (dlls) that are referenced from one to another. We even have some projects separated into N layers too (one for the data Access, other for the business logic, and so on).
Since a time ago the solution does not work properly. When we start Visual Studio (2008), it shows hundreds of errors. The vast majority are:
"Type 'XXXX' is not defined.
And after doing some "clean" + several "build" we are able to get the binaries.
Anyone has any idea about what is the problem and how to solve it??
You have a problem with the dependencies, maybe incorrect order or circular reference. For example this can be one of the scenarios you are suffering:
Project A depends on Project B
Project B depends on Project C
Project C depends no nothing.
You build all projects in this order A,B,C. First time you build A,B fails but C works. Second time A fails but B,C works. Third attempt A,B,C works.
In theory Visual Studio should work out the correct order... but if you have unused dependencies or circular references it fails.
You have to review and clean your dependencies. You can use
nDepend (http://ndepend.com) or similar tool (Resharper, Architecture Explorer in VS2010 Ultimate, etc.) to make a dependency map and then use it to clean your dependencies. You can
find a lot of good tools in this SO question.
Once you review and clean your dependencies it would build at the first attempt.
All you need is to correct your project build order.
First work out what projects depend on what others. The ones that do not depend on anything should go to the top of list and ones that depend on others should go below in order.
Then right-click your solution and select Project Build Order... from the popup menu.
The following dialog box will appear. Set the project build order in correct sequence. Use the Dependencies tab to specify the dependencies correctly.
So I have a CommonAssemblyInfo.cs linked into all the projects in my solution and is dynamically generated by my rake/albacore scripts which is not checked into source control.
I also have a CommonAssemblyInfo.cs.local for use when there is no ruby available, mainly to be used by devs.
Is it possible to have a msbuild task or something that runs before any of the other project compilation that will copy CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs before trying to compile my solution? I hate having to have a command you have to just know about and type in order to open and buidl the solution in Visual Studio.
UPDATE
So I ended up using a batch file as a solution wide pre-build event as described here: Solution-wide pre-build event?, it checks to see if CommonAssemblyInfo.cs exists and if not copies CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs just using a simple batch file.
This is the solution I ended up with.
I have each project in the solution link to a CommonAssemblyInfo.cs which is automagically generated for me by my build scripts (rake + albacore).
Since I cannot check CommonAssemblyInfo.cs into source control, I create a CommonAssemblyInfo.cs.local.
Simple solution: create go.bat which copies CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs that devs must run the first time they check out the project before opening the solution in VS.
For purely political reasons, if I did this people would have had hissy fits about me doing "nonstandard" things. Complex solution follows:
I created a project in the solution called PreBuild which every project in the solution depends on. This forces the build order to be such that this project is built first. This project contains a pre-build event which calls the following batch file:
echo verifying CommonVersionInfo.cs exists
cd
IF NOT EXIST ..\..\..\CommonAssemblyInfo.cs COPY ..\..\..\CommonAssemblyInfo.cs.local ..\..\..\CommonAssemblyInfo.cs
So now any developers who choose to keep their heads in the sand may checkout the project and blissfully open it up in VS unaware that any build scripts exist at all.
Are you talking about compilation in the VS IDE, or compilation through team build? If you are talking about team build, then you can use the "AfterGet" event as a place to use the standard "copy" msbuild task. If you are talking about the VS IDE, then you can still use the "copy" msbuild task.
I know there are many questions out there regarding this same information. I have read them all, but my brain is all turned around and I don't know which way to go. Plus the lack of documentation really hurts.
Here is my scenerio. We are trying to use WIX to create an installer for our application that goes out to our dealers for our product information. The app includes about 2000 images and documents of our products and a SQL CE database that are updated via Microsoft Sync Framework. The data changes so often that keeping these 2000 as content files in the app's project is very undesirable. The app relies on .NET Framework 3.5 SP1, SQL Server CE 3.5, Microsoft Sync Framework 1.0 and ADO.NET Sync Services 2.0.
Here are the requirements for the app:
The dealers will be given the app on a CD every year for any updates (app or data updates).
The app must update itself from the internet to get any new images, documents or data.
The prerequisites must be installed if they do not exist on the client machine.
The complete installer should be generated from an MSBuild script with as little human interaction as possible (we don't want to be manually updating the 2000+ file list).
What we have accomplished so far is that we have a Votive project in our solution. We have manually specified the binaries in a .wxs file. Web have modified the .wixproj file to use the HeatDirectory task to gather our data (images and documents and database) from a specified location (This is broken and giving an ICE38 error). This seems all right, but still is a lot of work. We have to manually update our data by running the program in release mode and copying it to the specified directory.
I am looking to see what other people would do in this situation.
How would you arrange your solution with regards to the 2000+ data files? Would you create a custom build script that gets the current data from the server or would you include them as content files in the main project?
How would you get WIX to include all of the project output (including the referenced assemblies) and all of the data files? If you have any complete samples, that would be great. All I have found are little clips here and there and not an entire example from start to finish.
How would you deal with the version numbers? Would you put them as a constant in the build script and reference them through the $(var.VersionNumberName)? Would you have the version number automatically picked up from the project being deployed? If so, How?
If there is any better information than what I am finding, please include. I have read numerous articles, blogs, Stackoverflow questions, the tuturial, the wiki, etc. Everything seems to be in bits and pieces. The tutorial is nice, but doesn't explain anything about MSBuild and Votive. I would like to see a start to finish tutorial on using MSBuild and Votive and all the WIX MSBuild targets. If no one knows of a tutorial like this I may put one together. I have already spent the entire week gathering info and reading. I'm new to MSBuild as well, so if anyone has any great articles on MSBuild, please include them.
The key is to isolate the different types of complexities into separate merge modules and put them altogether into an MSI as part of the build. That way things that change often can change without impacting things that hardly change at all.
1) For the data files:
We use Paraffin to generate the WiX and hence the merge modules for an html + Flash based help system consisting of thousands of files (I can't convince the customer to go to CHM).
Compile these into a merge module all by themselves.
2) Assemblies: assuming that this is a set that changes less often just make a merge module by hand or with WixEdit with the correct files and dependencies.
3) For the version number there a lot of ways to manage this depending on your build system. The AssemblyInfoTask is pretty straight forward way to make sure all your assemblies are versioned appropriately. The MSBuild Extension Pack has some versioning stuff if you are using TFS.
I had a similar scenario and was unable to find a drop in solution so ended up with the following:
I wrote a custom command line program called wixgen.exe for generating wxs manifest files. It is pretty specific to our implementation in that it only knows how to create 2 types of wxs files. One for IIS Website/Virtual Directory deployments and another for Windows Service deployments.
Each time a build is triggered by our continuous integration server a post-build task runs wixgen with the right args to generate a new manifest.wxs for the project being changed. It automatically includes all the files needed for the deployment. These builds also version the dlls using a variation of the technique at: http://richardsbraindump.blogspot.com/2007/07/versioning-builds-with-tfs-and-msbuild.html
A seperate build which is manually triggered is then used to build the wixproj projects containing the generated wxs files and produce the msi's.
I would ditch the CD delivery (so 90's) and got with ClickOnce. This solution seems to fit well since you already use the .NET framework. With ClickOnce you should be able to just keep updating the content of your solution and make updates available to your heart's content. Let me know if you need, sample ClickOnce deployment code.
You can find more ClickOnce information here.
Similar to dkackman's answer, you should seperate your build into several components, isolating build components to be built seperately.
I come from a mainly Java background, however for building MSIs and NET executables we use maven; with the 'maven-wix-plugin' plugin for building the installers, and using the NMaven plugin for compiling any NET code. However, as we're only performing very basic development in NET, with most development in Java, we don't need too much complexity from the NMaven plugin (which is probably a 'good thing' (TM) as it's only at version 0.17).
If you're a purely NET house, you could also look into Blydan (http://www.codeplex.com/byldan), which seems to be the focus of development there at the moment (it's the same team for NMaven and Byldan).
If you do want more information on NMaven or Byldan raise another question and I'll give as much info as I can (which is not a huge amount, as stated I only do very limited NET development).
To compile my current project, one exe with ~90,000 loc + ~100 DLL's it takes about a half hour or more depending on the speed of the workstation.
The build process is one of running devenv from Powershell scripts. This works very well with no problems.
The problem is that it is slow. I want to speed up this build process.
MSBuild (using VS-2005) is one option but there's a bug specifying icons to the vb compiler/linker on the command line such that it won't successfully link.
What other options are there to "make" VB.NET programs?
(Faster workstation is not an option.)
Do you absolutely have to compile the whole solution every time? With that many assemblies it seems unlikely that they all need to be built unless they actually change. If your solution is made up of multiple projects, you might consider creating multiple solutions in your build environment. One master solution could contain all the projects, another that includes the ones that change most often. You can then configure your build process to focus on the projects that have changed. Depending on the source control system you use, you may be able to query the system to determine which projects have changed since the last build, and only build those projects.
There's NAnt, and Cruisecontrol.NET for continuous build.
You mentioned that getting a faster PC is not an option, but how much memory do you have? 2GB should be the minimum for a developer machine. Also, using a fast 10K RPM hard disk makes a big difference.
Have you tried disabling any virus scanner during your build?
If you can, upgrade to the 3.5 version of MSBuild. It can build solution files, and enables support for multiprocessor support (or here if you need to host it yourself) enabling it to build projects in parallel.
The caveat is that you need to be using project references so it knows what to build.
Also, how long is it taking now? Have you looked at the CPU/Memory Usage (using something like PerfMon) to see if it is a bottleneck?
There's not much you can do to make the build process any faster short of adding more cores, CPU power, and memory to your machine, but that isn't an option in your case.
Most large projects are not self-contained in a single EXE. More often, logical units are moved into seperate assemblies, which can either be a DLL or EXE. The end result is a whole bunch of little assemblies, instead of one enormous one.
To cite one example, one project that I worked on was enormous, consisting of 700+ forms and 10s of 1000s of classes. Functionally related forms, such as those related to printing, report generation, user interrogation, etc were self-contained in their own EXEs. If I was working on the reports, I'd exclude all projects not related to reports from the build process, and this helps bring the compilation time down from a half hour to a few seconds.
This programming style can be tricky, but when it done right, it simply works and works flawlessly.
If you have a big number of projects then you should try to reduce them. You can always split them up in dll's later. The fewer projects the faster it can build. Especially if it has to build them in a certain order.
Breaking them in smaller solutions is also an option.