If I have an msbuild target that just copies changed files, why would incremental building be better than just using the Copy task and setting SkipUnchangedFiles = true?
Who said it would be?
The only reason I can think of would be if you extended your target later on to perform other tasks then just copying.
Related
My colleagues and I have user specific settings in csproj.user files. They are not checked into the repository. I would like for the build server to use its own set of csproj.user files, overriding certain properties, leaving the "base" project configuration at a decent developer default. But from the looks of it there is no such option in the msbuild command-line for doing that.
Is there really no way, other than copy csproj.user-files to where it'll be picked up by subsequent msbuild invocations?
While writing I realize I'm too much of a prude about these things and should just copy as a step prior build. Still posting in case someone knows a better way, for instance a way that does not modify the source tree.
Passing properties to the MSBuild command line overrides properties in the solution, including dependent projects. Here omitting debug information in build server, otherwise generated for release build to improve profiling:
msbuild MySolution.sln /p:DebugType=none ...
This does not work should I want different properties for different projects. Building projects individually should work nicely though.
Finally, passing arguments on command line can get messy, so to get a more "settings file"-like experience one may instead use #file arguments and MSBuild response files.
I just joined a team that has no CI process in place (not even an overnight build) and some sketchy development practices. There's desire to change that, so I've now been tasked with creating an overnight build. I've followed along with this series of articles to: create a master solution that contains all our projects (some web apps, a web service, some Windows services, and couple off tools that compile to command line executables); created an MSBuild script to automatically build, package, and deploy our products; and created a .cmd file to do it all in one click. Here's a task that I'm trying to accomplish now as part of all this:
The team currently has a practice of keeping the web.config and app.config files outside of source control, and to put into source control files called web.template.config and app.template.config. The intention is that the developer will copy the .template.config file to .config in order to get all of the standard configuration values, and then be able to edit the values in the .config file to whatever he needs for local development/testing. For obvious reasons, I would like to automate the process of renaming the .template.config file to .config. What would be the best way to do this?
Is it possible to do this in the build script itself, without having to stipulate within the script every individual file that needs to be renamed (which would require maintenance to the script any time a new project is added to the solution)? Or might I have to write some batch file that I simply run from the script?
Furthermore, is there a better development solution that I can suggest that will make this entire process unnecessary?
After a lot of reading about Item Groups, Targets, and the Copy task, I've figured out how to do what I need.
<ItemGroup>
<FilesToCopy Include="..\**\app.template.config">
<NewFilename>app.config</NewFilename>
</FilesToCopy>
<FilesToCopy Include="..\**\web.template.config">
<NewFilename>web.config</NewFilename>
</FilesToCopy>
<FilesToCopy Include"..\Hibernate\hibernate.cfg.template.xml">
<NewFilename>hibernate.cfg.xml</NewFilename>
</FilesToCopy>
</ItemGroup>
<Target Name="CopyFiles"
Inputs="#(FilesToCopy)"
Outputs="#(FilesToCopy->'%(RootDir)%(Directory)%(NewFilename)')">
<Message Text="Copying *.template.config files to *.config"/>
<Copy SourceFiles="#(FilesToCopy)"
DestinationFiles="#(FilesToCopy->'%(RootDir)%(Directory)%(NewFilename)')"/>
I create an item group that contains the files that I want to copy. The ** operator tells it to recurse through the entire directory tree to find every file with the specified name. I then add a piece of metadata to each of those files called "NewFilename". This is what I will be renaming each file to.
This snippet adds every file in the directory structure named app.template.config and specifies that I will be naming the new file app.config:
<FilesToCopy Include="..\**\app.template.config">
<NewFilename>app.config</NewFilename>
</FilesToCopy>
I then create a target to copy all of the files. This target was initially very simple, only calling the Copy task in order to always copy and overwrite the files. I pass the FilesToCopy item group as the source of the copy operation. I use transforms in order to specify the output filenames, as well as my NewFilename metadata and the well-known item metadata.
The following snippet will e.g. transform the file c:\Project\Subdir\app.template.config to c:\Project\Subdir\app.config and copy the former to the latter:
<Target Name="CopyFiles">
<Copy SourceFiles="#(FilesToCopy)"
DestinationFiles="#(FilesToCopy->'%(RootDir)%(Directory)%(NewFileName)')"/>
</Target>
But then I noticed that a developer might not appreciate having his customized web.config file being over-written every time the script is run. However, the developer probably should get his local file over-written if the repository's web.template.config has been modified, and now has new values in it that the code needs. I tried doing this a number of different ways--setting the Copy attribute "SkipUnchangedFiles" to true, using the "Exist()" function--to no avail.
The solution to this was building incrementally. This ensures that files will only be over-written if the app.template.config is newer. I pass the names of the files as the target input, and I specify the new file names as the target output:
<Target Name="CopyFiles"
Input="#(FilesToCopy)"
Output="#(FilesToCopy->'%(RootDir)%(Directory)%(NewFileName)')">
...
</Target>
This has the target check to see if the current output is up-to-date with respect to the input. If it isn't, i.e. the particular .template.config file has more recent changes than its corresponding .config file, then it will copy the web.template.config over the existing web.config. Otherwise, it will leave the developer's web.config file alone and unmodified. If none of the specified files needs to be copied, then the target is skipped altogether. Immediately after a clean repository clone, every file will be copied.
The above turned out be a satisfying solution, as I've only started using MSBuild and I'm surprised by its powerful capabilities. The only thing I don't like about it is that I had to repeat the exact same transform in two places. I hate duplicating any kind of code, but I couldn't figure out how to avoid this. If anyone has a tip, it'd be greatly appreciated. Also, while I think the development practice that necessitates this totally sucks, this does help in mitigating that suck factor.
Short answer:
Yes, you can (and should) automate this. You should be able to use MSBuild Move task to rename files.
Long answer:
It is great that there is a desire to change from a manual process to an automatic one. There are usually very few real reasons not to automate. Your build script will act as living documentation of how build and deployment actually works. In my humble opinion, a good build script is worth a lot more than static documentation (although I am not saying you should not have documentation - they are not mutually exclusive after all). Let's address your questions individually.
What would be the best way to do this?
I don't have a full understanding of what configuration you are storing in those files, but I suspect a lot of that configuration can be shared across the development team.
I would suggest raising the following questions:
Which of the settings are developer-specific?
Is there any way to standardise local developer machines so that settings could be shared?
Is it possible to do this in the build script itself, without having to stipulate within the script every individual file that needs to be renamed?
Yes, have a look at MSBuild Move task. You should be able to use it to rename files.
...which would require maintenance to the script any time a new project is added to the solution?
This is inevitable - your build scripts must evolve together with your solution. Accept this as a fact and include in your estimates time to make changes to your build scripts.
Furthermore, is there a better development solution that I can suggest that will make this entire process unnecessary?
I am not aware of all the requirements, so it is hard to recommend something very specific. I can say suggest this:
Create a shared build script for your solution
Automate manual tasks as much as possible (within reason)
If you are struggling to automate something - it could be an indicator of an area that needs to be rethought/redesigned
Make sure your team mates understand how the build works and are able to make changes to it themselves - don't "own" the build and become a bottleneck
Bear in mind that going from no build script to full automation is not an overnight process. Be patient and first focus on automating areas that are causing the most pain.
If I have misinterpreted any of your questions, please let me know and I will update the answer.
I'm trying to improve build times using CruiseControl.NET and MSBUILD, and one of the commandline switches, maxcpucount can be used to allow the build occur in parallel. Our solution has 60+ projects so any improvement would be helpful. However, whenever I up the maxcpucount above one, we have frequent build failures due to:
"The process cannot access the file xxxx because it is being used by
another process. msbuild"
It appears that the additional parallel build threads/processes are locking each other.
I think I found a solution. It appears that if I add the /nodeReuse:false switch I don't get the file locks. It seems like the nodeReuse functionality is keeping msbuild processes around and those are hanging on to file locks for subsequent builds.
http://msdn.microsoft.com/en-us/library/ms164311.aspx
Are you building from a solution file? If so, make sure that you are using direct project-to-project references and not using the Solution's project-dependency feature. If you happen to be using a bit of both, there can be issues. See this article.
Better yet, if at all possible, ditch the solution file and create your own MSBuild file to drive your build.
Your assembly is probably being used by another assembly thats being built. Make sure each assembly gets built before it's needed by other assemblies
So I have a CommonAssemblyInfo.cs linked into all the projects in my solution and is dynamically generated by my rake/albacore scripts which is not checked into source control.
I also have a CommonAssemblyInfo.cs.local for use when there is no ruby available, mainly to be used by devs.
Is it possible to have a msbuild task or something that runs before any of the other project compilation that will copy CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs before trying to compile my solution? I hate having to have a command you have to just know about and type in order to open and buidl the solution in Visual Studio.
UPDATE
So I ended up using a batch file as a solution wide pre-build event as described here: Solution-wide pre-build event?, it checks to see if CommonAssemblyInfo.cs exists and if not copies CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs just using a simple batch file.
This is the solution I ended up with.
I have each project in the solution link to a CommonAssemblyInfo.cs which is automagically generated for me by my build scripts (rake + albacore).
Since I cannot check CommonAssemblyInfo.cs into source control, I create a CommonAssemblyInfo.cs.local.
Simple solution: create go.bat which copies CommonAssemblyInfo.cs.local to CommonAssemblyInfo.cs that devs must run the first time they check out the project before opening the solution in VS.
For purely political reasons, if I did this people would have had hissy fits about me doing "nonstandard" things. Complex solution follows:
I created a project in the solution called PreBuild which every project in the solution depends on. This forces the build order to be such that this project is built first. This project contains a pre-build event which calls the following batch file:
echo verifying CommonVersionInfo.cs exists
cd
IF NOT EXIST ..\..\..\CommonAssemblyInfo.cs COPY ..\..\..\CommonAssemblyInfo.cs.local ..\..\..\CommonAssemblyInfo.cs
So now any developers who choose to keep their heads in the sand may checkout the project and blissfully open it up in VS unaware that any build scripts exist at all.
Are you talking about compilation in the VS IDE, or compilation through team build? If you are talking about team build, then you can use the "AfterGet" event as a place to use the standard "copy" msbuild task. If you are talking about the VS IDE, then you can still use the "copy" msbuild task.
I'm trying to work with an existing home grown implementation of click-once. Currently we manually update the manifest for assemblies that we actually changed. I'm attempting to make it automatic based on a binary comparison of the existing assemblies and the newly built assemblies. Unfortunately, it seems that each time I run clean + build (automated build script) there are small differences to the assemblies, essentially invalidating the use of our click-once solution at all. I'm guessing that these differences are caused by some sort of guid generation or something along those lines. Is there anyway to prevent the differences in the assemblies?
And unfortunately, due to our branching/CI strategy I don't have the option of not cleaning because each release is from a new branch.
Otherwise, any suggestions on how I can compare two assemblies to see if any code has changed, without having access to the source code.
Thanks,
David
Typically, autobuild systems check the filesystem timestamps of the binary vs the source files (or object files vs source files, depending on the language). If the source is newer than the binary/object, a rebuild is triggered. This strategy may work better for you instead of actually diffing binaries/
I found BitDiffer a tool from www.BitWidgets.com that compares what has changed in an assembly. While this runs slower than a binary comparison, it removes the need to have MSBuild create an identical assembly.
Thanks,
David