At my current client, we are using CruiseControl.Net for continuous integration. We are building website projects using MSBuild.
The issue that I have run up against is the following: We want to move all code and associated DLLs with every release. However, we also have (sometimes) hundreds of megs of files and images that are unchanged. In the CC.Net world, I will probably split these in to two different builds, one for a site and the other for static content.
What I would like to know is if anyone knows of an MSBuild or CCNet plugin that will allow me to do a binary diff on my static content? I'd like to be able to know if an image has chnaged or been added and then package only the changed/added content up and push them to a QA environment.
I did that with svn logs and MsBuild tasks.
Every time I Deploy the website, I save somewhere the revision number (basically in a flat file).
Then at the next Build-Deploy, I will do a svn log to see the diff between the latest release and now. Then my MsBuild Task takes the svn log xml output, parses it and outputs in another file the interesting static files added/modified/deleted.
Then the usual tasks come in and process only the modified files instead of every file.
That was a huge piece of work (above all with internal logic : our "static" contents are dynamically generated for each environment, each partner and each theme) but it was worth it.
Either way, you can try using a ChecksumCopy, but I don't know if it's less expensive...
Related
I have an automated build process set up to run from a build definition in TFS, which publishes a web application and generates/executes a database project script successfully via publish profiles that are passed as msbuild arguments in the build process definition. Everything is now running as expected except that several unnecessary files are being deployed to the target folder, including the .csproj file, all of the config transforms, and the properties folder which contains all of my publish profiles.
This is strange because 1. It's definitely not including ALL files/folders and mostly appears to be including ones used by the publish profile like transforms, while applying the transform correctly and excluding any explicitly excluded file (as defined in the pubxml), and 2. The process works perfectly if I do it by publishing from the project in Visual Studio 2013. I have the profile configured to only include files needed by the application, and I've confirmed in the csproj file that this property is there.
I tried excluding the properties folder from deployment in the pubxml file, but this causes the build to crash because it can't find the assembly file. What I've gathered is that the process is keeping all files it needs to complete the build, and dropping all of those files in my destination folder. FWIW, I'm using the "file system" method and I'm not sure yet if web deploy will make a difference. I haven't been able yet to connect to the target server with web deploy, but that's a separate problem to solve. Is there something in the build that I can configure so that my destination folder has only the files it needs to run the application, and not the files needed to BUILD the application?
FYI I also have not been using a drop folder, I'm not sure if that makes a difference or not but that might be the only thing I haven't tested as it doesn't seem necessary since I'm using a publishprofile and don't want to use the default tfs build configuration.
I found a solution that works well enough, after reading this: http://www.asp.net/web-forms/tutorials/deployment/advanced-enterprise-web-deployment/excluding-files-and-folders-from-deployment
This was a little uglier solution than I wanted, since it requires hard-coding the names of excluded files, but it does the trick and only requires identifying the files and folders in one location instead of altering a publish profile for each target environment. I created a wpp.targets file and used the ExcludeFromPackageFolders and ExcludeFromPackageFiles elements to identify the extra files. Ironically, if I don't also name the wpp.targets file in the exclude element, THAT file is included in my package. It's possible MSDeploy doesn't have the same issues with TFS as filesystem, but after spending half a day trying to work through a different set of issues and permissions workarounds, we decided that file system is a cleaner publishing method.
Please don't be too harsh, because I do not grasp this entirely correctly still, but msbuild/msdeploy is giving me some headaches lately.
Hopefully someone can provide a textual aspirin of some kind? So here is what I want to do:
I have a web application project, that has multiple configurations, thus multiple web.config-transforms.
I would like to deploy this project from command line.
I would rather not want to modify its project file. (I want to be able to do this for several web applications so as least as editing as possible is much appreciated)
I would like to be able to build it only once and then deploy the different configurations from it.
So far I deployed from command line using something like this:
msbuild D:\pathToFile\DeployVariation01.csproj
/p:Configuration=Debug;
Platform=AnyCpu;
DeployOnBuild=true;
DeployTarget=MSDeployPublish;
MSDeployServiceURL="localhost";
DeployIisAppPath="DeployApp/DeployThis01";
MSDeployPublishMethod=InProc
And this performs just what I want, except it only deploys the "Debug"-Configuration.
How can I, with minimal adjustments, make it deploy my other configurations as well?
I was thinking maybe I could build a package that includes all my configurations and then deploy from that and decide "while deploying" which configuration to deploy?
Unfortuanetly I am pretty much stuck here, the approaches I have read about all seem to require some modifications to project files, is there a way around that?
UPDATE:
I am still not really where I want to be here :).
But I looked into this PackageWeb-approach (also interesting video about that here) and it seems pretty nice; I can now build a package that includes all my transforms and then deploy from that as often as I want into multiple configurations.
One thing that I dislike about this is that I have to store my password in plain text into the generated parameters file for the powershell script, does someone know a way around this, I really would rather have that being an encrypted password.
Also other approaches to solve my original problem are still appreciated.
I am working on the same problem and am taking two paths using Microsoft Web Deploy or MSDeploy which is now in version 3.0.
I first compile the project using MSBUILD using the Package target passing in system.configuration, system.packagelocation. The Package Target generates a set of package files including a {PackageName}.SetParameters.xml file. The SetParameters.xml file by default allows on-publish changes to ConnectionStrings without recompiling when using msdeploy.exe to publish the file. The publish transformation process can also be customized by adding a parameters.xml file to the process defining additional parameterized web.config settings which can be changed at deploy time.
After the initial build I use the {PackageName}.deploy.cmd file generated by MSBUILD during the Package process to deploy the package to the target website. The Package process essentially duplicates the process you are currently doing from MSBUILD in that I can publish one Build-Configuration web.config transform from one compile. The process provides a consistent deployment process that can target remote servers from a central CI environment, which is great from a purely deployment process. The PackageBuild/Deploy process is parameterized within TeamCity, requiring changes to only a few parameters to setup a new deployment.
Like you, I cannot, however, compile a single version of code and deploy to multiple servers using the process as it exists today - which is my current focus. I want to parameterize the transform in a Continuous Deployment, build-once-deploy-many pattern to Dev, QA, User Testing, Staging, and Production.
I anticipate using one of two methods:
Create a Parameters.xml file for each project defining the variable deployment parameters along with a custom {ServerName}.SetParameters.xml for each target deployment, both to be used in conjunction with msdeploy.exe.
a. I am not sure defining a parameters.xml is a flexible enough process for my needs as the current project inserts and removes a variable number of web.config settings. Implementing a parameters file incorporating all of the variables could be too complex for my taste. I would also end up creating all of the target transformations, instead of the current developers initiated process. Not ideal.
I am following up on very recent updates to VS2012 Web Tools 2012.2 which allow tying a web.config transform to the publish profiles (profile.pubxml) now stored under SolutionName/Properties/PublishProfiles in VS2012.
VS2012 release 2012.2 adds the capability to create a second transform tied to the publish profile. The resulting transform process first runs the build configuration transformation, followed by the publish transformation, i.e. Release Transform followed by TargetServer Transform. Sayed Hashimi has a great YouTube video demonstrating the entire process using MSBUILD.
What is not entirely clear is whether the second transform is supported separately from the build using MSDeploy in a Continuous Deployment, build-once-deploy-many Pattern, or if the publish transformation is only supported during a separate Package/Build for each target transformation.
Option 1 will definitely work for some environments and was my first plan for tackling a Continuous Deployment process. I would much rather use Web Transforms to accomplish the process if possible.
An outside third possibility is using one of several CodePlex commandline projects that are capable of transforming web.config using the XDT transform engine. Unfortunately, using these tools would mean splicing the results into the Build/Package MSBUILD process in order to get the resulting web.config transformation into the deployment package - something I've not yet been successful in accomplishing. Sayed Hashimi also has a PackageWeb project from 2012 that might work as well. I am hoping his more recent work replaces the need for the extra steps involved in the packageweb solution.
Let me know if you decide on a solution - as I am definitely interested.
Our solution has got a ton of projects in it, and our MSBuild file is becoming a gigantic, unstable morass of angle brackets that has gotten so large that I've begun to be scared to touch it. However, I noticed that most of our projects fall into two groups.
There are web projects, which have a .csproj extension and are built and deployed using web deployment projects to dump them into a directory. There is also a configuration transform process. Basically, to deploy a web project, one only needs the project name, the output directory, and the config transforms.
Similarly, we have an ever-increasing number of service apps that will live under TopShelf. These are basically dlls that are built from .csproj files. Like the above websites, they undergo a config transform, but they are copied to the output directory instead of going through the web deployment steps.
It occurred to me that it would be really slick if I could simply provide the MSBuild project with a couple of text files (one for the webs and the other for the services) that MSBuild could then use to dynamically compile all the various projects. I was thinking that if I could just feed it a comma-delimited list of projects, output locations, and config replacement files that there might be a way to get msbuild to read them in, iterate over them, and dump out the projects where they are supposed to go.
However, my MSBuild-fu is weak. How do I even get started on this?
You can start here, this is part 2.
For config files I'm using XmlPreprocess tool for config files manipulation. It is using one mapping file for multiple environments. You can edit mapping file by Excel. It is very easy to use.
I am using MSbuild to create a deployment package (simply copying various files from the projects in my solution to different folders) I would like the root folder to be of the format
DeploymentPackage2.3.4.5ForRelease
How can I get MSbuild to put the Assembly number in the folder name automatically?
EDIT:
The solution has a great deal of projects in it (too many really) they all get their version number from a SharedAssemblyInfo.cs file that is manually updated but in the fullness of time will pick up the svn build number (but that is a job for later)
I am building using an external .bat file that calls a custom written .targets/.proj setup that simply calls msbuild on the .sln of the solution.
The 'create package' step I am trying to create happens after a succesful build and will eventually be run by our CI framework, however I would like to be able to run it locally too.
I have created a "CreatePackage" target that does the copying that I want, however it is currently into a fixed folder. I need the folder name to reflect the AssemblyVersion of one of the final dll's.
If there is a better way then I want to know about it... but I am going to use this I think
MSBuild Task to read version of dll
Ok, so I've got a somewhat complicated problem with my build environment that I'm trying to deal with.
I have a solution file that contains multiple C# projects which is built by a NAnt script calling MSBuild - passing MSBuild the name of the solution file and a path to copy the binaries to. This is because I want my automated build environment (CruiseControl.Net) to create a folder named after the revision of each build - this way I can easily go back to previous binaries for any reason.
So idealy I have a folder layout like this
c:\build\nightly\rev1
c:\build\nightly\rev2
c:\build\nightly\rev3
...
c:\build\nightly\rev10
etc.
The problem that's arisen is I recently added the latest version of the Unity IoC container to my project, checking it directly out of MS's online SVN repository. What's happening is I have a Silverlight 3 project that references the Silverlight version of Unity but I also have other projects (namely my Unit testing project) that reference the standard (non-Silverlight) version of Unity.
So what happens is since MSBuild is dumping everything into one single folder the Silverlight version of the Unity assembly is overwriting the non-Silverlight version because they have the exact same assembly file name.
Then when CruistControl runs my unit tests they fail because they don't have the proper dependencies available anymore (they try to load the Silverlight specific Unity assembly which obviously doesn't work).
So what I want to do is:
keep my desired output directory
structure (folder\revision)
I don't want to have to manually edit
every single proj file I have as this
is error prone when adding new
projects to the solution
Idealy I would like MSBuild to put everything into a folder structure similar to this:
nightly\revision1\project1
nightly\revision1\project2
nightly\revision1\project3
...
nightly\revision2\project1
nightly\revision2\project2
nightly\revision2\project3
etc
I can't modify the Unity project to give it a different file name because it comes from another SVN repository I cannot commit changes to. I found a similar question posted here and the suggested solution was to use a "master" MSBuild file that used a custom task to extract all the project file names out of the solution then loop over each one building them. I tried that but it doesn't build them in the order of their dependencies, so it fails for my project.
Help?
Firstly I would always have the build server delete the old working copy and check out a fresh copy to avoid any problems with stale artifacts from the previous build.
Next I would have nant or msbuild build the solutions as before with the artifacts from each build going to their local working output folders.
After that I'd move the artifacts from their working paths to their output paths, this shouldn't require digging through the project files since you can just tell msbuild/nant to copy working\project1\bin\release\**\*.* to artifacts\project1\.
The script that does this should ideally be stored along with the source with the main file, e.g. build.nant or build.proj in top level of the trunk.
For third party libraries I would simple include the DLLs directory in your repository. Nothing worse than writing some code and having a third party dependency break your build because of changes on their end.
Simply document the versions of the libraries you are using, and if you must update them, you'll have a better sense of what breaks the build before you even check it in.
Also, doesn't CC.Net automatically handle the providing of releases based on revision? I'm using TeamCity and it keeps a copy of the artifacts of every build.
I highly recommend reading JP Boodhoo's Automating Builds with NAnt blog series. That's been my starting point and have made lots of changes for my own taste. I also highly recommend checking out the builds of many open sources projects for examples. I've learned a lot from the builds of the Castle/Nhibernate/Rhino-Tools stack.