I come from a Continuous Integration and Continuous Delivery (CI-CD) implementation project background for java web applications. Now i am working for a .NET based project. Microsoft technologies is completely new to me. It is using the MsBuild for the build process via Jenkins. I am learning MsBuild at this time. The more i read, the more confused i am with the Microsoft way of doing this.
I noticed that the msbuild is executed for every environment where the app is going to be deployed using various configuration and profiles based on the environment for the deployment. Below are some msbuild commands for 2 different environments (PIE & TEST)
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=PIE /m:4 /nr:false src/myapp.sln
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=TEST /t:Rebuild /m:2 /p:DeployOnBuild=true /p:PublishProfile=TEST src/myapp.sln
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=STAGE /t:Rebuild /p:DeployOnBuild=true /p:PublishProfile=STAGE /m:2 SprintA/src/myapp.sln
i may be wrong, but i feel that the code being deployed to the two environments (when the code progress from PIE to TEST) is being build for each environment which is not the real code progression concept. IMHO, the build is done once and its progressed to subsequent environments for testing/validation as long as there are no bugs in the code. The various environment specific settings are handled via config files inside the package and the containers (tomcat for a java app) are started with the parameters that reads/parse the confif files.
Is there a way to handle this in .NET? The app is deployed in IIS
UPDATE:
The more i do research reading various docs and blogs, i came across the web publishing method using msbuild for each configuration and the deploy/publish profiles. IS this just the standard way that the mass follows for a .net project's CICD?
Yes, this is something Microsoft realized and is enforced using the new Release System.
Basically you have a "process" (Build) building your code and producing artifacts (ie. website file structure, nuget package, installers, etc) in this process you typically take care of things like applying the version value to your assemblies, minifiying js and css files or anything not related with the any specific environment.
Then you have another "process" (Release) to configure your artifacts based on the environment where they will be deployed (ie. modifying web.config files from your website) and deploy those artifacts to the desired environment without having to build them again. (ie. push nuget package to some pre-production nuget feed, copy you website structure to the server, etc)
How do you implement these two "processes"?. Well, that depends on your preference and tools. If you use Visual Studio Team Services you have these processes clearly defined out of the box by the infrastructure, and a lot of built in task to support them.
I have not worked with Jenkins but as long as you can use msbuild you could have 2 msbuild projects one to build your artifacts from the source code on different branches and another one to deploy to different environments base on some configurations (ei. your PIE & TEST) and of course you could use tools like powershell or MSbuild custom tasks to support more advanced scenarios within your processes.
Related
Some original info was changed to make the post more focused on the real issue after it was found.
These are some of the details of the current environment. I listed these only because questions were raised in other posts to determine what was and was not working in the current environment:
Upon check-in TFS 2017 successfully builds a web project on the build agent.
A VS 2017 publish profile can manually transform the project properly
The build machine artifact location includes both the transform and profile files
The artifact location is shown below:
I have researched this in depth on Microsoft's VS site, SO and other forums, but there are so many different answers, many of them for older versions, I have been unable to piece this together. As a result I have several sub-questions.
1) Can transforms be engaged in both Builds and Releases?. I read that transforms are applied during the publish process, not the build process, and that made me wonder if it is even possible to do this during a Build. But then when I was exploring releases, I saw all the same tasks usable in a Build, which suggests I can publish with a transform in either Build or Release. Is that correct?
2) Does TFS 2017 require a lot of special handling to engage a transform file? Some of the posts instructed the editing of the .proj file. I wanted to get a confirmation before doing that kind of detailed manipulation, especially given the improvements in TFS 2017.
The following information is the state of the current build definition named "confPanner-CI". The shaded PS script was successfully used to upload to the hosting location to test the whole process, but that is not adequate for the task at hand which requires transforms to be applied:
The full MSBuild Arguments which also created a temp location for the powershell script are:
/p:DeployOnBuild=True /p:DeployDefaultTarget=WebPublish /p:WebPublishMethod=FileSystem /p:DeleteExistingFiles=True /p:publishUrl=c:\ConfPlnrWeb
If I were to add a task for publishing I saw the Publish Build Artifacts task:
But none of the settings as shown below seem to relate to transforms:
The bottom line question is: How do I configure the build so the web project upload has the proper web transform applied?
Update: The following added after the answer below led to at least one place where VS transforms can be applied during a build, and presumably also a release.
Inside the MSBuild Build solution task set the Configuration as shown below:
Publish Build Artifacts task is used to publish the related artifacts ( The “a” working directory contains the artifacts (also known as the “drop”) that are uploaded at the end of the build) to Visual Studio Team Services/TFS or a file share.
Usually it should be a package and be used in a deploy task such as Deploy: WinRM - IIS Web App Deployment or Azure App Service Deployment to achieved the deployment.
1) Can transforms be engaged in both Builds and Releases?
Yes, you could also do this in a build pipeline with the useage of build deploy task. You need to add the task after the publish build artifacts task.
2) Does TFS 2017 require a lot of special handling to engage a transform file?
update
The BuildConfiguration variable is different in TFS 2017, it's inside
the MSBuild task! Transforms are now applied according
to the MSBuild task Configuration setting.
Edit the .proj file is a method to do the transform. If you don't need to change the transform, it will auto do it during the build.You could also use some 3-rd party task/extension for extra transform such as: XDT Transform
Usually we separate the build and release for the deployment, cause it's easy to configure multiple environments and easy to debug issue. You definitely could do this only in build but with a bloated process. You could refer this tutorial: Build and Deploy Azure Web Apps using Team Foundation Server/Services vNext Builds.
For a separate build and release solution, you could take a look at this blog: Using web.config transforms and Release Manager – TFS 2017/Team Services edition
We have with success been updating our development web site at a daily basis using msdeploy from TFS2010.
This was working fine until we upgraded to VS2012, our application from .NET Framework 4.0 to 4.5 and ASP.NET MVC from 3.0 to 4.0. It look like all is well and assemblies deployed but nothing has actually been deployed.
I have been looking into this for two days now and can't figure out why this is happening and now I am running out of ideas.
Below is part of my build script in the way it has been working before the upgrade.
<MSBuild
Projects="$(SolutionRoot)\My.Web\My.Web.csproj"
Properties="MvcBuildViews=False;AllowUntrustedCertificate=True;AuthType=Basic;Configuration=Dev;CreatePackageOnPublish=True;DeployIisAppPath=dev.myweb;DeployOnBuild=True;DeployTarget=MsDeployPublish;MSDeployPublishMethod=WMSvc;MsDeployServiceUrl=https://10.xxx.xxx.xxx:8172/MsDeploy.axd;UserName=UserName;Password=Password;UseMsdeployExe=True"
ContinueOnError="False"
/>
When the upgrade was initiated and my problem discovered we were using Web Deploy 2.0 but now we have upgraded to Web Deploy 3.0. I have also made sure we are building with ToolsVersion="4.0".
UPDATE --
msbuild.exe /p:AllowUntrustedCertificate=True
/p:AuthType=Basic
/p:Configuration=Dev
/p:CreatePackageOnPublish=True
/p:DeployIisAppPath=dev.myweb
/p:DeployOnBuild=True
/p:DeployTarget=MsDeployPublish
/p:MSDeployPublishMethod=WMSvc
/p:MsDeployServiceUrl=https://10.xxx.xxx.xxx:8172/MsDeploy.axd
/p:UserName=UserName
/p:Password=Password
/p:UseMsdeployExe=True
E:\Builds\1\WhatEver\Daily_Build\Sources\My.Web\My.Web.csproj
Now I also tried to run the above msbuild command from our TFS and no response which frustrates me completely. Nothing in the event log of TFS, nothing in log file no matter verbosity... Any ideas?
It does work using msdeploy directy like below;
<Exec Command=""C:\Program Files\IIS\Microsoft Web Deploy V3\MSDeploy.exe" -verb:sync -source:contentPath="E:\Builds\1\WhatEver\Daily_Build\Sources\My.Web\My.Web.csproj" -dest:contentPath="E:\dev.my.web",computername=https://10.xxx.xxx.xxx:8172/MsDeploy.axd,username=UserName,password=Password,authtype=Basic -allowUntrusted=True"
ContinueOnError="false" />
--
UPDATE 2 --
It appears Microsoft added a check for what type of projects that are publishable projects and our web application are not, since the Output Type is Class Library. This has been valid with v4.0 but apparently not for v4.5.
Anyone have an idea of what to do make it work again? Do I need to change the project type? Create publishing package up front and then deploy that? Or what?
--
Anyone else that has had the same problem? Have you found a solution to share?
Could there be an issue with version of MSBuild?
Here is what I would recommend. In VS2012 we have made it easy to automate publishing your web projects using the publish profiles which are created by the publish dialog. In your case create a new MSDeploy profile. When you create that profile we will save the settings into a file under Properties\PublishProfiles (or My Project\PublishProfiles for VB). The extension of this file will be .pubxml. Those files are actually MSBuild files, which you can customize if needed. You can continue to use the publish dialog as well. The password will be stored in a .user file and encrypted such that only you can decrypt it.
After you have created that profile you can publish with the command below if you are building the .sln file.
msbuild mysoln.sln /p:DeployOnBuild=true /p:PublishProfile=<ProfileName> /p:Password=<Password>
If you are building the .csproj/.vbproj then you need to tweak this a bit in the following way
msbuild mysoln.sln /p:DeployOnBuild=true /p:PublishProfile=<ProfileName> /p:Password=<Password> /p:VisualStudioVersion=11.0
More on why VisualStudioVersion is required at http://sedodream.com/2012/08/19/VisualStudioProjectCompatabilityAndVisualStudioVersion.aspx.
Once you do this you will be able to build+publish just like you did previously. FYI we have shipped all these new web publish features for VS2010 in the Azure SDK https://www.windowsazure.com/en-us/develop/net/#.
Also in your question I noticed that you are specifying some custom properties, like MvcBuildViews. You can now place those properties directly inside the publish profile (the .pubxml file) if you want. Of course you can still pass them in on the command line if that makes more sense for your scenario.
More info on this at http://sedodream.com/2012/06/15/VisualStudio2010WebPublishUpdates.aspx.
If you take a look at the approach that we had for developers to automate publishing it was to specify properties and targets to be executed during the build. The problem with this approach is that this limits our ability to enhance the web publish experience. In the new release we have introduced an abstraction, the publish profile, which allows us to change the underlying targets of the web publish pipeline and your automation scripts will continue to run. Hopefully from this point forward you will not have to re-visit this issue.
I had much the same problem today. I too was trying to get a .NET 4.5 web application automatically deployed using a machine that did not have Visual Studio 2012 installed on it. There were a couple of minor differences in my situation, however: I was using TeamCity instead of TFS, and our solution was created with .NET 4.5 as opposed to being one that had been upgraded from .NET 4.0.
Nonetheless, I did have the same problem described. I'd use MSBuild to build the web app and deploy it to IIS, in much the same way. This approach worked fine on my dev machine. However, when I ran MSBuild on the CI server, it quite happily built the web app, but it stopped after that: no errors, no warnings, nothing, just a message that the build was successful. There wasn't the slightest hint of an attempt at deploying the app to IIS.
It seems MSBuild was missing the relevant targets to perform the web deployment.
The fix was to copy the folder C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web from my dev machine to the CI server, copying it to the same place on the CI server as it was on my machine.
Once I did that, MSBuild then grumbled about needing Web Deploy 3.0, but that was fixed easily enough. After installing that on the CI server too, MSBuild quite happily deployed the web app.
To extend Luke Woodward's answer:
I, too, found that deploying C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\ from my local machine to the build server was the fix.
However, the real fix is to install the Microsoft Web Developer Tools as part of the VS 2012 installation, which will create this folder, among other things. This addresses Ieppie's licensing objection.
I tested this by...
Deleting C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\
Running the VS 2012 installer and adding MS Web Dev tools.
Verifying that, after the install, C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\ was back.
Our main website is a collection of 10 separate ASP.NET projects and applications. At the moment, to do a complete deployment onto a fresh server involves running ten separate msdeploy jobs; each application is built, configured (using config transforms) and packaged, but we don't have any solution for deploying all the packages as a single operation.
I can see several possibilities that might work in this scenario, but would love to hear from anybody who has succeeded - or failed - in setting up something similar:
A folder full of packages and deploy.cmd scripts, with a "master script" that will call each individual app script in turn and deploy that app to the target server.
Using a staging server where we deploy the latest build of each package from TeamCity using the production configuration, but then use msdeploy to capture that server into a single enormous msdeploy ZIP package, which is then deployed onto each production server as a single msdeploy step.
Creating a single, enormous Visual Studio solution that references EVERY project in our codebase (perhaps via svn:externals?), compiles and cross-references them ALL, and hence supports using a single msbuild job to create a huge monolithic package containing our entire codebase, built from the latest revision in source control and configured for the target environment.
I've studied Troy Hunt's excellent "You're Deploying it Wrong" series, and Scott Hanselman's "Web Deployment Made Awesome" article, but I think I'm looking for something a step beyond either of these approaches that incorporates multiple projects and applications without necessarily building them from source in a single step - any ideas?
We had a very similar scenario in our company, and we created an installation package using WIX. Our config transform happens at installation time, so now we create a single build, then deploy that to each server via an MSI install package. WIX is very flexible, but also has a steep learning curve. We modify our configs using our own custom action, but it could be done other ways.
We use Team Foundation Server and MSBuild to do our builds. This is pretty straight forward, but did take some work to set up correctly with as many projects and solutions as we had.
Other options we looked into, and even tried were:
InstallShield - Not flexible enough.
Writing our own C# Install - WIX already thought of everything we
were trying to accomplish so why reinvent the wheel?
Just saying to heck with it all and installing things manually - 2 or
3 months of development time in WIX and MSBuild have easily paid for
the hours we would have spent of the last year doing things manually.
I think the deployment tools built into Visual Studio were designed for a single application with just a few deployments. It sounds like you need external tools, and development effort, to get your deployments quicker, and eliminate the need for doing things manually. That's why we invested in the above solution, and it has really paid off.
I'll pick Installshield.
Installshield latest versions support creating webdeploy packages.
You can define the IIS configurations for all apps in a single project and create releases if you want to create packages by separate or one single release for all web apps.
Installshield project has an object model where you can automate basically every task from build scripts, also the projects are simple xml files that you can also modify in automation scripts if required
Developers can modify update WixXML projects by separate and you can add those projects builds as merge modules to your installshield projects through your build scripts with some little tweaks to the installshield project xml (at least in 2011 version, this part is not supported by installshield but can be done)
You don't even need to modify Visual Studio Projects for groups of web apps that follow a same pattern, neither manually modify your installshield project to add new web apps for these cases, you can create packages for new web apps without intervention setting one time your build scripts for the installshield project automation task based on the root VS build output
Here's what I want to do:
Build an existing .csproj that targets "Package"
Publish the package with MSDeploy to an IIS 6 server
This is for a TeamCity build and release that I'm trying to configure in a single step. I could create a custom build file but I'm trying to tackle this without adding any additional configurations to the app.
There are a lot of examples around of MSBuild parameters which can do this publishing via WMSVC - here's a great one - but that's not going to play ball with IIS 6. Are there equivalent params which can be used when there is a dependency on MsDepSvc? Is this even possible or am I left with either a custom build script or a package build followed by a publish build?
You can modify an existing .csproj file to add any additional target needed (it's just an MSBuild file) and publishing to an IIS6 server can be done via different MSDeploy providers (webServer60, metaKey, or contentPath via a share).
While this would be possible to do by adding a Target in the MSBuild of your project, I'd recommend that you split these two activities out into two separate targets. By splitting them into two separate targets you can still call them together msbuild /t:Package;Deploy but you can also call them independently.
This would allow you to create a deployment package and have TeamCity include it as an artifact of the build. You could then download this package from TeamCity and deploy it to any server independently, even if you deployed it automatically. If TeamCity also creates your release builds, you know have your production deployment
I've noticed projects such as Msbuild Extension Pack and MsBuild Community Tasks give msbuild the power to install assemblies, sql, and setup IIS. These features seem to be oriented to doing installs and not builds.
So I was wondering how many people out there are using msbuild, perhaps in conjunction with Cruise Control.Net to do installs on staging environments?
I use MsBuild to build, and part of the build process runs Wix to create an installer(MSI) which is used to deploy to production.
I wrote up a little sample of templating configurations for different target environments with msbuild: http://blog.privosoft.com/2010/10/msbuild-environment-sandboxing.html
We use CC.NET & MSBuild to build and then also to publish to our dev and stage environments, however we do not have the push live on CruiseControl.NET, we run that MSBuild by hand. We just thought it would be way to tempting with a button to publish live ;) It took probably 2 or 3 revisions to get our MSBuild set up right. But now everything is in one file, and everything is based on Targets and Properties to do all the work. About 6 months ago, was what should be the last update and that was a multi-server push so we are ready for scaling up. We can now push any combination of parts to any combination of servers. So if we want 5 database servers, 3 contenet servers, and 2 web servers we have that ability. No need to use anything else. MSBuild can do it.
I created a deployment system where a central coordinator can:
- identify the right target server for a given component (e.g Windows service goes to a given server, web services go to another, etc.)
- perform a PsExec of a deployment MSBuild script on the target server
- the deployment MSBuild script is responsible for:
a) downloading the right component package (in my case a .zip)
b) backing up previous versions of the component
c) extracting the package to the right place
d) tailoring the installation steps to the type of component to deploy (e.g. needs to perform an Exec task of installutil.exe on a Windows service )
e) logging the result of the deployment
This system is built using a mix of:
- core MSBuild tasks
- [Tigris MSBuild community tasks][1]
- [MS SDC tasks][2]
- and custom tasks
The system allows us to perform consistent deployment of complex apps across partitioned environments (e.g. DEV, QA, UAT, etc) made of virtual servers.
I use MSBuild to build a fairly large client/server application. I use InstallShield 2008 to create a separate client and server install set.
By adding a custom target into the build process you can combine the creation of the installers into the build.
I would recommend that you create and test the build and the installer separately, before attempting to integrate the two.
I know this is an old question... but...
I am currently using MSBuild with MSBuild Extension Pack (http://msbuildextensionpack.codeplex.com) to do my entire deployment. The database portion is handled with the VS database command-line tool (vsdbcmd.exe - http://msdn.microsoft.com/en-us/library/dd193283.aspx). That Extension Pack is pretty amazing, and is letting me build web sites, app pools, Windows services, update config, and much more.
I've also put Team City agents on the test servers, so I can deploy as part of a build chain (introduced in version 7 of Team City). And running my deploy MSBuild script is super easy from Team City.
I used to use MSBuild, now I'm using PowerShell. MSBuild is a build language. It is painful to script in. There is a lot I wanted to do in it that were difficult and sometimes impossible.
Over the past year, I've created an PowerShell module somewhat equivalent to MSBuild Extension Pack called Carbon.
I strongly, strongly encourage everyone out there to learn and use PowerShell.