Our main website is a collection of 10 separate ASP.NET projects and applications. At the moment, to do a complete deployment onto a fresh server involves running ten separate msdeploy jobs; each application is built, configured (using config transforms) and packaged, but we don't have any solution for deploying all the packages as a single operation.
I can see several possibilities that might work in this scenario, but would love to hear from anybody who has succeeded - or failed - in setting up something similar:
A folder full of packages and deploy.cmd scripts, with a "master script" that will call each individual app script in turn and deploy that app to the target server.
Using a staging server where we deploy the latest build of each package from TeamCity using the production configuration, but then use msdeploy to capture that server into a single enormous msdeploy ZIP package, which is then deployed onto each production server as a single msdeploy step.
Creating a single, enormous Visual Studio solution that references EVERY project in our codebase (perhaps via svn:externals?), compiles and cross-references them ALL, and hence supports using a single msbuild job to create a huge monolithic package containing our entire codebase, built from the latest revision in source control and configured for the target environment.
I've studied Troy Hunt's excellent "You're Deploying it Wrong" series, and Scott Hanselman's "Web Deployment Made Awesome" article, but I think I'm looking for something a step beyond either of these approaches that incorporates multiple projects and applications without necessarily building them from source in a single step - any ideas?
We had a very similar scenario in our company, and we created an installation package using WIX. Our config transform happens at installation time, so now we create a single build, then deploy that to each server via an MSI install package. WIX is very flexible, but also has a steep learning curve. We modify our configs using our own custom action, but it could be done other ways.
We use Team Foundation Server and MSBuild to do our builds. This is pretty straight forward, but did take some work to set up correctly with as many projects and solutions as we had.
Other options we looked into, and even tried were:
InstallShield - Not flexible enough.
Writing our own C# Install - WIX already thought of everything we
were trying to accomplish so why reinvent the wheel?
Just saying to heck with it all and installing things manually - 2 or
3 months of development time in WIX and MSBuild have easily paid for
the hours we would have spent of the last year doing things manually.
I think the deployment tools built into Visual Studio were designed for a single application with just a few deployments. It sounds like you need external tools, and development effort, to get your deployments quicker, and eliminate the need for doing things manually. That's why we invested in the above solution, and it has really paid off.
I'll pick Installshield.
Installshield latest versions support creating webdeploy packages.
You can define the IIS configurations for all apps in a single project and create releases if you want to create packages by separate or one single release for all web apps.
Installshield project has an object model where you can automate basically every task from build scripts, also the projects are simple xml files that you can also modify in automation scripts if required
Developers can modify update WixXML projects by separate and you can add those projects builds as merge modules to your installshield projects through your build scripts with some little tweaks to the installshield project xml (at least in 2011 version, this part is not supported by installshield but can be done)
You don't even need to modify Visual Studio Projects for groups of web apps that follow a same pattern, neither manually modify your installshield project to add new web apps for these cases, you can create packages for new web apps without intervention setting one time your build scripts for the installshield project automation task based on the root VS build output
Related
I currently have a custom XAML build workflow which performs a lot of validations based on code structure, folders, file content and even database configuration.
Currently we need to consider to migrate it to TFS 2017, and I'm trying to understand the possibilities, effort and pitfalls and this is where I could use your help.
I have logic built in C# and then I look at MSDN documentation the language seems to be Javascript, GitHub samples seem to be Typescript. Am I forced to use one of this languages? any preferred one or limitations that I should be aware of? Can for instance, I connect to a database directly? or it's better to have an API and just use REST calls?
Can I for instance, use .Net object which have already some wrapped logic?
I also noticed that on the manifest file I can declare inputs, but can I somehow use outputs? For example, I have a task that does validation and in the end I would have a task to send an e-mail with the result of that validation. Is this possible?
Thank you in advance.
You are right in concluding that XAML builds are deprecated. The new Task based system is much more flexible and now that we even have YAML support, you can move to Builds as Code.
The reason you're seeing Build Tasks in TypeScript (which is a superset of JavaScript) because of the cross platform nature of NodeJS. This allows your tasks to run on Windows, Linux and Mac. In the mean time we also got .NET Core support. You can also use PowerShell if that's more of your liking.
Regarding your 'can I connect to a database' question. Yes you can, if you want it is another question. This will massively slow your build down and will make your build fail if there are issues with database connectivity. I would suggest asking a second question here on Stackoverflow to describe your situation and discuss if there is a better alternative.
You can use output variables from your task and reuse these in the next tasks. That's all supported and no problem (see https://roadtoalm.com/2016/08/11/set-output-variable-in-a-powershell-vsts-build-task/ for an example).
There are too many questions here and they are broad.
Whatever, XAML Build are deprecated in TFS 2017 and will not be available in the next version of TFS. VSTS already no longer supports them. So you need to Migrate from XAML builds to new builds (vNext builds). Also this similar thread for your reference : Converting TFS XAML Build Custom Activities into TFS 2015 Build Tasks
Marketplace
While you can make use of custom build tasks with XAML build there is
no centralized or integrated solution for browsing available tasks,
installing them or managing them. With the new builds you can make use
of the quickly-growing Visual Studio Team Services Marketplace. Here,
you can search and browse the many extensions currently available in
the Marketplace.
See WHY SHOULD I LEAVE XAML BUILDS for details.
So, you can find the existing extensions in Marketplace and install them to extend the build process (add correspinding tasks). You can also custom your own extension based on your specific requirements. See Extensions overview.
You can build custom applications or services that integrate with your
Visual Studio Team Services (VSTS) and Team Foundation Server (TFS)
accounts by using the REST APIs to make direct HTTP calls, or
utilize our .NET Client Libraries.
Source here : Integrating with VSTS and Team Foundation Server
To add a custom task you can reference below articles:
Add a build task
Reference for creating custom build tasks within extensions
Developing a Custom Build vNext Task
For the "outputs" validation, you can try it with build alerts.
I come from a Continuous Integration and Continuous Delivery (CI-CD) implementation project background for java web applications. Now i am working for a .NET based project. Microsoft technologies is completely new to me. It is using the MsBuild for the build process via Jenkins. I am learning MsBuild at this time. The more i read, the more confused i am with the Microsoft way of doing this.
I noticed that the msbuild is executed for every environment where the app is going to be deployed using various configuration and profiles based on the environment for the deployment. Below are some msbuild commands for 2 different environments (PIE & TEST)
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=PIE /m:4 /nr:false src/myapp.sln
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=TEST /t:Rebuild /m:2 /p:DeployOnBuild=true /p:PublishProfile=TEST src/myapp.sln
C:\\Program Files (x86)\\MSBuild\\12.0\\Bin\\MSBuild.exe" /p:Configuration=STAGE /t:Rebuild /p:DeployOnBuild=true /p:PublishProfile=STAGE /m:2 SprintA/src/myapp.sln
i may be wrong, but i feel that the code being deployed to the two environments (when the code progress from PIE to TEST) is being build for each environment which is not the real code progression concept. IMHO, the build is done once and its progressed to subsequent environments for testing/validation as long as there are no bugs in the code. The various environment specific settings are handled via config files inside the package and the containers (tomcat for a java app) are started with the parameters that reads/parse the confif files.
Is there a way to handle this in .NET? The app is deployed in IIS
UPDATE:
The more i do research reading various docs and blogs, i came across the web publishing method using msbuild for each configuration and the deploy/publish profiles. IS this just the standard way that the mass follows for a .net project's CICD?
Yes, this is something Microsoft realized and is enforced using the new Release System.
Basically you have a "process" (Build) building your code and producing artifacts (ie. website file structure, nuget package, installers, etc) in this process you typically take care of things like applying the version value to your assemblies, minifiying js and css files or anything not related with the any specific environment.
Then you have another "process" (Release) to configure your artifacts based on the environment where they will be deployed (ie. modifying web.config files from your website) and deploy those artifacts to the desired environment without having to build them again. (ie. push nuget package to some pre-production nuget feed, copy you website structure to the server, etc)
How do you implement these two "processes"?. Well, that depends on your preference and tools. If you use Visual Studio Team Services you have these processes clearly defined out of the box by the infrastructure, and a lot of built in task to support them.
I have not worked with Jenkins but as long as you can use msbuild you could have 2 msbuild projects one to build your artifacts from the source code on different branches and another one to deploy to different environments base on some configurations (ei. your PIE & TEST) and of course you could use tools like powershell or MSbuild custom tasks to support more advanced scenarios within your processes.
I have create a sample ASP.NET 5 application (pretty much the example one from New Solution), and pushed it to GIT hosted on Visual Studio Team Services (former Visual Studio Online). I want to set up continuous integration to Azure Web App (former Azure Web Site). I have tried to set it up from Azure portal itself, it did create a new build definition, but it fails to build ASP.NET 5. I have found a guide how to do this, but it never really worked for me, I get errors like this e.g.
Error parsing solution file at C:\a\1\s\Frontend\src\Frontend\Frontend.xproj: Exception has been thrown by the target of an invocation.
Predefined type 'System.Void' is not defined or imported
Another problem is that it seems it really takes a lot of time to install dnvm, get packages, etc. So all in all it's a pain to make it work.
So are there real alternatives for that or more importantly is Microsoft is planning to implement something like a Build ASP.NET 5, Deploy ASP.NET to Azure and such to make it easy as I suppose it's easy with the current ASP.NET 4 apps. I really hope that it will be an option soon since it's quite impossible to work with current build system.
For "System.Void" issue, please check the runtime version in "global.json" file and make sure it is consistent with the dependencies in "project.json" file.
For dnvm install issue, since AspNet5 runtime environment isn't installed on VSTS Hosted Build Agent for now and the different users may use different runtime versions, it requires the user to add a "PreBuild" PowerShell step to read the runtime version in "global.json" file and then install it. If you can make sure that you will always only use one version (For example: 1.0.0-rc1-update1), you can deploy your own build agent and install "1.0.0-rc1-update1" on it, then you can skip the dnvm installation during the build process.
Take a look on http://riffer.eu/wordpress/?p=112. There I have a solution vor asp.net core RC_1.
Amazingly you need only two powershell scripts - there is no compiling / visual studio necessary.
I've noticed projects such as Msbuild Extension Pack and MsBuild Community Tasks give msbuild the power to install assemblies, sql, and setup IIS. These features seem to be oriented to doing installs and not builds.
So I was wondering how many people out there are using msbuild, perhaps in conjunction with Cruise Control.Net to do installs on staging environments?
I use MsBuild to build, and part of the build process runs Wix to create an installer(MSI) which is used to deploy to production.
I wrote up a little sample of templating configurations for different target environments with msbuild: http://blog.privosoft.com/2010/10/msbuild-environment-sandboxing.html
We use CC.NET & MSBuild to build and then also to publish to our dev and stage environments, however we do not have the push live on CruiseControl.NET, we run that MSBuild by hand. We just thought it would be way to tempting with a button to publish live ;) It took probably 2 or 3 revisions to get our MSBuild set up right. But now everything is in one file, and everything is based on Targets and Properties to do all the work. About 6 months ago, was what should be the last update and that was a multi-server push so we are ready for scaling up. We can now push any combination of parts to any combination of servers. So if we want 5 database servers, 3 contenet servers, and 2 web servers we have that ability. No need to use anything else. MSBuild can do it.
I created a deployment system where a central coordinator can:
- identify the right target server for a given component (e.g Windows service goes to a given server, web services go to another, etc.)
- perform a PsExec of a deployment MSBuild script on the target server
- the deployment MSBuild script is responsible for:
a) downloading the right component package (in my case a .zip)
b) backing up previous versions of the component
c) extracting the package to the right place
d) tailoring the installation steps to the type of component to deploy (e.g. needs to perform an Exec task of installutil.exe on a Windows service )
e) logging the result of the deployment
This system is built using a mix of:
- core MSBuild tasks
- [Tigris MSBuild community tasks][1]
- [MS SDC tasks][2]
- and custom tasks
The system allows us to perform consistent deployment of complex apps across partitioned environments (e.g. DEV, QA, UAT, etc) made of virtual servers.
I use MSBuild to build a fairly large client/server application. I use InstallShield 2008 to create a separate client and server install set.
By adding a custom target into the build process you can combine the creation of the installers into the build.
I would recommend that you create and test the build and the installer separately, before attempting to integrate the two.
I know this is an old question... but...
I am currently using MSBuild with MSBuild Extension Pack (http://msbuildextensionpack.codeplex.com) to do my entire deployment. The database portion is handled with the VS database command-line tool (vsdbcmd.exe - http://msdn.microsoft.com/en-us/library/dd193283.aspx). That Extension Pack is pretty amazing, and is letting me build web sites, app pools, Windows services, update config, and much more.
I've also put Team City agents on the test servers, so I can deploy as part of a build chain (introduced in version 7 of Team City). And running my deploy MSBuild script is super easy from Team City.
I used to use MSBuild, now I'm using PowerShell. MSBuild is a build language. It is painful to script in. There is a lot I wanted to do in it that were difficult and sometimes impossible.
Over the past year, I've created an PowerShell module somewhat equivalent to MSBuild Extension Pack called Carbon.
I strongly, strongly encourage everyone out there to learn and use PowerShell.
Generally when I use ClickOnce when I build a VB.NET program but it has a few downsides. I've never really used anything else, so I'm not sure
what my options are.
Downsides to ClickOnce:
Consists of multiple files - Seems easier to distribute one file than manageing a bunch of file and the downloader to download those files.
You have to build it again for CD installations (for when the end user dosn't have internet)
Program does not end up in Program Files - It ends up hidden away in some application catch folder, making it much harder to shortcut to.
Pros to ClickOnce:
It works. Magically. And it's built
into VisualStudio 2008 express.
Makes it easy to upgrade the
application.
Does Windows Installer do these things as well? I know it dosen't have any of the ClickOnce cons, but It would be nice to know if it also has the ClickOnce pros.
Update:
I ended up using Wix 2 (Wix 3 was available but at the time I did the project, no one had a competent tutorial). It was nice because it supported the three things I (eventually) needed. An optional start-up-with-windows shortcut, a start-up-when-the-installer-is-done option, and three paragraphs of text that my boss thinks will keep uses from clicking the wrong option.
Have you seen WiX yet?
http://wix.sourceforge.net/
It builds windows installers using an XML file and has additional libraries to use if you want to fancify your installers and the like. I'll admit the learning curve for me was medium-high in getting things started, but afterwards I was able to build a second installer without any hassles.
It will handle updates and other items if you so desire, and you can apply folder permissions and the like to the installers. It also gives you greater control on where exactly you want to install files and is compatible with all the standardized Windows folder conventions, so you can specify "PROGRAM_DATA" or something to that effect and the installer knows to put it in C:\Documents and Settings\All Users\Application Data or C:\ProgramData depending on if you're running XP or Vista.
The rumor is that Office 2007 and Visual Studio 2008 used WiX to create their installer, but I haven't been able to verify that anywhere. I do believe is is developed by some Microsoft folks on the inside.
I agree with Joseph, my experience with ClickOnce is its great for the vast majority of projects especially in a corporate environment where it makes build, publish and deployment easy. Implementing the "forced upgrade" to ensure users have the latest version when running is so much easier in ClickOnce, and a main reason for my usage of it.
Issues with ClickOnce: In a corporate environment it has issues with proxy servers and the workarounds are less than ideal. I've had to deploy a few apps in those cases from UNC paths...but you can't do that all the time. Its "sandbox" is great, until you want to find the executable or create a desktop shortcut.
Have not deployed out of 2008 yet so not sure if those issues still exist.
Creating an installer project, with a dependency on your EXE (which in turn depends on whatever it needs) is a fairly straightforward process - but you'll need at least VS Standard Edition for that.
Inside the installer project, you can create custom tasks and dialog steps that allow you to do anything you code up.
What's missing is the auto-upgrade and version-checking magic you get with ClickOnce. You can still build it in, it's just not automatic.
I don't believe there is any easy way to make a Windows Installer project have the ease or upgradability of ClickOnce. I use ClickOnce for all the internal .NET apps I develop (with the exception of Console Apps). I find that in an enterprise environment, the ease of deployment outweighs the lack of flexibility.
ClickOnce can be problematic if you have 3rd party components that need to be installed along with your product. You can skirt this to some extent by creating installers for the components however with ClickOnce deployment you have to create the logic to update said component installers.
I've in a previous life used Wise For Windows Installer to create installation packages. While creating upgrades with it were not automatic like ClickOnce is, they were more precise and less headache filled when it came to other components that needed to be registered/added.