Vagrant in production - virtualhost

I've been reading about Vagrant, and I find it quite useful for my development. I am currently managing a series of services (mail, web, LDAP, file sharing, etc.), and often one of these falls and needs a quick backup. Is it possible (and recommended) to use Vagrant for these purposes?
So far I've virtual machines installed like real machines.
I would also like to know about an alternative to Vagrant which would allow me to setup a simple configuration file and put a virtual machine, for example, with Zimbra, and quickly have an alternate mail server, enable RabbitMQ, etc.

Vagrant should be used more like a staging environment to test your infrastructure changes. It should be your test bed for automated infrastructure changes.
The way we use it at my company is like so:
Create VMs for our managed servers in Vagrant.
Create puppet definitions for each server.
Create cucumber tests for each server.
Make infrastructure changes via puppet and add cucumber tests.
Launch our servers to test for failures.
Fix bugs, release and/or back to step 4.
Basically when we're happy with our changes, we'll pull our puppet changes into production to make it happen.
I'd not recommend using vagrant to manage VMs for real production. I'd use something else like razor, virsh, OpenStack or one of the many other vm management systems out there.

This page suggests that the Vagrant push command is meant for deploying to production:
https://www.hashicorp.com/blog/vagrant-push-one-command-to-deploy-any-application/
"Additionally, multiple config.push.define declarations can be in a Vagrantfile to define multiple pushes, perhaps one to staging and one to production, for example."

From my experience, Vagrant mainly used in a development environment.
Vagrant configuration and provisioning options are limited compared to Terraform for example.
If you are working on a cloud based environment, you can use Terraform for infrastructure provisioning.
If your environment is local or your VMs will be hosted on a datacenter, you can use Ansible, chef or puppet for you configuration management and automation.

Hashicorp just published Otto, which is meant to be the Vagrant's successor. It is designed to support deployment environments.
From their Github page:
The key features of Otto are:
Automatic development environments: Otto detects your application
type and builds a development environment tailored specifically for that
application, with zero or minimal configuration. If your application depends
on other services (such as a database), it'll automatically configure and
start those services in your development environment for you.
Built for Microservices: Otto understands dependencies and versioning
and can automatically deploy and configure an application and all
of its dependencies for any environment. An application only needs to
tell Otto its immediate dependencies; dependencies of dependencies are
automatically detected and configured.
Deployment: Otto knows how to deploy applications as well develop
them. Whether your application is a modern microservice, a legacy
monolith, or something in between, Otto can deploy your application to any
environment.
Docker: Otto can use Docker to download and start dependencies
for development to simplify microservices. Applications can be containerized
automatically to make deployments easier without changing the developer
workflow.
Production-hardened tooling: Otto uses production-hardened tooling to
build development environments (Vagrant),
launch servers (Terraform), configure
services (Consul), and more. Otto builds on
tools that powers the world's largest websites.
Otto automatically installs and manages all of this tooling, so you don't
have to.

I had the same question and have been investigating the use of Vagrant push which as per their documentation, as of version 1.7, Vagrant is capable of deploying or "pushing" application code in the same directory as your Vagrantfile to a remote such as an FTP server.
I'm considering having vagrant spin up in a VM for developers, while also giving you the option to deploy your code to a live server for production environments.

As mentioned by #andrerpena, Otto is the successor of Vagrant.
From www.ottoproject.io :
Otto can deploy your application. Users of Vagrant for years have wanted a way to deploy their Vagrant environments to production. Unfortunately, the Vagrantfile doesn't contain enough information to build a proper production environment with industry best practices. An Appfile is made to encode this knowledge, and deployment is a single command away.

Related

To virtualize or not to virtualize a bare metal server for a kubernetes deployment

I'd like to deploy kubernetes on a large physical server (24 cores) and I'm uncertain as to a number of things.
What are the pros and cons of creating virtual machines for the k8s cluster other than running on bare-metal.
I have the following considerations:
Creating vms will allow for work load isolation. New vms for experiments can be created and assigned to devs.
On the other hand, with k8s running on bare metal a new NAMESPACE can be created for each developer for experimentation and they can run their code in it. After all their code should be running in docker containers.
Security:
Having vms would limit the amount of access given to future maintainers, limiting the amount of damage that could be done. While on the other hand the primary task for any future maintainers would be adding/deleting nodes and they would require bare metal access to do that.
Authentication:
At the moment devs would only touch the server when their code runs through the CI pipeline and their running deployments are deployed. But what about viewing logs? Could we setup tiered kubectl authentication to allow devs to only access whatever namespaces have been assigned to them (I believe this should be possible with the k8s namespace authorization plugin).
A number of vms already exist on the server. Would this be an issue?
128 cores and doubts.... That is a lot of cores for a single server.
For kubernetes however this is not relevant:
Kubernetes can use different sized servers and utilize them to the maximum. However if you combine the master server processes and the node/worker processes on a single server, you might create unwanted resource issues. You can manage those with namespaces, as you already mention.
What we do is use continuous integration with namespaces in a single dev/qa kubernetes environment in which changes have their own namespace (So we run many many namespaces) and run full environment deployments in those namespaces. A bunch of shell scripts are used to manage this. This works both with a large server as what you have, as well as it does with smaller (or virtual) boxes. The benefit of virtualization for you could mainly be in splitting the large box in smaller ones so that you can also use it for other purposes then just kubernetes (yes, kubernetes runs except MS Windows, no desktops, no kernel modules for VPN purposes, etc).
I would separate dev and prod in the form of different vms. I once had a webapp inside docker which used too many threads so the docker daemon on the host crashed. It was limited to one host luckily. You can protect this by setting limits, but it's a risk: one mistake in dev could bring down prod as well.
I think the answer is "it depends!" which is not really an answer. Personally, I would split up the machine using VM's and deploy that way. You've got better flexibility as to how much of the server's resources you carve out and you can easily create new environments, then destroy easily.
Even if these vms are really big, I think it's still easier to manage also given that you have existing vm's on the machine.
That said, there's not a technical reason that you can't run a single node server, but you may run into problems with downtime with upgrades (if that's an issue), as well as if that server needs patched or rebooted, then your entire cluster is down.
I would look at your environment needs for HA and uptime, as well as how you are going to deploy VM's (if you go that route), and decide what works the best for you.

Configuring Development Environments

If the development environment is run on the host, is there a fast way to configure these and deploy them to multiple machines? If so, how?
Vagrant does this but as virtual machines, which may be painfully slow.
What are some viable alternatives?
Absolutely, if you are talking about using a resource pool to deploy and configure your VMs (dev VMs).
For the sake of simplicity, I assume your virtual environment is hosted on VMWare.
Here are a few things to start off with
For Automating creation of VMs (deploying from templates, configure networking etc) you can use VMWare's PowerCLI (Powershell cmdlets) - more on this here
If your dev environment is purely Windows then, for configuration management, you can use DSC (free) or if you (your org) can afford go the expensive route of using SCCM
If you have a lot of Linux boxes to configure then, fortunately, you have more than one option - Ansible, Chef, Puppet or SaltStack
Spinning off and configuring these machines on demand entirely depends on your needs. One of the (more common ways) to do is, create a VM template (aka base vm) and then deploy a VM from this template.
This base template is usually a bare-bones OS + some common utils + tools - Once a VM is deployed from the template, you can then use one of the above mentioned CM (Configuration Management) tools to install/configure this VM.
Hope this gives some pointers in the right direction

Automating/Maintaining Development, Production and Staging environments

Currently at my company, we have three servers: Development , Staging, and Production.
When a new feature is requested, it is developed in the 'Development' environment (server/VM). Once the developer is satisfied that it's complete, they move it to the Staging environment and test it. Once it is fully tested, it is moved to the Production server.
The problem is that the team is very small and it's hard to manually maintain the Staging and Production Environment. We need the staging environment to be exactly like the Production environment, and it is time consuming task to do that manually everytime.
Is there a software out there that can help? or a process?
One method we thought could be possible is Cloning the Production Environment every time and use it as Staging. Once the feature is tested, merge the two VM's. But that might take a lot of time and cause downtimes on production environment.
Any hints/suggestions?
Thanks
I would suggest you to have a look at an environment provisioning tool like Ansible, Puppet or Chef. It requires some time to get used to it but after that you'll be able to auto provision any system (be it a server or just a developer machine) with just a few terminal/command line commands.

Suggestions for software to ease setting up a build server

I'm currently setting up a new build server and I'm interested in any suggestions the community may have about software such as Hudson or CruiseControl.NET that may simplify and add additional value to the build process.
Previously I had a build server set up using custom batch files which would run msbuild and other such tools and these were triggered by subversion hooks to allow for a continuous builds to be done per branch. The idea was that eventually we would also execute automated tests and/or static analysis although we never really got that far. This server also acted as our source code repository, a test machine for web project builds, and a web server for custom dashboard and portal for developers on the team.
At this point my thoughts are to separate some of the responsibilities of the old build server and at least a Build Server which is responsible only for creating builds, a web server which is responsible for acting as the intranet style dashboard site for developers, and perhaps an additional web server as the Subversion repository. If it turns out to be better or easier to keep the Subversion code on the same server as SvnServe then I'll probably opt to place the Subversion repository on the web server but still keep the build server separate. Having no personal experience with any of the popular build server and CI solutions out there I'm curious how CruiseControl.NET, Hudson or other solutions would fit into this type of configuration. It appears that both of CC.NET and Hudson have web interfaces for example but the documentation doesn't clearly layout how this plays out with different hardware/system configurations so I'm not sure if either requires the web portion to be on the build server itself or not.
As far as technologies I'm dealing with .NET/C# based code which is a mix of Web/WinForms/WPF and we use a few separate Subversion repositories to host these projects. Additionally it would be nice to support Visual FoxPro and Visual Source Safe for some legacy applications. I would also like to get more team members involved in monitoring builds and would like to eventual have developers create build setups for their own projects as well with as much simplicity as possible. Also I should mention that I have no experience setting up a Java based web application in IIS but I do have quite a bit of experience setting up and managing ASP.NET applications so if that may make .NET based products more favorable unless I can be convinced otherwise.
UPDATE (after researching Hudson): After all the recommendations for Hudson I started looking into what is involved to get it up and running on my two Windows 2008 servers. From what I can gather the web portion (master) would run on my webserver but it seems that IIS isn't supported so this would greatly complicate things since I want to host it on the same machine as my other web applications. On the build server, I would be installing a second copy of Hudson that would act as a slave and only perform builds that are delegated to it by the master. To get this to work I would be installing Hudson as a Windows Service and would also need to install some unix compatibility utilities. Unfortunately the UnxUtils download link appears to be broken when I checked as well so I can't really move forward until I get that resolved. All of this is really sounding just as complex if not more complex than installing CruseControl.NET. For now this unfortunately leaves me to looking into CruiseControl.NET and TeamCity.
UPDATE (about TeamCity): After looking into TeamCity a little closer I realized that at least the server portion is also written in Java and is deployed in a manner very similar to Hudson. Fortunately it appears that Tomcat can be used to host servlets inside IIS although I can't find a good straight forward guide to describe how to actually do accomplish this. So skipping that for now I looked further when I ran into what looks like what might be a major snag.
TeamCity Professional edition only
supports TeamCity Default
Authentication and does not support
changing the authentication scheme.
Since windows authentication is likely the direction we will want to go, it's now looking like it might be back to evaluating CruiseControl.NET or possibly Hudson if I can get my hands on the UnxUtils and also find out more about how I can host the dashboard portion of Hudson within my existing IIS configuration. Any pointers?
UPDATE (about Jenkins): I ended up experimenting enough with Hudson that I ended up with a reasonable build server setup that I'm happy with and that can be extended to do much more if I need. Of course I went the rout of converting to Jenkins once Oracle took over Hudson and Jenkins is what I'm using today with little bits of powershell to help tie things together. I'm very happy with this approach right now and besides being Java based, Jenkins has quite a bit of support for other development environments such as .NET and MSBuild.
I'd vote for TeamCity here. Its is very, very easy to get stood up and running, integrates with all your .NET stuff without any trouble. The builds themselves are run by agents which can be on the build server or another machine depending on requirements--they could even be on a machine running an entirely different OS on a different network in a different country.
I highly recommend using Hudson. Not only will it allow you to build .NET applications on a continual basis, but you can also run code analysis and unit tests as well. It's easy to install (just deploy a WAR file to a web server such as Tomcat) and has many configuration options. There is also a large number of plugins available that you can use, many written by other Hudson users. Best of all, it is free and actively supported.
For our decision making process we started with following overview.
http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix
Our main objective was java, easy to configure/use even after nobody created a job for 6 months. We moved away from a old version of Cruise Control, since nobody really knew how to use it. Some of the commercial products are nice if you want to go beyond just continuous integration. Have a look and decide for yourself.
Be careful, I don't know how up to date this matrix is. So some of the projects might have implemented more functions right now.
An interesting alternative could be Jira studio by Atlasian. If you use the hosted version you don't have much on support issues and it comes with subversion, bamboo, and goodies (jira+greenhopper, confluence, crucible, fisheye). http://www.atlassian.com/hosted/studio/
I agree with Wyatt Barnett. TeamCity is the best choice. It is very easy to configure and use. Moreover, TeamCity has a Free Professional Edition. Previously we used CruiseControl.NET on our project. This is also a powerful tool, but it is very complicated and hard to understand.
What s.ermakovich said: Both TeamCity and Hudson separate the web UI from build agents. You shouldn't need to install IIS on a build agent. You'd need to install a JVM and the agent software on any build node - very straightforward.

Publish an web application on build with NAnt, MSBuild or any other tool

I have a scenario where I have to setup a test environment where I want to be able to tell my NAnt or other build tool to make an new IIS web application, put the latest bins in the newly created IIS web application, and post me an email where the new address and port where the new application are addressed, is this possible and how? which tool?
There are several ways to approach this:
Set up a continuous integration (CI) server on the test environment. This is a viable option if your test environment machine doesn't change often and it's a single machine.
Push the installation from your development machine using tools like PsExec
Combination of the two: you have a build CI server which pushes the installation to (multiple) test environments.
Of course, you also need a good build script which will set up the IIS application (NAnt offers tasks for this). Emailing to you can be done by CI server (CruiseControl.NET Email Publisher, Hudson...).
I suggest taking some time to read this excellent article series: Automation for the people: Deployment-automation patterns
Our CruiseControl .Net build server does exactly this as part of it's NAnt build-script process...
Once the code is retrieved from source control, it's all built/compiled in turn. Web projects are then handled slightly differently to normal .dlls, as they are deployed to a particular folder (either on the current machine or otherwise) where IIS (also set-up by the script) to serve the pages.
Admittedly, we're using Virtual Directories instead of creating and disposing of new website instances on the server, as otherwise we'd have to manage the port numbers for each website.
NAnt has the capabilities of doing all of this IIS work, as well as all of the email work too - I'd certainly recommend looking at this avenue of enquiry to solve your problem. Plus, you also get the continous integration aspect as a side-benefit in your case!