Currently at my company, we have three servers: Development , Staging, and Production.
When a new feature is requested, it is developed in the 'Development' environment (server/VM). Once the developer is satisfied that it's complete, they move it to the Staging environment and test it. Once it is fully tested, it is moved to the Production server.
The problem is that the team is very small and it's hard to manually maintain the Staging and Production Environment. We need the staging environment to be exactly like the Production environment, and it is time consuming task to do that manually everytime.
Is there a software out there that can help? or a process?
One method we thought could be possible is Cloning the Production Environment every time and use it as Staging. Once the feature is tested, merge the two VM's. But that might take a lot of time and cause downtimes on production environment.
Any hints/suggestions?
Thanks
I would suggest you to have a look at an environment provisioning tool like Ansible, Puppet or Chef. It requires some time to get used to it but after that you'll be able to auto provision any system (be it a server or just a developer machine) with just a few terminal/command line commands.
Related
What is the normal process for working with Postgres in a Node, React, Express application?
In that past, I've used MongoDB Atlas to quickly create and host a database for my development environment. My current project is using Postgres for the database, however, I'm trying to figure out the process/workflow.
For example, trying to find a Postgres as a service equivalent to Atlas, Heroku and Digital Ocean seem to match that bill. However, it looks like Heroku needs a deployed project to work. I don't want to deploy yet and really just want a development database.
Is it normal practice for everyone on the project to spin up a local Postgres database and then deploy/host a single production database at the end? Thanks for the help!
I would suggest following workflow that I've recently used on a project.
local instances on docker or something (feature development purposes)
first staging instance that could be used as stable dev env (can be used for specific feature testing, or as a internal testing environment)
second staging instance where stable master branch with hashed prod data can be used for external testing
prod or prod mirror, depends on project requirements
It would be also helpful to have migrations within the code, so that you can keep track of DB development as well. Also look into https://sequelize.org/ as it could be helpful.
If the staging environment is an isolated environment for testing by testers and also sandbox is an isolated environment for testing, So what are those differences?
Actually I could not find any useful and clear information on.
Good question. Given the background you provide they appear the same. This is true in that they are both isolated from the production environments. They should not contain production data. etc. However there are a number of differences particularly in how they are used.
Staging environment
A good staging environment will be a close replica (less the data) of the production system. It is used to test upgrades and patches prior to going to production. This means that it should be a controlled environment where the engineers responsible for the production deployment are allowed to test the rollout instructions.
Access restrictions in a staging environment should be as close to production as possible. I.E. deployment by those engineers who are responsible for deployment. No root (or privileged access for developers).
Sandbox environment
As the name suggests this typically a playground for the engineering team. It has less restrictions than a staging environment because it is designed to allow the engineers to try out things easily and quickly. A sandbox environment is likely to drift away from the production environment as engineers try out different versions of the product, dependancies, plugins etc.
Access to a sandbox environment typically allows privileged access to any engineer (developer, QA etc.) who are working on the project for easy / quick deployment and debugging.
I've been reading about Vagrant, and I find it quite useful for my development. I am currently managing a series of services (mail, web, LDAP, file sharing, etc.), and often one of these falls and needs a quick backup. Is it possible (and recommended) to use Vagrant for these purposes?
So far I've virtual machines installed like real machines.
I would also like to know about an alternative to Vagrant which would allow me to setup a simple configuration file and put a virtual machine, for example, with Zimbra, and quickly have an alternate mail server, enable RabbitMQ, etc.
Vagrant should be used more like a staging environment to test your infrastructure changes. It should be your test bed for automated infrastructure changes.
The way we use it at my company is like so:
Create VMs for our managed servers in Vagrant.
Create puppet definitions for each server.
Create cucumber tests for each server.
Make infrastructure changes via puppet and add cucumber tests.
Launch our servers to test for failures.
Fix bugs, release and/or back to step 4.
Basically when we're happy with our changes, we'll pull our puppet changes into production to make it happen.
I'd not recommend using vagrant to manage VMs for real production. I'd use something else like razor, virsh, OpenStack or one of the many other vm management systems out there.
This page suggests that the Vagrant push command is meant for deploying to production:
https://www.hashicorp.com/blog/vagrant-push-one-command-to-deploy-any-application/
"Additionally, multiple config.push.define declarations can be in a Vagrantfile to define multiple pushes, perhaps one to staging and one to production, for example."
From my experience, Vagrant mainly used in a development environment.
Vagrant configuration and provisioning options are limited compared to Terraform for example.
If you are working on a cloud based environment, you can use Terraform for infrastructure provisioning.
If your environment is local or your VMs will be hosted on a datacenter, you can use Ansible, chef or puppet for you configuration management and automation.
Hashicorp just published Otto, which is meant to be the Vagrant's successor. It is designed to support deployment environments.
From their Github page:
The key features of Otto are:
Automatic development environments: Otto detects your application
type and builds a development environment tailored specifically for that
application, with zero or minimal configuration. If your application depends
on other services (such as a database), it'll automatically configure and
start those services in your development environment for you.
Built for Microservices: Otto understands dependencies and versioning
and can automatically deploy and configure an application and all
of its dependencies for any environment. An application only needs to
tell Otto its immediate dependencies; dependencies of dependencies are
automatically detected and configured.
Deployment: Otto knows how to deploy applications as well develop
them. Whether your application is a modern microservice, a legacy
monolith, or something in between, Otto can deploy your application to any
environment.
Docker: Otto can use Docker to download and start dependencies
for development to simplify microservices. Applications can be containerized
automatically to make deployments easier without changing the developer
workflow.
Production-hardened tooling: Otto uses production-hardened tooling to
build development environments (Vagrant),
launch servers (Terraform), configure
services (Consul), and more. Otto builds on
tools that powers the world's largest websites.
Otto automatically installs and manages all of this tooling, so you don't
have to.
I had the same question and have been investigating the use of Vagrant push which as per their documentation, as of version 1.7, Vagrant is capable of deploying or "pushing" application code in the same directory as your Vagrantfile to a remote such as an FTP server.
I'm considering having vagrant spin up in a VM for developers, while also giving you the option to deploy your code to a live server for production environments.
As mentioned by #andrerpena, Otto is the successor of Vagrant.
From www.ottoproject.io :
Otto can deploy your application. Users of Vagrant for years have wanted a way to deploy their Vagrant environments to production. Unfortunately, the Vagrantfile doesn't contain enough information to build a proper production environment with industry best practices. An Appfile is made to encode this knowledge, and deployment is a single command away.
The development environment is obviously for developing the web app locally.
The production is the operational public website with everything as the final version.
what is the testing environment and what is the difference between that and staging environment?
and if its what I think it is, why not just deploy to an alternate website address and use that?
The testing environment is an environment in which Rails will perform the tests that you write. This is a slimmer environment in which certain things are disabled such as sending emails, and loading the asset pipeline. The database in the test environment is also generally not persisted between test runs.
For more information on how and why to test your rails applications check out the Rails Guides. If you are not currently testing your applications, you should really start as it can save days of hunting for broken code, and many many headaches down the road.
Staging environments are setup to mimic most of the functionality of a production environment. The staging 'stack' should be as similar to the production environment as possible; e.g. same hardware, same OS, same installed packages, etc.
Things I generally limit when setting up a staging environment are email sending, I will use a mail interceptor to send all email to my address. Think of the staging environment as a sandbox in which to test changes to your production environment, without actually testing in production.
For most small apps, a staging environment is not always necessary. However, for a large, enterprise level application it is a must as you generally cannot afford any downtime for your production system. With the staging environment you can catch those changes before they are pushed to production.
Hope this helps.
hey everyone.
ok, a little background to the project... I've released a program to a couple clients that use a WCF service for them to connect to our servers in the office. Being that i used the clickOnce setup utility in VS2010, when i am doing testing on a VM i publish to a different spot on the server as to not give untested code to clients. However, the WCF service only gets published to one place for both versions (development and release).
What i'm working on now requires a change to the WCF service as a couple additional things get transferred between client and server. If I publish the modified WCF, will it affect the current clients, or will i be able to test my development version with no worries? I'm afraid i already know that the answer will be, yes it will affect them.
thanks!
dave k.
Isolation of test, dev and production:
Whenever you need to test something or put it in production, you need a separate environment. So you need a separate machine to develop and do local testing, another to test the checked-in code (for use by a tester, a customer etc.), and another to run the production code -- at the very least.
If your service interacts with other software, especially with software that gets updated a lot, this is an important way to make sure that you don't introduce side-effects and that what you build will be compatible with what is running on your production server.
So: isolate and make your test environment a "clone" of your production environment.
Two versions in parallel:
If you update your own code for customer X, you can still host a previous version of your production code for customer Y on another (virtual?) server. Customer Y can then choose when to switch to your new version, after which you can take the old code out of production.
You should create a seperate VM for your test environment.