We are looking for a solid, declarative (yaml), based proceedure to automate the setup of our Kubernetes cluster and application deployments on Google Container Engine.
As our last resort in a serious failure we want to be able to:
Create a new GCE cluster
Execute all our deployments to their latest versions
Execute all the steps in the correct order
What are the solutions people are currently using. Doing this manually takes us about an hour and is error prone. Really it could take 15-20 mins if automated.
You should take a look at Google Cloud Deployment Manager. It "automates the creation and management of your Google Cloud Platform resources for you" meaning that it can create a Google Container Engine cluster as well as create your deployments.
Looking through the GKE deployment manager example should help get you started.
Related
I have 2 apache nifi servers that are development and production hosted on AWS, currently the migration between development and production is done manually. I would like to know if it is possible to automate this process and ensure that people do not develop in production?
I thought about uploading the entire nifi in github and having it deploy the new nifi on the production server, but I don't know if that would be correct to do.
One option is to use NiFi registry, store the flows in the registry and share the registry between Development and Production environments. You can then promote the latest version of the flow from dev to prod.
As you say, another option is to potentially use Git to share the flow.xml.gz between environments and using a deploy script. The flow.xml.gz stores the data flow configuration/canvas. You can use parameterized flows (https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Parameters) to point NiFi at different external dev/prod services (eg. NiFi dev processor uses a dev database URL, NiFi prod points to prod database URL).
One more option is to export all or part of the NiFi flow as a template, and upload the template to your production NiFi, however registry is probably a better way of handling this. More info on templates here: https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#templates.
I believe the original design plan behind NiFi was not necessarily to have different environments, and to allow live changes in production. I guess you would build your initial data flow using some test data in production and then once it's ready start the live data flow. But I think it's reasonable to want to have separate environments.
We are using prometheus in our production envirment recently. Before we only have 30-40 nodes for each service and those servers not change very often, so we just write it in the prometheus.yml, but right now it become too long to hold in one file and change much frequently then before, so my question is should i use file_sd_config to put those server list out of yml file and change those config files sepearately, or using consul for service discovery(same much easy to handle changes).
I have install 3 nodes consul cluster in data center and as i can see if i change to use consul to slove this problem , i also need to install consul client in each server(node) and define its services info. Is that correct? or does anyone have good advise.
Thanks
I totally advocate the use of a service discovery system. It may be a bit hard to deploy at first but surely it will worth it in the future.
That said, Prometheus comes with a lot of service discovery integrations. It's possible that you don't need a Consul cluster. If your servers are in a cloud provider like AWS, GCP, Azure, Openstack, etc, prometheus are able to autodiscover the instances.
If you keep running with Consul, the answer is yes, the agent must be running in every node. You can also register services and nodes via API but it's easier to deploy the agent.
I just started trying out Spinnaker. I have gone through the tutorial, https://www.spinnaker.io/guides/tutorials/codelabs/gcp-kubernetes-source-to-prod/, and got it working without issues.
Now I want to go a bit more advanced and do a rolling release or a canary deployment (https://www.spinnaker.io/concepts/#deployment-strategies), where it is possible, for instance, to only expose a new release to 5% of the customers.
I cannot find any guide on spinnaker.io (or google) on how to set that up. Can anyone guide me in the right direction?
I have currently been experimenting and doing PoC's on Spinnaker and Canary Deployments myself of late, and here is what I have found thus far.
To implement a rolling release, just create a Deploy stage in Spinnaker, and set the Deployment Strategy to RollingUpdate in your Server Group config. You will need to make sure that the Deployment checkbox is checked before you can change the Deployment Strategy.
For the Canary Deployment, it is a little more involved. I don't think that the Canary Stage currently supports Kubernetes Deployments(yet), but apparently you can manually deploy a canary(e.g. 1 replica) into the same Kubernetes LoadBalancer where your app is running. This is done using a separate Spinnaker Server Group.
Then you can add a Manual Judgement to your Spinnaker pipeline that will pause until you test/validate the canary. Once the canary has been validated, you "Continue" the Manual Judgement, and the new Server Group gets deployed, and the old Server Group gets disabled, and the canary destroyed.
If you don't want to use a Manual Judgement, and want this fully automated, you can add an ACA Stage(Automated Canary Analysis). This involves setting up a judge, that Spinnaker can connect to, that will gather various metrics and provide an ACA score. You can then use that score to decide whether to proceed with a deployment, or stop the deployment.
I have a Python app running on a Kubernetes Cluster. I want to get app specific monitoring information e.g. logging info that I have in my app (using python logging module) and also metrics info that I am collecting using collectd.
I understand Operations Management Suite can be used to monitor the cluster itself but can it also provide access to app specific logs and metrics?
Appreciate any pointers on how to do this.
Thanks
Rajeev
For end to end devops automation I want to have an environment on demand. For this I need to Spun up and environment on kubernetes which is eventually hosted on GCP.
My Use case
1. Developer Checks in the code in feature branch
2. Environment in Spun up on Google Cloud with Kubernetes
3. Application gets deployed on Kubernetes
4. Gets tested and then the environment gets destroyed.
I am able to do everything with Spinnaker except #2. i.e create Kube Cluster on GCP using Spinnaker.
Any help please
Thanks,
Amol
I'm not sure Spinnaker was meant for doing what the second point in your list. Spinnaker assumes a collection of resources (VM's or a Kubernetes cluster) and then works with that. So instead of spinning up a new GKE cluster Spinnaker makes use of existing clusters. I think it'd be better (for you costs as well ;) if you seperate the environments using Kubernetes namespaces.