I am looking for some thing like Amazon Code Deploy In-place deployments. Where you deploy with out actually baking the instance again. Was not able to find any documentation on doing it. Any help and inputs are appreciated.
Use the Stage "Find Image from Tags" and use the Parameter "AMI_NAME".In Value field add the "AMI ID". In this case Spinnaker will skip the AMI baking and use the AMI you defined for Deployment.
Related
is there a way to copy a file to all the nodes in EMR cluster thought EMR command line? I am working with presto and have created my custom plugin. The problem is I have to install this plugin on all the nodes. I don't want to login to all the nodes and copy it.
You can add it as a bootstrap script to let this happen during the launch of the cluster.
#Sanket9394 Thanks for the edit!
If you have the control to Bring up a new EMR, then you should consider using the bootstrap script of the EMR.
But incase you want to do it on Existing EMR (bootstrap is only available during launch time)
You can do this with the help of AWS Systems Manager (ssm) and EMR inbuilt client.
Something like (python):
emr_client = boto3.client('emr')
ssm_client = boto3.client('ssm')
You can get the list of core instances using emr_client.list_instances
finally send a command to each of these instance using ssm_client.send_command
Ref : Check the last detailed example Example Installing Libraries on Core Nodes of a Running Cluster on https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-jupyterhub-install-kernels-libs.html#emr-jupyterhub-install-libs
Note: If you are going with SSM , you need to have proper IAM policy of ssm attached to the IAM role of your master node.
First of all, sorry if this thread is not appropiated in Stack Overflow, but I think that is the best place of all.
We are using Rancher to manage a microservices solution. Most of the containers are NodeJS + Express apps, but there are others like Mongo or Identity Server.
We use many environment variables like endpoints or environment constants and, when we upgrade some of the containers individually, we forget to include them (most of the times, the person who deploys an upgrade is not the person who made the new version).
So, we're looking a way to manage them. We know that using a Dockerfile could be the best way, but if we need to upgrade just one container, we think that is too many work for just a minor change.
TLDR; How do you manage your enviromental variables in Rancher? How do you document them or how you extract them automatically?
Thanks!
Applications in Rancher are generally managed using Stacks/Services. Dockerfile is used to build a container image. docker-compose/rancher-compose files are used to define the applications. The environment variables can be specified in docker-compose file.
When you upgrade a service in rancher, the environment variables information is carried forward and also it's possible to edit them before upgrade.
Also Rancher "Catalog" feature might be something useful for you. Checkout: https://rancher.com/docs/rancher/v1.6/en/catalog/
Creating AMI's from EBS backed instances is exceedingly easy, but doing the same from an instance-store based instance seems like it can only be done manually using the CLI.
So far I've been able to bootstrap the creation of an 'instance-store' based server off of an HVM Amazon Linux AMI with Ansible, but I'm getting lost on the steps that follow... I'm trying to follow this: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-instance-store-ami.html#amazon_linux_instructions
Apparently I need to store my x.509 cert and key on the instance, but which key is that? Is that...
one I have to generate on the instance with openssl,
one that I generate/convert from AWS,
one I generate with Putty, or
one that already exists in my AWS account?
After that, I can't find any reference to ec2-bundle-vol in Ansible. So I'm left wondering if the only way to do this is with Ansible's command module.
Basically what I'm hoping to find out is: Is there a way to easily create instance-store based AMI's using Ansible, and if not, if anyone can reference the steps necessary to automate this? Thanks!
Generally speaking, Ansible AWS modules are meant to manage AWS resources by interacting with AWS HTTP API (ie. actions you could otherwise do in the AWS Management Console).
They are not intended to run AWS specific system tools on EC2 instances.
ec2-bundle-vol and ec2-upload-bundle must be run on the EC2 instance itself. It is not callable via the HTTP API.
I'm afraid you need to write a custom playbook / role to automate the process.
On the other hand, aws ec2 register-image is an AWS API call and correspond to the ec2_ami Ansible module.
Unfortunately, this module doesn't seem to support image registering from an S3 bucket.
I have a kubernetes cluster, and I am wondering how (best practice) to update containers. I know the idea is to tear down the old containers and put up new ones, but is there a one-liner I can use, do I have to remove the replication controller or pod(s) and then spin up new ones (pods or replicaiton controllers)? With this I am using a self hosted private library that I know I have to build from the Dockerfile and the push to anyway, this I can automate with gulp (or any other build tool), can I automate kubernetes update/tear down and up?
Kubectl can automate the process of rolling updates for you. Check out the docs here:
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/kubectl_rolling-update.md
A rolling update of an existing replication controller foo running Docker image bar:1.0 to image bar:2.0 can be as simple as running
kubectl rolling-update foo --image=bar:2.0.
Found where in the Kubernetes docs they mention updates: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/replication-controller.md#rolling-updates. Wish it was more automated, but it works.
EDIT 1
For automating this, I've found https://www.npmjs.com/package/node-kubernetes-client which since I already automate the rest of my build and deployment with a node process, this will work really well.
The OpenShift Origin project (https://github.com/openshift/origin) runs an embedded Kubernetes cluster, but provides automated build and deployment workflows on top of your cluster if you do not want to roll your own solution.
I recommend looking at the example here:
https://github.com/openshift/origin/tree/master/examples/sample-app
It's possible some of the build and deployment hooks may move upstream into the Kubernetes project in the future, but this would serve as a good example of how deployment solutions can be built on top of Kubernetes.
Can someone elaborate more on the details of how to remotely start a EC2 instance remotely?
I have a Linux box set up locally, and would like to set up a cronjob on it to start an instance in Amazon EC2. How do I do that?
I've never worked with API's, if there are ways to use API's, can someone please explain how to do so...
Pretty Simple.
Download EC2 API. There is a CLI with it.
keep EC2_PRIVATE_KEY and EC2_CERT in as your envt variables, where they are private key and certificate files that you generate from EC2 console.
then call ec2-reboot-instances instance_id [instance_id ...]
Done.
Refer: http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-RebootInstances.html
Edit 1
Do I download this directly onto my Linux box? And how do I access the CLI on the linux box of the EC2 API? Sorry to ask so many questions, just need to know detailed steps of how to do this.
Yes. Download it from here
If you have unzipped the API in /home/naishe/ec2api, you can call /home/naishe/ec2api/bin/ec2-reboot-instance <instance_id>. Or event better set unzipped location as your envt variable EC2_API_HOME and append $EC2_API_HOME/bin to your system's PATH.
Also, try investing some time on Getting Started Doc which is amazingly simple.