I see Fargate as a good service for deploying a Docker Compose based stack, but I was wondering if it is any good for "long-running" web services, not just ones where you need dynamic scaling and undeterminate workloads (e.g. containers that are created and die on demand).
That depends on your use-case. ECS lets you quickly deploy containerized applications. With Fargate we don't need to manage the underlying infrastructure (say server-less approach for containers!). Fargate is suitable for long-running apps, microservices, and batch jobs.
Few of my observations on Fargate are:
Fargate storage is ephemeral - We cannot store container data in disk such as volumes. (although Fargate provides 10 GB of volume mounts that is nonpersistent empty storage.)
Logs can be sent to Cloudwatch using awslogs driver. Recently AWS announced the support for Splunk log driver.
Fargate uses only awavpc network mode.
Fargate supports environment variables. Environment variables are the only way to pass parameters to the container.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 months ago.
Improve this question
Can somebody explain the difference in tooling between Crossplane and Cluster API from the perspective of a managed kubernetes platform provision ?
https://github.com/kubernetes-sigs/cluster-api
I was using crossplane for sometime to create k8s clusters and recently got to know that Cluster API is also having the same capability to provision k8s clusters.
Cluster API
Cluster API is one of the projects in CNCF that you can use the create and manage the Kubernetes clusters. it's more like Kubeadm, kind, k3s, minikube(not for prod usecase) with YAML config & CLI.
You can create the YAML file and provision the Kubernetes cluster and manage it with that. There are different providers(AWS, AKS, GCP) available so you can use a provider-set service account or Access key secret and once you apply YAML to cluster API it will create a K8s cluster based on config and provider.
So with this, you can manage multiple Kubernetes clusters.
With cluster API YAML config you can create/manage K8s cluster on AWS, GCP, On-prem etc.
Crossplane
Consider crossplane as a deployment or service you are running on Kubernetes first.
You pass YAML config to cross-plane service based on rules it will create/manage the resources outside on cloud providers. It will also create/Manage the RDS, CloudSQL instances, Kubernetes clusters and other resources that the cloud provider supports.
It has also concept of provider(AWS, GCP, AKS)
Crossplane uses Kubernetes as the underlying platform to expose these features and workflows to manage cloud infrastructure outside of Kubernetes, such as data centers, the cloud or the edge. It installs a set of components called providers by extending Kubernetes to manage resources from cloud providers
Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management.
Kubernetes container orchestration technology has swept the enterprise world, with 75% of organizations running containers in production soon (most orchestrated by Kubernetes). Kubernetes delivers a cloud-native platform for application delivery, runtime, and scaling. While these are huge benefits for scale out applications, providing all these capabilities comes at the expense of the complexity of the platform itself. These complexities can be categorized as
Complex and heavyweight lifecycle management: significant time and resources needed to provision, upgrade, and maintain the control planes of clusters. Each cloud has a different set of IaaS primitives that need to be cobbled together to make a “cluster”.
Intricate and involved integrations for basic components such as storage, networking, security, and others.
Blockquote
Difficult and not always consistent multi-cluster management and operations.
Cluster API
Provisioning Kubernetes clusters is never an easy task. When there are 1000+ clusters, definitely want to have a standardized approach to ease tasks. Some of might know tools like kOps, Kubespray. Imagine Cluster API as their alternative solution, but more powerful!
According to the official page, "Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters."
Here are some highlighted points of Cluster API:
Pure YAML-based. Kubernetes style. Super handy.
Support any mainstream infrastructure provider. Provision your Kubernetes clusters in cloud/on-premise environments in the same place.
Managed Kubernetes services support. AWS EKS, Azure AKS, GCP GKE all are supported.
Bring your own infrastructure. Reuse existing infrastructures. Focus on provisioning Kubernetes clusters.
Crossplane
Crossplane uses Kubernetes as the underlying platform to expose these features and workflows to manage cloud infrastructure outside of Kubernetes, such as data centers, the cloud, or the edge.
It installs a set of components called providers by extending Kubernetes to manage resources from cloud providers. Providers build on the Kubernetes API, along with the custom resources and controllers, and expose declarative APIs to manage cloud provider resources.
Crossplane is geared toward platform teams and operators looking to build abstractions and easier-to-consume infrastructure components based on an organization's needs. It has a few concepts, such as CompositeResourceDefinition and Composition, that enable teams to extend and build on top of the default resources exposed for a cloud provider.
First of all lets understand what is Crossplane and what is Cluster API
Cluster API
Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters.
Provide declarative APIs
Give the tools to simplify provisioning, upgrading and operating multiple clusters
Helps to manage the lifecycle (create, scale, upgrade, destroy) of Kubernetes-conformant cluster (uses a declarative API)
Give you the ability to work on different environments (on-premisses and cloud)
Can find this and more about kubernets cluster api at https://cluster-api.sigs.k8s.io/
Crossplane
Crossplane is a framework for building cloud native control planes without needing to write code. It has a highly extensible backend that enables you to build a control plane that can orchestrate applications and infrastructure no matter where they run, and a highly configurable frontend that puts you in control of the schema of the declarative API it offers.
I've found a few interesting posts about Crossplane vs Kubernetes API's
https://blog.crossplane.io/crossplane-vs-cloud-infrastructure-addons/
https://www.infracloud.io/blogs/cluster-provisioning-using-crossplane/
Its not much but hope this helps
I have a service, that runs on run on kubernetes, uses Apache Ignite to store some data for processing, runs in replication mode with native persistence enabled. How to rightly mount the volume so the data is persisted the disk? Please note, this question is not related to mounting volumes in Kubernetes, rather the configuration/method to enable persistence in service running with embedded Ignite server in Kubernetes.
Note: The application may run multiple replicas.
Edit: As volumes (pvc) cannot be shared by multiple pods, only pod runs successfully, and other pods are in pending state.
The stateless means the system does not have dependency during its start or execution, but only be as stateless as possible. So, as the need itself is persistence, the Ignite has to be deployed as stateful using the StatefulSet. The StatefulSet will automatically provision separate volumes & mount it to every pod.
Checkout out Ignite guides for mounting K8 on AWS, GKE, and Azure
Does google cloud or aws provide manage Apache tomcat which just take war file and do auto-scaling based on load increase and decrease ? not compute engine. I dont want to create VM. this should be manage by manage service.
Google App Engine can directly take and run a WAR file - just use the appcfg deployment method.
You will have more options if you package with docker, as this then provides an image type that can be run in many places (Multilpe GCP, AWS and Azure options, on-prem Kubernetes, etc). This can even be as simple as building a dockerfile that just copies the WAR into a jetty image:
FROM jetty:latest
COPY YOUR_WAR.war /var/lib/jetty/webapps
It might be better to explode the war though - see discussion in this question
AWS provide ** AWS Elastic Beanstalk **
The AWS Elastic Beanstalk Tomcat platform is a set of environment configurations for Java web applications that can run in a Tomcat web container. Each configuration corresponds to a major version of Tomcat, like Java 8 with Tomcat 8.
Platform-specific configuration options are available in the AWS Management Console for modifying the configuration of a running environment. To avoid losing your environment's configuration when you terminate it, you can use saved configurations to save your settings and later apply them to another environment.
To save settings in your source code, you can include configuration files. Settings in configuration files are applied every time you create an environment or deploy your application. You can also use configuration files to install packages, run scripts, and perform other instance customization operations during deployments.
It also provide autoscaling
The Auto Scaling group in your Elastic Beanstalk environment uses two Amazon CloudWatch alarms to trigger scaling operations. The default triggers scale when the average outbound network traffic from each instance is higher than 6 MB or lower than 2 MB over a period of five minutes. To use Amazon EC2 Auto Scaling effectively, configure triggers that are appropriate for your application, instance type, and service requirements. You can scale based on several statistics including latency, disk I/O, CPU utilization, and request count.
I wanted to share some data to every worker in a swarm cluster.what are the possible methods to do the same.The swarm was created from docker cloud with azure integration.
Can I attach a single data disk to all worker VM's in an azure swarm cluster?
adding a single datadisk to all worker VM's is not possible. As Azure is not able to provide a 'shared disk' facility. The only things which comes close to it is the usage of Azure Files see herr --> https://learn.microsoft.com/en-us/azure/storage/storage-how-to-use-files-linux
for further details
I am a recent grad and wanted to learn about doing web application using AWS. I have gone through the documentation and ran their sample Travel Log application Successfully.
But still I am not clear about the terminologies used. can anyone explain me the difference between Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), Amazon SimpleDB in simple words.
I am looking to come up with a web app that has a signin page and people posting some text there. may i know what services of amazon would be required for me to build this app.
Thanks
Amazon Simple Storage Service (S3) is for load static content , maybe images, videos, or something you want to save, You could think of it like a hard drive for storage.
Amazon Elastic Compute Cloud: ( EC2) basically is your Virtual Operative System, you can install whatever OS you want (Debian, Ubuntu, Fedora, Centos, Windows Server, Suse enterprise). ( if your application uses server side processing this will be its home)
Amazon Simple DB, is a no-sql database system, that you could use for your aplications, and Amazon gives you as a service, but if you want to use something more, you could install yours on EC2, or use RDS for Database server (MySql for example)
If you want to know more, there are some books, like: "programming Amazon EC2" or see Amazon screencast at http://www.youtube.com/user/AmazonWebServices or its presentation on http://www.slideshare.net/AmazonWebServices
Amazon Simple Storage Service (Amazon S3)
Amazon S3 (Simple Storage Service) is a scalable, high-speed, low-cost web-based service designed for online backup and archiving of data and application programs. It allows to upload, store, and download any type of files up to 5 TB in size. This service allows the subscribers to access the same systems that Amazon uses to run its own web sites. The subscriber has control over the accessibility of data, i.e. privately/publicly accessible.
Amazon Elastic Compute Cloud (Amazon EC2)
Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) cloud. Using Amazon EC2 eliminates your need to invest in hardware up front, so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic.
Amazon SimpleDB
Amazon SimpleDB is a highly available NoSQL data store that offloads the work of database administration. Developers simply store and query data items via web services requests and Amazon SimpleDB does the rest.
Unbound by the strict requirements of a relational database, Amazon SimpleDB is optimized to provide high availability and flexibility, with little or no administrative burden. Behind the scenes, Amazon SimpleDB creates and manages multiple geographically distributed replicas of your data automatically to enable high availability and data durability. The service charges you only for the resources actually consumed in storing your data and serving your requests. You can change your data model on the fly, and data is automatically indexed for you. With Amazon SimpleDB, you can focus on application development without worrying about infrastructure provisioning, high availability, software maintenance, schema and index management, or performance tuning.
For more information, go through these:
https://aws.amazon.com/simpledb/
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html
https://www.tutorialspoint.com/amazon_web_services/amazon_web_services_s3.htm
Amazon S3 is used for storage of files. It is basically like the hard drives like on your system you use C or D your files. If you are developing any application you can use S3 for storing the static files or any backup files.
Amazon EC2 is exactly like your physical machine. Only difference is EC2 is on cloud. You can install and run software, applications store files exactly you do on your physical machines.
Amazon Simple DB is a a database on cloud. you can integrate it with your application and make queries.