Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 months ago.
Improve this question
Can somebody explain the difference in tooling between Crossplane and Cluster API from the perspective of a managed kubernetes platform provision ?
https://github.com/kubernetes-sigs/cluster-api
I was using crossplane for sometime to create k8s clusters and recently got to know that Cluster API is also having the same capability to provision k8s clusters.
Cluster API
Cluster API is one of the projects in CNCF that you can use the create and manage the Kubernetes clusters. it's more like Kubeadm, kind, k3s, minikube(not for prod usecase) with YAML config & CLI.
You can create the YAML file and provision the Kubernetes cluster and manage it with that. There are different providers(AWS, AKS, GCP) available so you can use a provider-set service account or Access key secret and once you apply YAML to cluster API it will create a K8s cluster based on config and provider.
So with this, you can manage multiple Kubernetes clusters.
With cluster API YAML config you can create/manage K8s cluster on AWS, GCP, On-prem etc.
Crossplane
Consider crossplane as a deployment or service you are running on Kubernetes first.
You pass YAML config to cross-plane service based on rules it will create/manage the resources outside on cloud providers. It will also create/Manage the RDS, CloudSQL instances, Kubernetes clusters and other resources that the cloud provider supports.
It has also concept of provider(AWS, GCP, AKS)
Crossplane uses Kubernetes as the underlying platform to expose these features and workflows to manage cloud infrastructure outside of Kubernetes, such as data centers, the cloud or the edge. It installs a set of components called providers by extending Kubernetes to manage resources from cloud providers
Cluster API is a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management.
Kubernetes container orchestration technology has swept the enterprise world, with 75% of organizations running containers in production soon (most orchestrated by Kubernetes). Kubernetes delivers a cloud-native platform for application delivery, runtime, and scaling. While these are huge benefits for scale out applications, providing all these capabilities comes at the expense of the complexity of the platform itself. These complexities can be categorized as
Complex and heavyweight lifecycle management: significant time and resources needed to provision, upgrade, and maintain the control planes of clusters. Each cloud has a different set of IaaS primitives that need to be cobbled together to make a “cluster”.
Intricate and involved integrations for basic components such as storage, networking, security, and others.
Blockquote
Difficult and not always consistent multi-cluster management and operations.
Cluster API
Provisioning Kubernetes clusters is never an easy task. When there are 1000+ clusters, definitely want to have a standardized approach to ease tasks. Some of might know tools like kOps, Kubespray. Imagine Cluster API as their alternative solution, but more powerful!
According to the official page, "Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters."
Here are some highlighted points of Cluster API:
Pure YAML-based. Kubernetes style. Super handy.
Support any mainstream infrastructure provider. Provision your Kubernetes clusters in cloud/on-premise environments in the same place.
Managed Kubernetes services support. AWS EKS, Azure AKS, GCP GKE all are supported.
Bring your own infrastructure. Reuse existing infrastructures. Focus on provisioning Kubernetes clusters.
Crossplane
Crossplane uses Kubernetes as the underlying platform to expose these features and workflows to manage cloud infrastructure outside of Kubernetes, such as data centers, the cloud, or the edge.
It installs a set of components called providers by extending Kubernetes to manage resources from cloud providers. Providers build on the Kubernetes API, along with the custom resources and controllers, and expose declarative APIs to manage cloud provider resources.
Crossplane is geared toward platform teams and operators looking to build abstractions and easier-to-consume infrastructure components based on an organization's needs. It has a few concepts, such as CompositeResourceDefinition and Composition, that enable teams to extend and build on top of the default resources exposed for a cloud provider.
First of all lets understand what is Crossplane and what is Cluster API
Cluster API
Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters.
Provide declarative APIs
Give the tools to simplify provisioning, upgrading and operating multiple clusters
Helps to manage the lifecycle (create, scale, upgrade, destroy) of Kubernetes-conformant cluster (uses a declarative API)
Give you the ability to work on different environments (on-premisses and cloud)
Can find this and more about kubernets cluster api at https://cluster-api.sigs.k8s.io/
Crossplane
Crossplane is a framework for building cloud native control planes without needing to write code. It has a highly extensible backend that enables you to build a control plane that can orchestrate applications and infrastructure no matter where they run, and a highly configurable frontend that puts you in control of the schema of the declarative API it offers.
I've found a few interesting posts about Crossplane vs Kubernetes API's
https://blog.crossplane.io/crossplane-vs-cloud-infrastructure-addons/
https://www.infracloud.io/blogs/cluster-provisioning-using-crossplane/
Its not much but hope this helps
Related
I have some questions according to https://docs.konghq.com/2.0.x/clustering
I’ll really appreciated if someone help me.
1)according to Clustering Reference I need a load balancer , could you please introduce me a free one which I can use in front of my Kong nodes?
2)I still don’t know is it better to implement kong nodes in different VMs or in Docker using docker-compose file for a fully production environment ?
Best Regards,
I think both your questions are highly dependent of your tech stack / architecture.
Regarding the load balancing question, I can think of several options for different options:
DNS Load Balancing, which depends on client side load balancing
Services in an Kubernetes/OpenShift environment, which provide load balancing across a bunch of pods
AWS Load Balancers, if you deploy Kong directly on EC2 machines. (I am sure other cloud providers have simar concepts)
Whether you deploy Kong on a VM or as a Docker Container is quite hard to answer. It depends on your tech stack you already have in place and on your requirements (see https://docs.konghq.com/2.0.x/sizing-guidelines/). However, I would not recommend to use docker-compose for this use case. If you decide for a Docker-based solution you should take a look at container management solutions such as Kubernetes or OpenShift. There you have solved the management of your Kong containers (such as how many replicas are running and what happens if one replica is failing) and you have solved the load balancing issue (by using Kubernetes/OpenShift services objects).
I see Fargate as a good service for deploying a Docker Compose based stack, but I was wondering if it is any good for "long-running" web services, not just ones where you need dynamic scaling and undeterminate workloads (e.g. containers that are created and die on demand).
That depends on your use-case. ECS lets you quickly deploy containerized applications. With Fargate we don't need to manage the underlying infrastructure (say server-less approach for containers!). Fargate is suitable for long-running apps, microservices, and batch jobs.
Few of my observations on Fargate are:
Fargate storage is ephemeral - We cannot store container data in disk such as volumes. (although Fargate provides 10 GB of volume mounts that is nonpersistent empty storage.)
Logs can be sent to Cloudwatch using awslogs driver. Recently AWS announced the support for Splunk log driver.
Fargate uses only awavpc network mode.
Fargate supports environment variables. Environment variables are the only way to pass parameters to the container.
Is there a way to do A/B or canary deployments with Lagom Persistent Entities?
I'm trying to plan out how we can build and deploy reactive microservices and I like the look of Lagom, but I can't see how we'd roll out new versions as canary or a/b testing releases. It seems like the stateful nature of the persistent entities would make that impossible?
Assuming it's possible, I'd also like to figure out how to use Istio on Kubernetes to deploy and manage these new releases, so any pointers on that would be great.
In my view, the Lagom framework and its associated tooling obviates many benefits of service meshes like Istio. Orthogonal to meshes is deployment; that said, there is documentation for deploying Lagom services using Kubernetes using Lightbend Orchestration.
The docs show an explicit example of Kubernetes canary deployments.
For a large online application, use k8s to run it. The scale maybe daily activity user 500,000.
The application inside k8s need messaging feature - Pub/Sub, there are these options:
Kafka
RabbitMQ
Redis
Kafka
It needs zookeeper and good to run on os depends on disk I/O. So if install it into k8s cluster, how? The performance will be worse?
And, if keep Kafka outside of the k8s cluster, connect Kafka from application inside the k8s cluster, how about that performance? They are in the different layer, won't be slow?
RabbitMQ
It's slow than Kafka, but for a daily activity user 500,000 application, is it good enough? If so, maybe it's a good choice.
Redis
It's another option. Maybe the most simple one. But from the internet I got that it will lose message sometimes. If true, that's terrible.
So, the most important thing is, use Kafka(also with zookeeper) on k8s, good or not in this use case?
Yes, running Kafka on Kubernetes is great. Check out this example: https://github.com/Yolean/kubernetes-kafka. It includes ZooKeeper and Kafka as StatefulSets.
PS. Running any of the services in your question on Kubernetes will be pleasant. You can Google the name of the service and "kubernetes" and find example manifests. Many examples here: https://github.com/kubernetes/charts.
For Kafka, you can find some suggestion here. Kubernetes 1.7+ supports local persistent volume, which may be good for Kafka deployment.
You can also take a look to the following project :
https://github.com/EnMasseProject/barnabas
It's about running Kafka on Kubernetes and OpenShift as well. It provides deploying with StatefulSets with persistent volumes or just in memory (for developing or just testing purpose). It provides deploying for Kafka Connect and Prometheus metrics as well.
Another simple configuration of Kafka/Zookeeper on Kubernetes in DigitalOcean with external access:
https://github.com/StanislavKo/k8s_digitalocean_kafka
You can connect to Kafka from outside of AWS/DO/GCE by regular binary protocol. Connection is PLAINTEXT or SASL_PLAINTEXT (user/password).
Kafka cluster is StatefulSet, so you can scale cluster easily.
We are using apache ignite as a IMDG in our micro services environment.
For scalability and load balancing we are considering to use a service registry like eureka or consul which is supported by spring cloud for the deployed micro services.
There is a concept of service grid providing support for node singleton and cluster singleton in apache ignite.
I also see WCF,weblogic and JBoss to having the same sort of features.
I am trying to understand what these service grids are and if i can use them to achieve the same benefits as the eureka service registry provided by netflix and supported by spring cloud.
Can someone guide if i can achieve the same using service grid in apache ignite.
No, you cannot use Apache Ignite Service Grid for the same purposes as Eureka. Eureka is used for load balancing and service discovery over WAN. Using Ignite clusters spanning over multiple AWS zones and remote client machines is not the most efficient way of using it.
More information on Ignite Service Grid can be found here - http://apacheignite.gridgain.org/docs/service-grid
Thanks!
UPD (for the 1st comment):
You cannot (in most cases) span and effectively use Ignite over WAN networks with high latencies and lower throughput characteristics.
As far as local clusters in non-cloud environments - go ahead! This is the best environment for systems of such kind.