I have a microservice architecture that has several Eureka clients, a Eureka registry, a cloud gateway with Circuit breakers and I want to enable load balancing. The system shall be deployed on AWS. I found Zuul/Ribbon to be a tool that may be fitting - correct me if I am wrong.
I am wondering, however, how Ribbon decides when to launch a new instance of a microservice. My use case for load balancing would specifically be in one of the eureka clients. That microservices allows to register up to X processes (added at run time by the user) and my idea would be to launch a new microservice instance when that X "slots" are full.
Is Zuul the tool I am looking for this task? If yes, how can Zuul load balancing on my "custom condition" or is there a better solution?
I am expecting an application that scales up to N microservices where as each microservice can run up to X processes simultaneously and the system shall spawn a new instance of the microservice when that slots are "full"
Related
I'm fairly new to the realm of microservices but know basics about load balancing. I recently read an article about the microservices: Enough with the microservices.
There it's mentioned that both the microservices and load balancers have clusters/different VM's for deploying many copies of application but in the case of microservices, we have a separate database in contrast to load balancers which backs a single database. Is it the only difference between them?
Here's the quoted text:
"multiple copies of the same microservice can be deployed in order to
achieve a form of scalability. However, most companies that adopt
microservices too early will use the same storage subsystem (most
often a database) to back all of their microservices. What that means
is that you don’t really have horizontal scalability for your
application, only for your service. If this is the scalability method
you plan to use, why not just deploy more copies of your monolith
behind a load balancer? You’ll accomplish the same goal with less
complexity."
You can not compare Micro-services with load balancer... you should compare it with monolithic or SOA architecture.
In monolithic approach you mainly have only one database for the whole system and a monolithic application as a single project for your business.
monolithic is single unit But SOA is a coarse-grain approach and Microservice is fine-grain approach. In microservice architecture instead of designing a monolithic system you design different micro-services around your business capabilities and base on your domain and bounded-context.
each micro-services may have their own database. for e.g. order micro-service may have mysql database, recommendation micro-service may have Cassandra database and user-search micro service may have Elasticsearch or SOLR database.
In microservices each micro-service can talk to another base on two different communication style:
Sync (Rest is suggested)
Async (via message brokers like Kafka, RabbitMQ, ActiveMQ or NATS
and etc.)
Scaling up-down in micro-services architecture is much easier than monolithic systems and you can even change a part of system and redeploy it independently without affecting the whole system.
Also micro-services adhere to let-it-crash paradigm and with using EIP patterns like Circuit-Breaker you can let user think system is always up and working and Base on CAP theorem you can have high-available system by compensating for consistency and having Eventual Consistency according to BASE instead of ACID
For load balancing Client-side Load Balancing with Ribbon devised by Netflix is very viable approach.
Also with using NginX, Docker Swarm and kubernetes you can implement load balancing.
In a nutshell there is nothing to do about comparing Microservices with Load balancer.
Here's the (hopefully) simplest answer to your question:
Microservices are a different (micro-) application each. Each with its own application logic and database.
Load Balancers are usually used to distribute client requests to a cluster of instances of the same application.
That means: You can also use a load balancer to distribute requests for a microservice that is deployed in a cluster with many instances. But a load balancer can also be used to distribute requests to many instances of a large monolithic application (as opposed to micro).
The probably best overview for what Microservices are supposed to be.
We have scaled out all sevices in our system by having more than one instance of them registered in eureka service registry.
Also, they are also proxied by a zuul server in the front.
My question is how can we ensure the scalability of zuul proxy when accessed from clients.
One solution i can think of is having multiple instances of the proxy registered in eureka registry. But if that is done how do we decide on which of the instances would be exposed to the clients.
We faced the same issue in our application, having multiple instances of multiple types of micro-service-type applications on our backend. All servers registered with Eureka. The problem is that we also had multiple security gateways configured (based on the architecture described in this excellent tutorial: https://spring.io/guides/tutorials/spring-security-and-angular-js/).
Eventually we decided to use a hardware http load balancer that calls our security gateways in a round-robin approach (our solution is on-prem).
We use Redis with #EnableHttpRedisSession annotation to have the spring session synced across all the servers, so the http load balancer does not have to deal with sticky sessions or stateful considerations. It just does a round-robin to all the security gateways. It doesn't matter if the load balancer hits SG1, SG2 or SG3, they all share the same session information coming from Redis (which is also configured for fail-over with Redis Sentinel).
Recently several service discovery tools have become popular/"mainstream", and I’m wondering under what primary use cases one should employ them instead of traditional load balancers.
With LBs, you cluster a bunch of nodes behind the balancer, and then clients make requests to the balancer, who then (typically) round robins those requests to all the nodes in the cluster.
With service discovery (Consul, ZK, etc.), you let a centralized “consensus” service determine what nodes for particular service are healthy, and your app connects to the nodes that the service deems as being healthy. So while service discovery and load balancing are two separate concepts, service discovery gives you load balancing as a convenient side effect.
But, if the load balancer (say HAProxy or nginx) has monitoring and health checks built into it, then you pretty much get service discovery as a side effect of load balancing! Meaning, if my LB knows not to forward a request to an unhealthy node in its cluster, then that’s functionally equivalent to a consensus server telling my app not to connect to an unhealty node.
So to me, service discovery tools feel like the “6-in-one,half-dozen-in-the-other” equivalent to load balancers. Am I missing something here? If someone had an application architecture entirely predicated on load balanced microservices, what is the benefit (or not) to switching over to a service discovery-based model?
Load balancers typically need the endpoints of the resources it balances the traffic load. With the growth of microservices and container based applications, runtime created dynamic containers (docker containers) are ephemeral and doesnt have static end points. These container endpoints are ephemeral and they change as they are evicted and created for scaling or other reasons. Service discovery tools like Consul are used to store the endpoints info of dynamically created containers (docker containers). Tools like consul-registrator running on container hosts registers container end points in service discovery tools like consul. Tools like Consul-template will listen for changes to containers end points in consul and update the load balancer (nginx) for sending the traffic to. Thus both Service Discovery Tools like Consul and Load Balancing tools like Nginx co-exist to provide runtime service discovery and load balancing capability respectively.
Follow up: what are the benefits of ephemeral nodes (ones that come and go, live and die) vs. "permanent" nodes like traditional VMs?
[DDG]: Things that come quickly to my mind: Ephemeral nodes like docker containers are suited for stateless services like APIs etc. (There is traction for persistent containers using external volumes - volume drivers etc)
Speed: Spinning up or destroying ephemeral containers (docker containers from image) takes less than 500 milliseconds as opposed to minutes in standing up traditional VMs
Elastic Infrastructure: In the age of cloud we want to scale out and in according to users demand which implies there will be be containers of ephemeral in nature (cant hold on to IPs etc). Think of a markerting campaign for a week for which we expect 200% increase in traffic TPS, quickly scale with containers and then post campaign, destroy it.
Resource Utilization: Data Center or Cloud is now one big computer (compute cluster) and containers pack the compute cluster for max resource utilization and during weak demand destroy the infrastructure for lower bill/resource usage.
Much of this is possible because of lose coupling with ephemeral containers and runtime discovery using service discovery tool like consul. Traditional VMs and tight binding of IPs can stifle this capability.
Note that the two are not necessarily mutually exclusive. It is possible, for example, that you might still direct clients to a load balancer (which might perform other roles such as throttling) but have the load balancer use a service registry to locate instances.
Also worth pointing out that service discovery enables client-side load balancing i.e. the client can invoke the service directly without the extra hop through the load balancer. My understanding is that this was one of the reasons that Netflix developed Eureka, to avoid inter-service calls having to go out and back through the external ELB for which they would have had to pay. Client-side load balancing also provides a means for the client to influence the load-balancing decision based on its own perspective of service availability.
If you look at the tools from a completely different perspective, namely ITSM/ITIL, load balancing becomes "just that", whereas service discovery is a part of keeping your CMDB up to date, and ajour with all your services, and their interconnectivity, for better visibility of impact, in case of downtime, and an overview of areas that may need supplementing, in case of High availability applications.
Furthermore, service-discovery only gives you a picture as of the last scan, and not near-real-time (of course dependent on which scanning interval you have set), whereas load balancing will keep an up-to-date picture of your application's health.
I am currently evaluating solutions for WCF Webservice HA with load balancing. I see 2 feasible approaches for the type of WS i am authoring which are
1) Using the RoutingService API / Class provided by .NET
2) Using a HTTP load balancer like nginx.
Which one is a better approach for WCF WS hosted on IIS.
It depends on a lot of factors. The main one being, is the load balancing requirement a pure availability/scalability driven requirement or is it a business requirement?
If you simply require scale, eg a round robin distribution, or high availability, eg. active/passive failover, and you already have a network load balancer in front of your servers, then I would definitely use that.
The simple reason is that then it will be looked after by your infrastructure people, which is how it should be. Load balancing for scale/availability etc is not normally a development concern.
However, if you have a requirement for routing based on message content, eg routing of high priority calls to one endpoint only, or meeting call processing SLAs for different content, then this becomes a business requirement, because routing logic will then be determined on the business context of the call.
This most definitely is a development concern. In this instance I would certainly use the routing service to implement these various business cases.
Hope this helps you.
My management is evaluating non-Azure Microsoft Windows Service Bus (Azure is out of consideration for security reasons). It will be used to setup topic/subscription model with a number of WCF services with netMessagingBinding that we building, so I just have a few basic questions about that.
Are there any specific hardware requirements like dedicated server, dedicated database etc. for WSB to run in production environment?
It's easy to configure WCF service to listen on a specific topic subscription. Is there any way for WCF service to listen to multiple subscriptions?
Appreciate the answers.
You can install the service components and the databases all on one server (that is the default). However, for a number of reasons, we installed the services on a dedicated app server and then created the Service bus databases on an existing database server. The install package allows you to specify a different db server. Check this article for the minimum server requirements
Yes you can get one WCF service to listen to multiple subscriptions. You would need to create two (or more) System.ServiceModel.ServiceHost instances and then run them inside one process. For example we had one windows service running two ServiceHost's. Each host listened at a different queue and therefore implemented a different contract. This meant where queues were logically grouped we didn't need a new windows service per queue. You could do the same with subscriptions.
For question one, you will have to go through the exercise of hardware sizing. the good news is that WCF services can scale vertically, so you can add up servers if there were issues in handling client load.
To do hardware sizing you will have to make an estimate the expected load and then do performance/scalablity testing to figure the load bearing capacity of your serviceBus/services .
you could find a lot of resources for load testing like this one http://seroter.wordpress.com/2011/10/27/testing-out-the-new-appfabric-service-bus-relay-load-balancing/
once you do load testing and come up with the numbers, you can then do sizing using references like this one http://msdn.microsoft.com/en-us/library/bb310550.aspx