I have node.js an application running on several instances under a load balancer.
Each instance is homogeneous (same code running on different servers). Users can be placed on different servers by the load balancer.
The problem is that when 2 users need to interact together, they need to be on the same instance (because they need to communicate together through websockets on the same server). So, I will need to swap the user into another server. I don't know which 2 users are going to interact together, so I don't know before hand which users need to be on the same server.
I haven't seen any configuration in any load balancers to handle this case. I will probably need to create one.
I can also try to use a cache (or any memory accessed by the all the servers), so the users that need to communicate together can do it through the cache, but I'm not sure if that's the best route.
I will appreciate any advice.
Related
I have 2 geographically separately hosted AEM (adobe experience manager) instances under the same TLD but with separate sub domains.
For example www.foo.com (instance 1) and www2.foo.com (instance 2)
Is it possible to have both these AEM instances appear under the same TLD? For example something like :
www.foo.com/instance1/ and www.foo.com/instance2/
Any help appreciated!
Yes, this sort of thing is done often using network tools. Typically a website that is built to handle a load will have some sort of load balancer in front of it. The load balancer would sit even ahead of the dispatchers in the overall flow. With a load balancer you can specify routing rules (such as an irule with an F5 load balancer) that will cause the load balancer to send traffic to different places based on the rules you set up--such as the differences in the initial folder structure of the URLs. Check out some articles on irules for more background, such as https://devcentral.f5.com/articles/the101-irules-ndash-introduction-to-irules.
The same can also be done via content delivery networks (CDNs). Ultimately, what you are looking to do must be done at some network layer before the request actually hits an AEM server. The AEM instances themselves won't know that other instances exist. They will just respond to the requests that reach them, and it will be up to the routing layers in the network upstream from them to determine which HTTP requests go to which AEM servers.
See also:
http route url parts to different server
server
Forward specific urls on same domain to different servers
I came across the "source" load balancing algorithm in HAProxy, which ensures that a user will connect to the same server, by choosing server based on a hash of the source IP.
Why and when is it important for a user to connect to the same server? I cannot think of a reason, assuming that all candidate servers serve identical content.
Furthermore, if there was the need for a user to always connect to the same server, then wouldn't load balancing be completely irrelevant for this user?
It is important for a user to connect to the same server if we want to achieve session persistence.
For example, when talking about a HTTP session, there are information/variables (think about a shopping cart) specific to the session in question.
This dynamic information is not shared by the candidate servers in case they are not configured to do so and it is simpler to deal with it at the load-balancing level.
The preferred way to deal with this in HAProxy is by using cookies, but this only works in HTTP mode. HAProxy offers the source load balancing algorithm, in case cookies can't be used. This can be used in TCP mode or with HTTP clients that refuse cookies.
Load balancing will be irrelevant for the user in question right, until the cookie expires. But we generally need load balancing when dealing with many users so that they can be served by multiple servers with each user sticking with one of them.
I am building a WCF Service, and for optimum connectivity for the users, I was going to run it on 3 or 4 servers on different internet connections. How would I setup the client to connect to the service, either select one by random, or a designated server. If its the former, if the service is down for whatever reason, can it automatically move onto the next one?
If you want to do it without purchasing a hardware based load balancer, you can do this via Windows Network Load Balancing, your clients will point to a virtual IP which will be distributed to multiple servers inside your network. There are many load balancing solutions that come at a price, but this one can be accomplished given you have a windows infrastructure with a couple of servers.
what's the best way to achieve high availability for a dynamic website? If I create a second copy on another server and do not wish to use a load balancer since it will mess up user sessions, what are the best alternatives?
You can store session data in a database instead, which gets around that problem, then you can round-robin the requests to the application servers.
(Good) Load Balancers can be configured to be "sticky" which means they send requests from the same IP to the same server each time.
Even if you have a load balancer sitting infront of two backend webservers, you just move the single point of failure onto the load balancer instead of the webserver. So your application would still not be highly available.
I highly recommend using a load balancer and at least a pair of web servers. At work, we use HA Proxy, which is fully capable of ensuring sessions are 'sticky', and are sent to the same web server unless it goes down, where it will fail over.
To make your load balancer highly available, you can set up two load balancing servers which are a mirror image of each other. Assign a single virtual IP to both of your load balancers. Write a script that will poll the other server to check if it's down; if it's down, have that script pick up that virtual IP address. The script should be running on both servers.
This link describes one way of managing a virtual IP address. Similar articles have been written for a large number of linux distros, but they are all based on the same method.
Loadbalancers. They should be configured in such a way that they can handle the sessions. Maybe by sending the same ip to the same backend every time. Or store them inside a database, or some shared memory if it needs to be really fast for some reason i haven't thought of.
From what I understand, if you have multiple web servers, then you need some kind of load balancer that will split the traffic amongst your web servers.
Does this mean that the load balancer is the main connecting point on the network? ie. the load balancer has the IP address of the domain name?
If this is the case, it makes it really easy to add new hardware since you don't have to wait for any dns propogation right?
There are several solutions to this "problem".
You could round-robin at the DNS-level. I.e. have www.yourdomain.com point to several IP-addresses (well all your servers).
This doesn't give you any intelligence in the load balancing, but the load will be more or less randomly distributed, but you wouldn't be resilient to hardware failures as they would still require changes to DNS.
On the other hand you could use a proxy or a loadbalancing proxy that has a single IP but then distributes the traffic to several back-end boxes. This gives you a single point of failure (the proxy, you could of course have several proxies to defeat that problem) and would also give you the added bonus of being able to use some metric to divide the load more evenly and intelligently than with just round-robin dns.
This setup can also handle hardware failure in the back-end pretty seamlessly. The end user never sees the back-end, just the front-end.
There are other issues to think about as well, if your page uses sessions or other smart logic, you can run into synchronisation problems when your user (potentially) hits different servers on every access.
It does (in general). It depends on what server OS and software you are using, but in general, you'll hit the load balancer for each request, and the load balancer will then farm out the work according to the scheme you have in place (round robin, least busy, session controlled, application controlled, etc...)
andy has part of the answer, but for true load balancing and high availability you would want to use a pair of hardware load balancers like F5 bigips in an active passive configuration.
Yes your domain IP would be hosted on these devices and traffic would connect firstly to those devices. Bigips offer a lot of added functionality including multiple ways of load balancing and some great url rewriting, ssl acceleration, etc. It also allows you to run your web servers on a seperate non routable address scheme and even run multiple sites on different ports with the F5's handling the translations.
Once you introduce load balancing you may have some other considerations to take into account for your application(s) like sticky sessions and session state but that is a different subject