Load balancer PublicIPReferencedByMultipleIPConfigs error on restart - azure-container-service

Following along from the Use a static IP address with the Azure Container Service (AKS) load balancer documentation I have created a static IP and assigned it to the load balancer. This worked fine on the initial run, but now I am getting the following error and the external ip for my load balancer is stuck <pending> (personal info omitted):
Failed to ensure load balancer for service default/[...]: network.LoadBalancersClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="PublicIPReferencedByMultipleIPConfigs" Message="Public ip address /subscriptions/[...]/providers/Microsoft.Network/publicIPAddresses/[PublicIPName] is referenced by multiple ipconfigs in resource
As far as I can tell, this isn't referenced by multiple configs - just the load balancer service that I'm trying to run. Removing the loadBalancerIP option from my yaml file allows this to work but then I don't think the server address is static - which is not ideal for the applications trying to communicate with this container
Is this supposed to be happening? Is there a way to configure this so that the same IP can be reused after the container restarts?

Seeing as this issue appears to still be present, for anyone else stumbling upon this issue it seems that the Azure load balancer resource itself may be taking the first configured static IP address.
GitHub issue response:
the first public IP address created is used for egress traffic
Microsoft Docs:
Once a Kubernetes service of type LoadBalancer is created, agent nodes are added to an Azure Load Balancer pool. For outbound flow, Azure translates it to the first public IP address configured on the load balancer.
As far as I can tell, once you provision an IP address and configure an AKS load balancer to use it, that IP gets picked up by the provisioned load balancer resource in Azure. My best guess is that when Kubernetes attempts to provision a new load balancer with the same IP address, if the previous Azure load balancer still exists the IP config will fail as it's still in use.
Workaround was to provision an extra static IP (one specifically for the Azure load balancer resource, and one for the actual AKS load balancer service) to avoid conflicts. It's obviously not ideal but it solves the issue...

Related

Traefik, Metallb portforwarding

I'm having problems portforwarding traefik. I have a deployment in Rancher, where i'm using metallb with traefik to have ssl certs. applied on my services. All of this is working locally, and i'm not seeing any error messages in the traefik logs. It's funny because, at times, i am able to reach my service outside of my network, but other times not.
I have portforwarded, 80, 433, 8080 to 192.168.87.135
What am i doing wrong? are there some ports im missing?
Picture of traefik logs
Picture of the exposed traefik loadbalancer
IPv4 specifies private ip address ranges that are not reachable from the internet because:
The Internet has grown beyond anyone's expectations. Sustained
exponential growth continues to introduce new challenges. One
challenge is a concern within the community that globally unique
address space will be exhausted.
(source: RFC-1918 Address Allocation for Private Internets)
IP addresses from these private IP ranges are not accessible from the internet. Your IP address 192.168.87.235 is part of the class C private ip address range 192.168.0.0/16 hence it is by nature not reachable from the internet.
Furthermore you state yourself that it is working correctly within your local network.
A follow up question to this is: How can I access my network if it's a private network?
To access your local network you need to have a gateway that has both an internal as well as a public IP so that you can reach your network through the public IP. One solution could be to have a DNS name thats mapped to the public IP and is internally routed to the internal load balancer IP 192.168.87.235 with a reverse proxy.
Unfortunately I can't tell you why it is working occasionally because that would require far more knowledge about your local network. But I guess it could i.e. be that you are connected with VPN to your local network or that you already have a reverse proxy that is just not online all the time.
Edit after watching your video:
Your cluster is still reachable from the internet at the end of the video. You get the message "Service unavailable" which is in fact returned by traefik everytime you wish to access a non-healthy application. Your problem is that the demo application is not starting up after you restart the VM. So what you need to do next is to check why the demo app is not starting. This includes checking the logs of the pod and events of the failing pod.
Another topic I'd like to touch is traefik and what it actually does. First to only call Traefik a reverse proxy, while not false,is not the entire truth. Traefik in a kubernetes environment is an ingress controller. That means it is a reverse proxy configured by kubernetes resources, namely by the "Ingress" object or the "IngressRoute" object. The latter is a custom resource introduced by Traefik itself (read here for further informations) because it introduces andvanced options to configure traefik.
The reason I tell you this is because you actually have two ingress controllers installed in your cluster, "Traefik" and "nginx-ingress-controller" and you just need a single one.

What can we do when load balancer becomes the bottleneck?

I just started learning load balancers. Taking a server side application (http/https) load balancer as an example, I assume it listens a specific ip address, then forward the http requests to available servers based on its algorithm.
So is it possible for a load balancer to become a bottleneck? Because it's listening a specific ip address, all requests will first go to the single load balancer. So I think there could be a scenario where the amount of traffic is beyond the limit/capacity of the load balancer.
When it becomes a bottleneck, what can we do? Can we use multiple load balancers?
I think one possible solution is to use multiple load balancers and expose all the ips to clients. (This sounds like client side load balancing) So when a client wants to send a request, it can pick from the ip pool and then send a request to one of the load balancers. (For example, ZooKeeper could be used here.) Is this a working solution? Is there any other way to use multiple load balancers?
Thanks.
Ethan
Your last suggestion works with adding a little twist: The usual approach is to publish the load balancer IP addresses under the same domain name.
This is called DNS load balancing. Clients will ask for the IP resolution for your load balancer's domain name and they will get different IP addresses on a round-robin fashion.
To configure DNS load balancing you have to add multiple A records for your load balancer's domain name to your DNS configuration. Here you can find an example guide for that.

Cannot add VM to Standard Azure Load Balancer

I have an Azure standard internal load balancer inside a VNET that contains several virtual machines. Two of the VMs are not listed as options when I want to add them to a back end pool of the load balancer. They are were created under ARM and not included in any other load balancer pool. They are also in the same VNET that is associated to the backend pool.
If I create a basic load balancer, I can see them and successfully add them to the pool. Is there documentation on the VM requirements that must be met before you can add a VM to a pool within a standard load balancer?
When you add the backend pool, you will see Only VMs in the same region with standard SKU public IP or no public IP can be attached to this load balancer.
In this case, you can randomly disable the public IP address from the virtual machine---network interface---Ip configurations---ipconfig1---disabled---save. Then you can add the desired VMs to backend pool again.

GCE: Block direct HTTP/S access. However, should only be available through N/W Loadbalancer

We are in the process of configuring and hosting our services on Google Cloud Services. We are using few instances of GCE. We also have a network load balancer. What I want to do is to block all direct HTT/S requests to individual instances and only be available via N/W load balancer.
Also, N/W load balancer will be mapped to a DNS.
By default GCE will not allow any ports be accessible from outside the network unless you create a firewall rule to allow.
One way is to remove the external IP for all the instances and use only one server as gateway instance with external IP to go to all the other instances. Make sure you add firewall rule allowing access from intended source to the gateway instance. This way only gateway instance is exposed to intended sources or external world based on firewall rule.
Then you can create network load balancer adding up instances that have no direct external access.
This is one way these are more ways to achieve.

Weblogic load balancing and request re-routing to another server

I'm totally new to clustering and load balancing.
What I'm trying to do is "Deploy Application on a Cluster which contains 2 managed servers. Now, If one of the managed server goes down, request should be redirected to another server which is Up."
For Example:
I've 2 managed servers (M1:7021 and M2:7022)
And I've a Cluster C1 having M1 and M2.
And I've an Application App1 deployed on C1 and a Data Source deployed on C1.
Application App1 is working fine.
The way through which I'm accessing application is:
http://10.184.111.11:7021/App1/
AND
http://10.184.111.11:7022/App1/
Now, Suppose if M1(7021) goes down, and request is coming like
:7021/App1/
Then, it should be redirected to :7022/App1/
Any help is highly appreciated. Thanks!
I believe you will need a load balancer (or a software equivalent) to sit above the weblogic servers and direct traffic down to those servers.
The idea being that you access your application on http://loadBalancer.com/App and then the Load Balancer forwards your request onto either one of weblogic servers. Meanwhile in the background the load balancer is continually performing health checks on the two weblogic servers to see if they are running.
In the event that one of the weblogic servers go down, the load balancer will mark it as inactive and forward all traffic to the weblogic server still running. Once the failed weblogic server has come back online the load balancer will begin routing traffic back through it.
#Garreth Well, in fact WebLogic DOES provide an internal load balancer. You are supposed to use OHS or Apache for load balancing in production environments, but for development, httpclusterservlet works great.