I have ignite cluster running on cloud, i want to use public IP for query and other things and private IP for sync within cluster (gossiping).
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>[public ip]:port</value>
<value>[public ip]:port</value>
<!--<when enable private ip not accessible from outside>--->
<!--<value>[private ip]:port</value>
<value>[private ip]:port</value>-->
</list>
</property>
</bean>
How should i utilize benefits of private ip ? as i can see huge spike in public bandwidth usage.
I think that you should:
Configure Communication and Discovery to use private IP as local host.
Configure ConnectorConfiguration/ClientConnectorConfiguration to use public IP where applicable.
Use JDBC/Thin Client to access from outside.
Related
I'm trying to set up a distributed cache using Ignite and my java app through a thin client in a Kubernetes environment.
In my Kubernetes cluster, I have 2 pods with the java app and 2 pods of ignite. For the java pods to communicate with ignite pods, I have configured a thin client to connect with the ignite kubernetes service. With this configuration, I was expecting that the load balancing was on the kubernetes side. Here's what I have done in java code:
ClientConfiguration cfg = new ClientConfiguration()
.setAddresses("ignite-service.default.svc.cluster.local:10800")
.setUserName("user")
.setUserPassword("password");
IgniteClient igniteClient = Ignition.startClient(cfg);
While storing and getting objects from ignite, I deleted one of the ignite pods and, after a while, I was getting errors saying that "Ignite cluster is unavailable":
org.apache.ignite.client.ClientConnectionException: Ignite cluster is unavailable
With this behavior, I assume that the method setAddresses in ClientConfiguration class stores one of the IPs of the pods and channels all communication to that pod.
Is this what's happening in this method?
Ignite version 2.7
Kubernetes version 1.12.3
You need to pass several IP addresses to enable the failover (aka. automatic reconnect) on the thin client end. Find more details here.
Although you might have resolved the issue since the question was posted a long time back, but still putting an answer here for others.
With the Apache Ignite version(2.7+), you can modify your deployment to use Kubernetes IP Finder. With this Kubernetes will take care of discovering and connecting all server and client nodes.
TcpDiscoveryKubernetesIpFinder module will help you achieve this.
This is the discovery SPI that needs to be added to your configuration (Replace with appropriate Namespace and Service Name)
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
<constructor-arg>
<bean class="org.apache.ignite.kubernetes.configuration.KubernetesConnectionConfiguration">
<property name="namespace" value="default" />
<property name="serviceName" value="ignite" />
</bean>
</constructor-arg>
</bean>
</property>
</bean>
</property>
Official documentation can be found here - https://ignite.apache.org/docs/latest/installation/kubernetes/amazon-eks-deployment
Ignite uses one network card during a re-balance. It should use multiple.
Our cluster using more than 1gbps bandwidth during re-balance, so we tried network bonding but ARP cache needs to be refresh. Instead we want to use separate network devices on a virtual machine. But ignite uses one of them per re-balancing. Virtual machines are centos7. ignite is 2.7.0-1
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>ip1:47500..47509</value>
<value>ip2:47500..47509</value>
<value>ip3:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
We expect ignite to re-balance trough ip1, ip2, ip3 at the same time.
UPDATE
We've made a bonded virtual network device with combining multiple devices, unfortunately it has required some down time. Problem solved.
Two options here:
Can you create a virtual NIC which will join two physical ones together? I think this should be doable.
Failing that, you can have two nodes per VM, one with localAddress nic1 and other with localAddress nic2. Please note that you should define localAddress on TcpCommunicationSpi since that's where traffic is. Share RAM fairly between those two nodes.
Maybe you could also have a custom TcpCommunicationSpi which will use two NICs, but I'm not sure if traffic will be distributed fairly even then.
We had a peculiar situation where we created two Ignite clusters (20 nodes and 20 nodes) in the same network (same subnet).
The issue what we are having is that data is colliding between both ignite cluster and unable to differentiate. We have found that we need to configure multicastgroup to diversify data set at each cluster level. But the question is, what IP we need to refer in the multicastgroup and what exactly that IP is referring to? is that referring to subnet IP?
ex: default-config.xml
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
<property name="multicastGroup" value="228.1.2.4"/>
<property name="addresses">
<list>
<value>IP:port..port</value>
</list>
</property>
</bean>
Appreciate your help on this.
If you want to use ONLY multicast, then you can omit the addresses property overall. TcpDiscoveryMulticastIpFinder allows to use both multicast and statically provided IPs. The addresses property is for the latter.
We were able to succeed by eliminating multicastgroup completely and only defining static IPs for each cluster seperately. Thanks for the help.
Apache Ignite Version is: 2.1.0
I am using TcpDiscoveryVmIpFinder to configure the nodes in an Apache Ignite cluster to setup a compute grid. Below is my configuration which is nothing but the example-default.xml, edited for the IP addresses:
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
<!-- <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder"> -->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>xxx.40.16.yyy:47500..47509</value>
<value>xx.40.16.zzz:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
If I start multiple nodes on individual machine, the nodes on respective machines discover each other and form a cluster. But, the nodes on the remote machines do not discover each other.
Any advise will be helpful...
First of all, make sure that you really use this config file and not a default config. With default configuration, nodes can find each other only on the same machine.
Once you've checked it, you also need to test that it's possible to connect from host 106.40.16.64 to 106.40.16.121(and vice versa) via 47500..47509 ports. It's possible that there is a firewall blocked connections or these ports is simply closed.
For example, it's possible to check it with netcat, run this from 106.40.16.64 host:
nc -z 106.40.16.121 47500
I have a server,and have a container on server, on which started Ignite node(s).
And know that server configs (IP,container port etc.).
And want to connect(find) to this node from my PC(from Intellij Idea).
Namely I want to start another Ignite to which must connect to node on server.
How do my new starting node configuration?
With TcpDiscoverySpi or CommunicationSpi and how with IP and port.
You need to start a node on your PC with a configuration where IP finder, that is set for TcpDiscoverySpi will contain list of IPs and ports of your remote cluster.
Most likely it will be more than enough to configure static IP finder on your side.
Simply you can create the static IP finder the way below and set this discovery bean into configuration of all the nodes (servers and clients)
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>server_1_ip:47500..47509</value>
<value>server_2_ip:47500..47509</value>
<value>server_3_ip:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>