For example, setting the remote{} configuration, does that also set the transport that is used internally for cluster communication, for example, the heartbeat messages.
I am not asking for any use case purpose I am asking so I better understand what's happening behind the scenes.
At the moment (Akka.NET 1.3) uses its own protocol for remote communication on top of TCP connection - only a single connection is used by every node-to-node connection. This video discusses it in greater detail.
In future, it will probably change to match JVM version of akka - two major ideas are:
"lanes": multiple connections for each pair of nodes, to avoid head-of-line blocking, that is inherent problem of TCP.
Add support for other protocols, such as Aeron, which is also supported by akka on JVM.
Related
I have a configuration where 3 applications run on 3 different Virtual Machine's and they must communicate via DDS i.e. RTPS protocol.
The configuration is as follows :
ROS2 based ADAS functions
Simulation Tool
Python/Tensorflow based machine learning functions
All 3 need to be on different VMs.
It is not possible at our end to allow multicasting for the MS AZURE VM and our network.
Here are some questions :
Is it still possible to communicate via DDS ?
If yes, through UNICAST i.e. peer to peer connection ?
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
Is it still possible to communicate via DDS ?
Yes, it is. Out of the box, DDS Participants only use multicast for discovering other DDS Participants, at startup. This discovery mechanism can be configured in several ways. For a an explanation on how to achieve that, see this RTI Community Knowledge Base article: Configure RTI Connext DDS to not use Multicast.
If yes, through UNICAST i.e. peer to peer connection ?
Yes, with the no-multicast setup, all communications are done over UDP unicast, peer to peer, connectionless.
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Not being able to use multicast does not remove any of the DDS advantages when comparing it to UDP. When using DDS, the transport/discovery configuration is typically invisible to the application and all Publish/Subscribe concepts remain unchanged.
If you are asking about the advantages of using DDS versus UDP, I think that warrants a new question on itself. The answer will be quite extensive :-)
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
With this configuration, your configuration settings will be dependent on the network that you are running on. This means that migration to a different network might need reconfiguration, for example providing different host names or IP addresses. This is inconvenient, but not hard.
Since your environment is restricting the use of multicast, I would not be surprised if there are more restrictions that you have not mentioned or discovered. For example, do you know anything about firewalls or network bandwidth restrictions? Again, DDS can be configured to deal with such things, but you need to be aware of them first.
There are multiple articles suggesting that load-balancer should be used in front of RabbitMQ cluster.
However, there are also multiple references that Spring AMQP is using some
failover implementation like connection reset when broker comes back to life.
I have several questions regarding this topic (given that those articles are more or less old and it's 2018 today)
When using Spring AMQP, is it load-balancing for still required?
If load-balancing is still suggested, how would I solve affinity of primary queue to its node? There would be much inter-connect between cluster nodes, because round-robin load-balancer would have 1-(1/n) success rate of hitting correct cluster node
Does Spring AMQP support some kind of topology awareness, which would allow it to consume from correct node?
There were some articles suggesting that clients should publish/consume to nodes respecting locality of queues. Does this still apply? How does this all fits together given load-balancing, Spring AMQP failover and CachingConnectionFactory?
Can anybody please provide answers to those topics and also provide relevant references, which would provide additional information for verification?
Thanks a lot
For each of your bullets:
a load balancer makes little sense with default configuration of Spring AMQP since it opens a single, long-lived, connection that is shared across all consumers. In, 2.0, you can configure the RabbitTemplate to use a separate connections; this is because it is a recommended configuration to use a different connection for publishers/consumers; this will be default in 2.1.
It might make sense to use a load balancer if you configure the connection factory to cache connections (instead of just channels) since, then, each component gets its own connection.
See next bullet.
See Queue Affinity and the LocalizedQueueConnectionFactory. It uses the management plugin to determine which node currently hosts the queue and connects to that. It will not work with a load balancer since it needs to connect to the actual node.
It is my understanding from several discussions that queue affinity is only needed in the most extreme environments and that, in most environments, the difference is immeasurable. However, environments/networks differ so much, YMMV so you may want to test. My general rule of thumb is to avoid premature optimization since the added complexity of the configuration may simply not be worth the benefit (and you may not have a problem in the first place).
We are evaluating IMDG technologies, Apache Geode vs Hazelcast, any real differences?
hazelcast has WAN replication. Also hazelcast can be used with Solace.
whats the difference?
Hazelcast supports different implementations for WAN replication through its WanPublisher and WanConsumer interfaces.
By default Hazelcast uses WanBatchReplication implementation which creates TCP connections to target members and send the WAN events in batches. This implementation also has in-memory (replicated) queues to send events asynchronously and to deal with differences of WAN link throughput.
You can replace this implementation with SolaceWanPublisher which publishes WAN events to Solace queues and doesn't use the direct TCP link approach used by WanBatchReplication.
Replication of WAN events are provided by Solace and target members consumes events from that queues.
So source and target members are unaware of each others network topology.
You may want to check Hazelcast Documentation or this white paper for details.
I have multiple applications (the producers) that produce messages to be processed by another application (the consumer). The messages will be sent through an ActiveMQ broker running on the same server. I don't have access to the applications' code, therefore the messages will be produced by executing a script (I currently don't know which language to use). The consumers will be Java application that will process the received messages.
I'm looking for an efficient transport that fits my use case. The VM transport cannot be used here. Also, I would like to avoid opening a TCP connection with the broker every time the producer script is executed (i.e. I would like to avoid using the TCP transport). I thought that UDP may be a good fit unless you know another transport which is more appropriate.
Thanks,
Mickael
There are pros and cons of both the TCP and UDP protocol
1)If ordering of messages and reliable-delivery of messages doesn't matter to you then UDP might be a nice choice,moreover in UDP it can also happen that duplicate messages are delievered to broker.
2)Using TCP offers reliable-delivery of messages along with ordering, but if you want to eliminate the stream Transport delay of TCP then you might think against it.
There are couple of others as well which you can retrospect based on your requirements
NIO protocol(USed in case of high traffic requirements)
HTTP protocol(In case you want to bypass firewalls)
Hope this helps!
Good luck!
Why only can java provide support for failover protocol in activemq whereas not other languages.
My doubt is that in the failover protocol like failover://(tcp://host1:61616,tcp://host2:61616)?randomize=false also the client uses one of the the inner urls like tcp://host1:61616 and then how does the broker comes to know that the call was using some failover protocol or not and then how the broker decides that it needs to replicate the message ?
Please understand that failover protocol is meant for reconnect logic on client side only and AMQ broker isn't even aware if a client is using failover protocol or not.
From the official AMQ documentation:
The Failover transport layers reconnect logic on top of any of the
other transports.
The Failover configuration syntax allows you to specify any number of
composite uris. The Failover transport randomly chooses one of the
composite URI and attempts to establish a connection to it. If it does
not succeed or if it subsequently fails, a new connection is
established to one of the other uris in the list.
Not sure what you mean by replication here but as per the official doc
The Failover transport tracks transactions by default. The inflight
transactions are replayed on reconnection.
There are different scenarios to put up a HA solution with ActiveMQ.
If clients connect using the failover protocol to host1,host2, then the broker setup needs to be setup for HA as well.
One solution is to cluster host1 and host2 in an Active-Active solution. Then messages are always propagated when they are asked for - the queues are shared in the entire cluster among all amq brokers.
Otherwise, if the active-active solution is not prefered, then a master-slave solution can be setup where the two brokers, host1 and host2, share the data area (for instance using a Database for persistance or a shared SAN disk).
There are more combinations of setups, but the failover protocol assumes that the entire solution can handle that messages arrives to different brokers, if one goes down. As far as I know, there is no other magic in the failover protocol, from the broker perspective.