I'm newbie to DDS/Fast-RTPS.
Based on my understanding, the discovery is LAN-based. It failed to discover a node which is not in the same LAN. is it correct?
I'm wondering if we can use fast-rtps to communication across networks?
ps. let's ignore NAT/Firewall issues. Assuming we have a IP/TCP full reachable network environment.
DDS uses Multicast UDP. If your switches and other network infrastructure is set to swallow Multicast packets, or if the TTL is set too low, then the default discovery implementation of DDS will not complete/be seen.
You can up the TTL on your infrastructure, or you can tell the DDS libraries to target specific addresses (see the documentation for your provider's libraries).
Related
I have a configuration where 3 applications run on 3 different Virtual Machine's and they must communicate via DDS i.e. RTPS protocol.
The configuration is as follows :
ROS2 based ADAS functions
Simulation Tool
Python/Tensorflow based machine learning functions
All 3 need to be on different VMs.
It is not possible at our end to allow multicasting for the MS AZURE VM and our network.
Here are some questions :
Is it still possible to communicate via DDS ?
If yes, through UNICAST i.e. peer to peer connection ?
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
Is it still possible to communicate via DDS ?
Yes, it is. Out of the box, DDS Participants only use multicast for discovering other DDS Participants, at startup. This discovery mechanism can be configured in several ways. For a an explanation on how to achieve that, see this RTI Community Knowledge Base article: Configure RTI Connext DDS to not use Multicast.
If yes, through UNICAST i.e. peer to peer connection ?
Yes, with the no-multicast setup, all communications are done over UDP unicast, peer to peer, connectionless.
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Not being able to use multicast does not remove any of the DDS advantages when comparing it to UDP. When using DDS, the transport/discovery configuration is typically invisible to the application and all Publish/Subscribe concepts remain unchanged.
If you are asking about the advantages of using DDS versus UDP, I think that warrants a new question on itself. The answer will be quite extensive :-)
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
With this configuration, your configuration settings will be dependent on the network that you are running on. This means that migration to a different network might need reconfiguration, for example providing different host names or IP addresses. This is inconvenient, but not hard.
Since your environment is restricting the use of multicast, I would not be surprised if there are more restrictions that you have not mentioned or discovered. For example, do you know anything about firewalls or network bandwidth restrictions? Again, DDS can be configured to deal with such things, but you need to be aware of them first.
http://activemq.apache.org/networks-of-brokers.html
I'm trying activemq network of brokers, following above article.
It works all fine with a fixed list of URIs.
But I have some problem with the multicast discovery. That is, the network bridge between two activemqs on the same machine can be started. But the bridge cannot establish between different machines(I tried telnet, it is ok).
I don't know which part went wrong. So I want to ask that is these two kind of network just difference in configuration?
Telnet is proving that Unicast networking is working, multicast may requires additional configuration in your network.
Are those machines in the same subnet?
Is there a router or Layer 3 switch between them? (it would then requires to be configured if the answer is yes..)
You could use iperf to test the multicast connectivity, you can look at Generating multicast traffic article to know how to do that.
I don't have a specific query here ; just need some design guidelines.
I came across this article on Node.js , MQTT and Websockets.
I guess we can achieve similar purpose using Node/Java + ActiveMQ + Websockets. My query is how to select between MQ and MQTT ? Can I safely use an "open" server like mosquitto in a medium-large scale project, compared to ActiveMQ ?
This article has had some insight, and it seems like I should use both MQ and MQTT, as MQTT may possibly help if I get lightweight clients in future.
Thanks !
Adding to what Shashi has said, these have different capabilities and use cases.
MQTT defines a standard wire protocol for pub/sub and, as Shashi noted, is designed for very lightweight environments. As such it has a very minimal wire format, a few basic qualities of service and a basic feature set.
Traditional message queueing systems on the other hand are generally proprietary (although AMQP aims to change that), cover both point-to-point and pub/sub, offer many qualities of service and tend to have a more heavyweight wire format, although this exists to support enhanced feature sets such as reply-to addressing, protocol conversion, etc.
A good example of MQTT would be where you have endpoints in phones, tablets and set-top boxes. These have minimal horsepower, memory and system resources. Typically connections from these either stay MQTT and they talk amongst themselves, or they are bridged to an enterprise-class MQ where they can intercommunicate with back-end applications. For example, an MQTT-based chat client might talk directly to another through the MQTT broker. Alternatively, an MQTT-based content-delivery system would bridge to an enterprise messaging network which hosted the ads and other content to be delivered to apps running on phones and tablets. The enterprise back-end would manage all the statistics of ad delivery and views upon which billings are based and the MQTT leg allows the content to be pushed with minimal battery or horsepower consumption on the end-user device.
So MQTT is used for embedded systems and end-user devices where power, bandwidth and network stability are issues. This is often in combination with traditional MQ messaging, although I haven't ever seen MQTT used as the exclusive transport for traditional messaging applications. Presumably, this is because MQTT lacks some of the more robust features such as message correlation, reply-to addressing and point-to-point addressing that have been core to messaging for 20 years.
MQTT protocol is suited for small devices like sensors, mobile phones etc that have small memory footprint. These devices are typically located in a brittle network and typically have low computing power.
These devices connect to an organizations back-end network via MQTT protocol for sending and receiving messages. For example a temperature sensor in an oil pipeline would collect temperature of the oil flowing through the pipe and send it to the control center. In response a command message could be sent over MQTT to another device to reduce/stop the flow of oil through that pipe.
WebSphere MQ has the capability to send/receive messages to/from the MQTT devices. So if you plan to implement a messaging based solution that involves devices & sensors, you can consider MQ and MQTT.
HTH
As already discussed, MQTT defines an applicative wire protocol (i.e. how the information is organized and then serialized, before to be transferred).
Mosquitto, or whatever else MQTT broker, is just an implementation of the Hub and Spoke Integration Pattern, just like JMS and AMQP based brokers, the difference consists in the wire protocol at transport level: AMQP defines a standardized transport wire protocol, instead JMS brokers like ActiveMQ defines their own proprietary format, namely the OpenWire. Of course, not standard implementations, like Mosquitto, implement proprietary wire transport protocol (this impacts interoperability, but can be a better choice in terms of perfomances).
Back to the question. Brokers like Mosquitto can be used in real scenarios, according to your needs in terms of scalability and reliability: normally, clustering is needed to assure i. Availability, ii. Reliability and iii. Scalability. Brokers thought for PAN (Private Area Netorks), normally do not provide OTB (Out of The Box) such features - ActiveMQ provides that.
Concluding, it's up to your requirements to pick for you the best solution.
What is the most simple and straightforward approach for serverless P2P UDP chat in Boost Asio? The chat will work across the internet. There are ready UDP examples but they all maintain client-server approach!
I'm assuming that by serverless you mean a P2P network without a central control server.
IMO your question has little to do with boost-asio. asio is a cross-platform network library. You seem to be asking more of a network engineering type question and asio is just one of the tools you can use for implementation.
The examples are client-server in that the example applications possibly fall under the client-server architecture. However the socket code (or asio usage) used to send and receive messages will look the same irrespective of client-server or P2P applications i.e. you send a message to an address and you receive messages on a specified port. The differences will come in at the protocol layer, but this has nothing to do with asio per se.
The following may or may not be of interest to you: there is no simple way IMO: users located behind firewalls and NAT means that you need to use techniques such as STUN, TURN and ICE to resolve addresses or in the worst case relay data. All these designs require a server.
It's unclear from the documentation why you should use Unicast rather than Multicast in a WebLogic cluster. Anyone have experience using either and the benefits of moving to Unicast?
The main difference between Unicast and Multicast is as follows
Unicast:
Say you have three servers (MS-1,MS-2,MS-3) in a cluster. If they have to communicate with each other, then they have to ping (i.e. heartbeats ) the cluster master for informing it that they are alive.
If MS-1 is the master then MS-2 and MS-3 would send the ping to MS-1
Multicast:
In Multicast, there is no cluster master. Instead each server has to ping each other to inform everyone that they are alive.
So MS-1 would send the ping to MS-2 & MS-3 and at the same time MS-2 would send the ping to MS-1 & MS-3 and MS-3 would ping MS-1 & MS-3.
Thus, in multicast there are more pings sent which makes the congestion in sending the pings much heavier compared to unicast. Because of this, WLS recommends using Unicast instead for less congestion in the network: Oracle Docs: Communications In a Cluster
The principle behind Multicast is that any message is received by all subscribers to the Multicast address. So MS-1 only needs to send 1 network packet to alert all other cluster members of its status. This means a status or JNDI update requires only 1 packet per Cluster for Muticast vs 1 packet per Server (approximately) for Unicast. Multicast also requires no "master" election. Multicast is thus far simpler to code and creates less network traffic.
So, Multicast is great? Not necessarily. It uses UDP datagrams which are inherently unreliable and un-acknowledged, so given the unreliable bearer protocol - Etherent - your message might never turn up (interpret: you fall out of the cluster). The whole concept of Multicast is based on subscription, it is not a 'routable' protocol in the normal sense, so by default routers must discard Multicast packets or risk a network storm. Hence the historical requirement for all cluster members to reside on the same network segment.
These shortcomings of Multicast mean Unicast is the way to go if your cluster spans networks or you're losing too many multicast packets.
The main benefit of Unicast over Multicast is ease of configuration. Unicast uses TCP communications and this usually requires no additional network configuration. Multicast uses UDP communication and Multicast addresses and this may require some network configuration and an additional effort in selecting the address to be used.
There is a great article from the Oracle A-Team with in-depth explanation: WebLogic Server Cluster Messaging Protocols.
In documentation for WLS 12.1.2 Oracle added Considerations for Choosing Unicast or Multicast in which they suggest to use Multicast in clusters with more than 10 Managed Servers.
In my personal experience I found that Unicast may give some problems in large clusters mainly because it is a new protocol introduced in WLS 10.0 and still suffer from some minor issues.
The answer here appears to conflict with the recommendation from the Oracle A-Team. Their recommendation is:
In general, the A-Team rule of thumb is to always recommend customers use multicast unless there is good reason why it isn’t possible or practical (e.g., spanning multiple subnets where routers are not allowed to propagate UDP multicast messages). The primary reasons for this are simply efficiency and resiliency to resource shortages.
The full article can be found here.
UPDATE
Weblogic defaults to unicast and documentation for 12c implies that multicast is only supported to ensure backward compatibility:
It is important to note that although parts of the WebLogic Server documentation suggest that multicast is only supported for backward compatibility—this is not correct. The multicast cluster messaging protocol is fully supported by Oracle. The A-team is working with WebLogic Server product management to address these documentation issues in the Weblogic Server 12c documentation.
Both multicast and unicast configurations have cluster masters. In addition to a cluster master, unicast has one or more leaders. The cluster leader may or may not be the cluster master.
Multicast is a broadcast; they do not ping each other like a tcp message. In both unicast and multicast cases, the traffic is usually trivial. But, multicast is almost always the best choice if you network supports it.
Unicast presents a simpler configuration than multicast in that you don't need multicast support. All routers/switches support TCP, but not all routers/switches support or have multicast enabled. But, unicast generates more network traffic than multicast.