I can deliver normal UDP messages between two computers using a direct ethernet connection using my software. But I couldn't do the same for multicasting messages, I've tried other softwares that can send and receive multicast UDP messages and they didn't work as well. So I thought and wanted to ask if a direct Ethernet connection is a proper pyhsical setup or not. And if not what should I do?
Yes, a direct ethernet supports multicast. As opposed to an NBMA network like frame relay (rare these days) which does not support multicast for example.
Make sure you are not actually using a switch (which is a bridge) that has IGMP snooping enabled. IGMP snooping will not propagate multicast to nodes that have not sent an IGMP join (assuming your software does not do the join).
Related
Consider a simple IPv6 multicast application:
A "talker" program periodically sends out IPv6 UDP packets to a well-known multicast-group, sending them to a well-known port.
Zero or more "listener" programs bind themselves to that well-known port and join the well-known multicast group, and they all receive the UDP packets.
That all works pretty well, except in the case where one or more of the listener-programs is unable to bind to the well-known UDP port because a socket in some other (unrelated) program has already bound to that UDP port (and didn't set the SO_REUSEADDR and/or SO_REUSEPORT options to allow it to be shared with anyone else). AFAICT in that case, the listener program is simply out of luck, there is nothing it can do to receive the multicast data, short of asking the user to terminate the interfering program in order to free up the port.
Or is there? For example, is there some technique or approach that would allow a multicast listener to receive all the incoming multicast packets for a given multicast-group, regardless of which UDP port they are being sent to?
If you want to receive all multicast traffic regardless of port, you'd need to use raw sockets to get the complete IP datagram. You could then directly inspect the IP header, check if it's using UDP, then check the UDP header before reading the application layer data. Note that methods of doing this are OS specific and typically require administrative privileges.
Regarding SO_REUSEADDR and SO_REUSEPORT, apps that do this will allow multiple programs to receive multicast packets sent to a given port. However, if you also need to receive unicast packets this method has issues. Incoming unicast packets may be set to both sockets, may always be sent to one specific socket, or sent to each in an alternating fashion. This also differs based on the OS.
http://activemq.apache.org/networks-of-brokers.html
I'm trying activemq network of brokers, following above article.
It works all fine with a fixed list of URIs.
But I have some problem with the multicast discovery. That is, the network bridge between two activemqs on the same machine can be started. But the bridge cannot establish between different machines(I tried telnet, it is ok).
I don't know which part went wrong. So I want to ask that is these two kind of network just difference in configuration?
Telnet is proving that Unicast networking is working, multicast may requires additional configuration in your network.
Are those machines in the same subnet?
Is there a router or Layer 3 switch between them? (it would then requires to be configured if the answer is yes..)
You could use iperf to test the multicast connectivity, you can look at Generating multicast traffic article to know how to do that.
It's unclear from the documentation why you should use Unicast rather than Multicast in a WebLogic cluster. Anyone have experience using either and the benefits of moving to Unicast?
The main difference between Unicast and Multicast is as follows
Unicast:
Say you have three servers (MS-1,MS-2,MS-3) in a cluster. If they have to communicate with each other, then they have to ping (i.e. heartbeats ) the cluster master for informing it that they are alive.
If MS-1 is the master then MS-2 and MS-3 would send the ping to MS-1
Multicast:
In Multicast, there is no cluster master. Instead each server has to ping each other to inform everyone that they are alive.
So MS-1 would send the ping to MS-2 & MS-3 and at the same time MS-2 would send the ping to MS-1 & MS-3 and MS-3 would ping MS-1 & MS-3.
Thus, in multicast there are more pings sent which makes the congestion in sending the pings much heavier compared to unicast. Because of this, WLS recommends using Unicast instead for less congestion in the network: Oracle Docs: Communications In a Cluster
The principle behind Multicast is that any message is received by all subscribers to the Multicast address. So MS-1 only needs to send 1 network packet to alert all other cluster members of its status. This means a status or JNDI update requires only 1 packet per Cluster for Muticast vs 1 packet per Server (approximately) for Unicast. Multicast also requires no "master" election. Multicast is thus far simpler to code and creates less network traffic.
So, Multicast is great? Not necessarily. It uses UDP datagrams which are inherently unreliable and un-acknowledged, so given the unreliable bearer protocol - Etherent - your message might never turn up (interpret: you fall out of the cluster). The whole concept of Multicast is based on subscription, it is not a 'routable' protocol in the normal sense, so by default routers must discard Multicast packets or risk a network storm. Hence the historical requirement for all cluster members to reside on the same network segment.
These shortcomings of Multicast mean Unicast is the way to go if your cluster spans networks or you're losing too many multicast packets.
The main benefit of Unicast over Multicast is ease of configuration. Unicast uses TCP communications and this usually requires no additional network configuration. Multicast uses UDP communication and Multicast addresses and this may require some network configuration and an additional effort in selecting the address to be used.
There is a great article from the Oracle A-Team with in-depth explanation: WebLogic Server Cluster Messaging Protocols.
In documentation for WLS 12.1.2 Oracle added Considerations for Choosing Unicast or Multicast in which they suggest to use Multicast in clusters with more than 10 Managed Servers.
In my personal experience I found that Unicast may give some problems in large clusters mainly because it is a new protocol introduced in WLS 10.0 and still suffer from some minor issues.
The answer here appears to conflict with the recommendation from the Oracle A-Team. Their recommendation is:
In general, the A-Team rule of thumb is to always recommend customers use multicast unless there is good reason why it isn’t possible or practical (e.g., spanning multiple subnets where routers are not allowed to propagate UDP multicast messages). The primary reasons for this are simply efficiency and resiliency to resource shortages.
The full article can be found here.
UPDATE
Weblogic defaults to unicast and documentation for 12c implies that multicast is only supported to ensure backward compatibility:
It is important to note that although parts of the WebLogic Server documentation suggest that multicast is only supported for backward compatibility—this is not correct. The multicast cluster messaging protocol is fully supported by Oracle. The A-team is working with WebLogic Server product management to address these documentation issues in the Weblogic Server 12c documentation.
Both multicast and unicast configurations have cluster masters. In addition to a cluster master, unicast has one or more leaders. The cluster leader may or may not be the cluster master.
Multicast is a broadcast; they do not ping each other like a tcp message. In both unicast and multicast cases, the traffic is usually trivial. But, multicast is almost always the best choice if you network supports it.
Unicast presents a simpler configuration than multicast in that you don't need multicast support. All routers/switches support TCP, but not all routers/switches support or have multicast enabled. But, unicast generates more network traffic than multicast.
Can anyone tell be where to use the UDP protocol except live streaming of music/video? What are default usecases for UDP?
UDP is also good for broadcast, such as service discovery - finding that newly plugged in printer.
Also of note is that broadcast is anonymous, you don't need to specify target hosts, as such it can form the foundation of a convenient plug-and-play or high-availability network.
UDP is stateless and is good for applications that have large numbers of clients connecting to a server such as time servers or DNS. The fact that no connection has to established and maintained reduces the memory required by the server. There is no handshaking involved and so this reduces the traffic on the network. On the downside, if the information transferred requires multiple packets there is no transmission control to ensure that all packets arrive and in the correct order - but in games packets lost are probably better than late or disordered.
Anything else where you need performance but can survive if a packet gets lost along the way. Multiplayer games come to mind, for example.
A very common use case is DNS, since the overhead of creating a TCP connection would by far outweight the actual payload.
Additional use cases are NTP (network time service) and most video games.
I use UDP to add chat capabilities to our applications. No need to create a server. It is also useful to dispatch events to all users of our applications.
This morning, there were big problems at work because an SNMP trap didn't "go through" because SNMP is run over UDP. I remember from the networking class in college that UDP isn't guaranteed delivery like TCP/IP. And Wikipedia says that SNMP can be run over TCP/IP, but UDP is more common.
I get that some of the advantages of UDP over TCP/IP are speed, broadcasting, and multicasting. But it seems to me that guaranteed delivery is more important for network monitoring than broadcasting ability. Particularly when there are serious high-security needs. One of my coworkers told me that UDP packets are the first to be dropped when traffic gets heavy. That is yet another reason to prefer TCP/IP over UDP for network monitoring (IMO).
So why does SNMP use UDP? I can't figure it out and can't find a good reason on Google either.
UDP is actually expected to work better than TCP in lossy networks (or congested networks). TCP is far better at transferring large quantities of data, but when the network fails it's more likely that UDP will get through. (in fact, I recently did a study testing this and it found that SNMP over UDP succeeded far better than SNMP over TCP in lossy networks when the UDP timeout was set properly). Generally, TCP starts behaving poorly at about 5% packet loss and becomes completely useless at 33% (ish) and UDP will still succeed (eventually).
So the right thing to do, as always, is pick the right tool for the right job. If you're doing routine monitoring of lots of data, you might consider TCP. But be prepared to fall back to UDP for fixing problems. Most stacks these days can actually use both TCP and UDP.
As for sending TRAPs, yes TRAPs are unreliable because they're not acknowledged. However, SNMP INFORMs are an acknowledged version of a SNMP TRAP. Thus if you want to know that the notification receiver got the message, please use INFORMs. Note that TCP does not solve this problem as it only provides layer 3 level notification that the message was received. There is no assurance that the notification receiver actually got it. SNMP INFORMs do application level acknowledgement and are much more trustworthy than assuming a TCP ack indicates they got it.
If systems sent SNMP traps via TCP they could block waiting for the packets to be ACKed if there was a problem getting the traffic to the receiver. If a lot of traps were generated, it could use up the available sockets on the system and the system would lock up. With UDP that is not an issue because it is stateless. A similar problem took out BitBucket in January although it was syslog protocol rather than SNMP--basically, they were inadvertently using syslog over TCP due to a configuration error, the syslog server went down, and all of the servers locked up waiting for the syslog server to ACK their packets. If SNMP traps were sent over TCP, a similar problem could occur.
http://blog.bitbucket.org/2012/01/12/follow-up-on-our-downtime-last-week/
Check out O'Reilly's writings on SNMP: https://library.oreilly.com/book/9780596008406/essential-snmp/18.xhtml
One advantage of using UDP for SNMP traps is that you can direct UDP to a broadcast address, and then field them with multiple management stations on that subnet.
The use of traps with SNMP is considered unreliable. You really should not be relying on traps.
SNMP was designed to be used as a request/response protocol. The protocol details are simple (hence the name, "simple network management protocol"). And UDP is a very simple transport. Try implementing TCP on your basic agent - it's considerably more complex than a simple agent coded using UDP.
SNMP get/getnext operations have a retry mechanism - if a response is not received within timeout then the same request is sent up to a maximum number of tries.
Usually, when you're doing SNMP, you're on a company network, you're not doing this over the long haul. UDP can be more efficient. Let's look at (a gross oversimplification of) the conversation via TCP, then via UDP...
TCP version:
client sends SYN to server
server sends SYN/ACK to client
client sends ACK to server - socket is now established
client sends DATA to server
server sends ACK to client
server sends RESPONSE to client
client sends ACK to server
client sends FIN to server
server sends FIN/ACK to client
client sends ACK to server - socket is torn down
UDP version:
client sends request to server
server sends response to client
generally, the UDP version succeeds since it's on the same subnet, or not far away (i.e. on the company network).
However, if there is a problem with either the initial request or the response, it's up to the app to decide. A. can we get by with a missed packet? if so, who cares, just move on. B. do we need to make sure the message is sent? simple, just redo the whole thing... client sends request to server, server sends response to client. The application can provide a number just in case the recipient of the message receives both messages, he knows it's really the same message being sent again.
This same technique is why DNS is done over UDP. It's much lighter weight and generally it works the first time because you are supposed to be near your DNS resolver.