Syslog is a network protocol as described in RFC 5424 and RFC 3164 before that.
-AND-
Syslog is a standard in the *nix world. It's how you do logging. Even on just the local machine, UDP packets are never created. (Right?) The protocol, and the RFC, do not apply here.
What is Syslog predominantly, a protocol or a logging standard? Why?
I'm asking this so that I can update the Wikipedia page with the relevant information. In editing the page, do I focus on the protocol or the standard. It's not possible to do both. The answer to this question will be taken into account whether the page needs to be split into [Syslog] and [Syslog protocol] or not.
Syslog names an event notification system which originated with BSD UNIX. There are two standard interfaces to it.
One interface is in the UNIX standard. It consists of the openlog(), setlogmask(), syslog(), and closelog() functions, and the definitions in syslog.h. Applications can use this interface to create events. Events might go to a file, a console, a list of users, or a remote host; this is not determined by the standard. In Linux, the disposition of events is handled by the syslogd process, following the local policy defined in the file /etc/syslog.conf.
The other interface is described by RFC 3164 and updated in RFC 5424. It consists of a protocol for transmitting events to a remote host with UDP. Applications can use this interface to create events, but the more common user of the syslog protocol is syslogd itself, when it is forwarding events to a remote host.
It is wrong to say that UDP syslog packets are never created by UNIX hosts. When there are several hosts to manage, it is common practice to funnel some or all of the events to a designated host, so that there is one error log to monitor instead of many.
Related
For example, setting the remote{} configuration, does that also set the transport that is used internally for cluster communication, for example, the heartbeat messages.
I am not asking for any use case purpose I am asking so I better understand what's happening behind the scenes.
At the moment (Akka.NET 1.3) uses its own protocol for remote communication on top of TCP connection - only a single connection is used by every node-to-node connection. This video discusses it in greater detail.
In future, it will probably change to match JVM version of akka - two major ideas are:
"lanes": multiple connections for each pair of nodes, to avoid head-of-line blocking, that is inherent problem of TCP.
Add support for other protocols, such as Aeron, which is also supported by akka on JVM.
I have a configuration where 3 applications run on 3 different Virtual Machine's and they must communicate via DDS i.e. RTPS protocol.
The configuration is as follows :
ROS2 based ADAS functions
Simulation Tool
Python/Tensorflow based machine learning functions
All 3 need to be on different VMs.
It is not possible at our end to allow multicasting for the MS AZURE VM and our network.
Here are some questions :
Is it still possible to communicate via DDS ?
If yes, through UNICAST i.e. peer to peer connection ?
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
Is it still possible to communicate via DDS ?
Yes, it is. Out of the box, DDS Participants only use multicast for discovering other DDS Participants, at startup. This discovery mechanism can be configured in several ways. For a an explanation on how to achieve that, see this RTI Community Knowledge Base article: Configure RTI Connext DDS to not use Multicast.
If yes, through UNICAST i.e. peer to peer connection ?
Yes, with the no-multicast setup, all communications are done over UDP unicast, peer to peer, connectionless.
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Not being able to use multicast does not remove any of the DDS advantages when comparing it to UDP. When using DDS, the transport/discovery configuration is typically invisible to the application and all Publish/Subscribe concepts remain unchanged.
If you are asking about the advantages of using DDS versus UDP, I think that warrants a new question on itself. The answer will be quite extensive :-)
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
With this configuration, your configuration settings will be dependent on the network that you are running on. This means that migration to a different network might need reconfiguration, for example providing different host names or IP addresses. This is inconvenient, but not hard.
Since your environment is restricting the use of multicast, I would not be surprised if there are more restrictions that you have not mentioned or discovered. For example, do you know anything about firewalls or network bandwidth restrictions? Again, DDS can be configured to deal with such things, but you need to be aware of them first.
Can someone point me to some documentation on how to implement a Bittorrent UDP application?
I am working on a Bittorrent application and I can successfully download using TCP but I want to implement UDP and I can't find any information on it. I am retrieving peers with UDP trackers but that approach appears different than downloading files.
If I sniff a UDP Handshake I see the following in Wireshark:
172.16.49.213 5.31.44.30 UDP 62 35507 → 18318 Len=20
5.31.44.30 172.16.49.213 UDP 62 18318 → 35507 Len=20
This is done before the handshake. Also, it looks like there are 20 bytes prepended to the Handshake (different than the 20 bytes above).
So I need some information to help me with what all this means.
The BitTorrent Peer Wire Protocol over UDP, called uTP is specified in:
BEP29 - uTorrent transport protocol
BitTorrentInc has also published uTP as a IETF RFC were they call it LEDBAT:
RFC 6817 - Low Extra Delay Background Transport (LEDBAT)
However, I wouldn't recommend anyone to do their own implementation, (except as a learning experience), as it involves a lot of time critical, low level network IO and is very tricky to get right.
Instead I recommend to use the library: https://github.com/bittorrent/libutp
Almost all clients implementing uTP uses this library. AFAIK, the only exception is libtorrent/rasterbar (used by qBittorrent and Deluge) and it don't work as good as libutp does.
"Because winpcap sends and receives data independent of host protocol such as TCP/IP, which means winpcap can't stop, filter and take control of other applications on the same machine control, it can only monitor them simply, so it can't provide support for similar network flow control, the quality of service, personal firewall and so on."
In this passage, what "winpcap can't stop, filter and take control of other applications on the same machine control" means?
It means that if some program on your machine is sending and receiving packets, the path those packets take through the operating system does not, and cannot, go through WinPcap; all WinPcap can do is listen to see copies of the packets sent and received by the machine.
This means that if you want to, for example, implement a firewall program to prevent other programs from connecting to particular addresses or ports or receiving connections from particular addresses or ports, or to modify packets sent by other programs before they get put onto the network or modify packets received from the network before they get passed to other programs, or do something else "active", you cannot use WinPcap.
WinPcap uses mechanisms in Windows that are intended to support programs such as packet sniffers, so that limitation is inherent in WinPcap; a library to support writing firewalls and the like would have to use different mechanisms in Windows (and those mechanisms might not support programs such as packet sniffers).
I have an application where I have potentially many processes must send a message to one process. I figured the most expedient way to do this would be to just send a UDP datagram.
When opening a socket, I need to specify which port to listen to. As I just want to fire and forget the UDP datagram, I'd like to just have the OS pick an available port. I'd rather not have to hunt for one, especially as many instances of my process may be running. Is there a way I can tell the OS to just pick an available port (or better yet, not even open a listening socket) to fire and forget my UDP datagram?
Yes. Specify 0 as the port. The OS will pick an available port for you.
Answering the "Problem" rather than the "Question"
If all the processes are operating on the same PC, you might want to look into shared files or some other means of communications other than a networking stack.
I'd suggest you explore the options in Boost's Interprocess library
http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess.html
Quick guide here:
http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess/quick_guide.html
Specifically, I suggest you use a named pipe or shared memory between your processes.