We can simply use redis to achieve the remote communication such as:
redis.StrictRedis(host=REDIS_IP,port=PORT)
I don't know whether redis achieve the remote and local in same pattern ?
Maybe I just want to know how redis achieve network communication and inter-process communcation in different way?
If there is something wrong, please point out. Thanks
Redis can handle classical TCP sockets, but also stream oriented unix domain sockets.
TCP sockets can be used to perform both network and local inter-process communication. Unix domain sockets can only support local inter-process communication.
Both kind of sockets are materialized by file descriptors. Redis is based on an event-loop working at the file descriptor level, so it processes TCP and unix domain sockets in the same exact way (using the standard network API). You will find most of the related source code in ae.c (event loop) and anet.c (networking).
You can use unix domain sockets to improve the performance of Redis roundtrips when client and server are hosted on the same box. It depends on your workload, but the performance benefit (in term of throughput) of unix domain sockets over TCP sockets is typically around 1.5 (i.e. you can process 50% more operations when using unix domain sockets).
Related
I have a configuration where 3 applications run on 3 different Virtual Machine's and they must communicate via DDS i.e. RTPS protocol.
The configuration is as follows :
ROS2 based ADAS functions
Simulation Tool
Python/Tensorflow based machine learning functions
All 3 need to be on different VMs.
It is not possible at our end to allow multicasting for the MS AZURE VM and our network.
Here are some questions :
Is it still possible to communicate via DDS ?
If yes, through UNICAST i.e. peer to peer connection ?
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
Is it still possible to communicate via DDS ?
Yes, it is. Out of the box, DDS Participants only use multicast for discovering other DDS Participants, at startup. This discovery mechanism can be configured in several ways. For a an explanation on how to achieve that, see this RTI Community Knowledge Base article: Configure RTI Connext DDS to not use Multicast.
If yes, through UNICAST i.e. peer to peer connection ?
Yes, with the no-multicast setup, all communications are done over UDP unicast, peer to peer, connectionless.
Is using DDS communication beneficial in this case if i already have the option of basic UDP socket programming ?
Not being able to use multicast does not remove any of the DDS advantages when comparing it to UDP. When using DDS, the transport/discovery configuration is typically invisible to the application and all Publish/Subscribe concepts remain unchanged.
If you are asking about the advantages of using DDS versus UDP, I think that warrants a new question on itself. The answer will be quite extensive :-)
Could you think of any restrictions/ further problems in using DDS for such a configuration ?
With this configuration, your configuration settings will be dependent on the network that you are running on. This means that migration to a different network might need reconfiguration, for example providing different host names or IP addresses. This is inconvenient, but not hard.
Since your environment is restricting the use of multicast, I would not be surprised if there are more restrictions that you have not mentioned or discovered. For example, do you know anything about firewalls or network bandwidth restrictions? Again, DDS can be configured to deal with such things, but you need to be aware of them first.
I have read about transport schemes supported by WCF several times. It looks very theoretical.Almost everywhere it is described as:
Following are the transport schemes supported by WCF:
HTTP/HTTPS - http://localhost:8001/MyService
TCP - net.tcp://localhost:8002/MyService
IPC - net.pipe://localhost/MyPipe
Peer network
MSMQ - net.msmq://localhost/private/MyQueue
Service bus - sb://MyNamespace.servicebus.windows.net/
However I could not understand who would require to consume WCF over TCP or IPC or MSMQ. Can anyone give a practical example of who would really need to consume WCF over TCP or any other sceme than HTTP and how would they do so?
Consuming a WCF-Service over TCP is in my case the most common way if you want two or more Programs to communicate. WCF over tcp can easily replace IPC-Communication.
For example:
You have a Windows-Service hosting an WCF-Service. With Vista and later you have to deal with the Session 0 isolation.
You now want to communiacte with that WCF-Service. A common way to achieve that is a tray-application.
Most Anti-Virus programs do it like this.
If you are operating on the same machine, tcp is way faster than HTTP-Binding.
I have found Tim's comments very useful. I have gone through the link he had provided and I will try to summarize whatever I have got from it.
When to use TCP - When host is WCF and consumer is WCF as well use TCP. The communication would be much faster than HTTP.
When to use Named pipes - When host and consumer both are WCF and reside on same machine. The communication would be faster and the
access to the host would be denied from other machines.
You can refer this image for more details about choosing appropriate transport. To dig down further refer this.
If I was to implement a server to handle multiple clients connecting simultaneously would it be better to use TCP?
Not taking efficiency into account (I know know UDP is quicker, but unreliable).
In UDP you can't have sockets for each client connection?
Because in UDP the socket is identified by only the destination port number (right?).
In Java, I know it is easy to create a multi-threaded server to handle multiple clients at the same time in TCP. But can it be done in UDP? I imagine that it would be very complicated.
I'm just trying to get an understanding of UDP here (I don't want to actually implement anything).
It depends on what kind of server you are developing. If you need your clients to stay connected and ready to receive data from server(for example a push service) you should implement it using TCP. If you want to implement a simple request-response service, then UDP is better choice.
From what I understand IMAP requires a connection per each user. I'm writing an IMAP client (currently just gmail) that supports many (100s, 1000s maybe 10000s+) users at a time. Obviously cutting down the number of open connections would be great. I'm wondering if it's possible to use thread pooling on my side to connect to gmail via IMAP or if that simply isn't supported by the IMAP protocol.
IMAP typically uses SSL over TCP/IP. And a TCP/IP connection will need to be maintained per IMAP client connection, meaning that there will be many simultaneous open connections.
These multiple simultaneous connections can easily be maintained in a non-threaded (single thread) implementation without affecting the state of the TCP connections. You'll have to have some sort of a flow concept per IMAP TCP/IP connection, and store all of the flows in a container (a c++ STL map for instance) using the TCP/IP five-tuple (or socketFd) as a key. For each data packet received, lookup the flow and handle the packet accordingly. There is nothing about this approach that will affect the TCP nor IMAP connections.
Considering that this will work in a single-thread environment, adding a thread pool will only increase the throughput of the application, since you can handle data packets for several flows simultaneously (assuming its a multi-core CPU) You will just need to make sure that 2 threads dont handle data packets for the same flow at the same time, which could cause the packets to be handled out of order. An approach could be to have a group of flows per thread, maybe using IP pools or something similar.
Can anyone tell be where to use the UDP protocol except live streaming of music/video? What are default usecases for UDP?
UDP is also good for broadcast, such as service discovery - finding that newly plugged in printer.
Also of note is that broadcast is anonymous, you don't need to specify target hosts, as such it can form the foundation of a convenient plug-and-play or high-availability network.
UDP is stateless and is good for applications that have large numbers of clients connecting to a server such as time servers or DNS. The fact that no connection has to established and maintained reduces the memory required by the server. There is no handshaking involved and so this reduces the traffic on the network. On the downside, if the information transferred requires multiple packets there is no transmission control to ensure that all packets arrive and in the correct order - but in games packets lost are probably better than late or disordered.
Anything else where you need performance but can survive if a packet gets lost along the way. Multiplayer games come to mind, for example.
A very common use case is DNS, since the overhead of creating a TCP connection would by far outweight the actual payload.
Additional use cases are NTP (network time service) and most video games.
I use UDP to add chat capabilities to our applications. No need to create a server. It is also useful to dispatch events to all users of our applications.