I have a UDP server implemented using the template in the documentation, which can be found here: https://docs.python.org/3/library/asyncio-protocol.html#udp-echo-server-protocol
I would like to know the addr of the client which lost connection. The connection_lost callback only has a single parameter, exc for the exception.
Edit: Following the downvotes I want to highlight that its not a very noob-friendly part of the module naming a callback in the datagram ServerProtocol class 'connection_made'.
The Python API designers need to document this properly.
It looks like connection_made() is called when you create the socket and connect it, which in turn only happens if you specify a non-None Remote_addr.
To understand all that, first you need to understand what connect() does to a UDP socket at the Berkeley Sockets API level:
It conditions the socket so that write() andsend()can be used as well assendto()`, both of which will only send to the connected target address.
It conditions the socket to filter out all datagrams that did not originate at the connect target.
It does not create a wire connection of any kind. Nothing is received by the peer or sent on the wire in any way.
You can connect() a UDP socket multiple times, either to a different address or to null, which completely undoes (1) and (2).
So, I can only imagine that the connection_lost() callback is called when (4) happens, which it isn't in your code.
Whatever it does, if anything, it certainly can't be used to detect when a client disconnects, as there is no such event in UDP.
Related
[edit]
Seems my question was asked nearly 10 years ago here...
Emulating accept() for UDP (timing-issue in setting up demultiplexed UDP sockets)
...with no clean and scalable solution. I think this could be solved handily by supporting listen() and accept() for UDP, just as connect() is now.
[/edit]
In a followup to this question...
Can you bind() and connect() both ends of a UDP connection
...is there any mechanism to simultaneously bind() and connect()?
The reason I ask is that a multi-threaded UDP server may wish to move a new "session" to its own descriptor for scalability purposes. The intent is to prevent the listener descriptor from becoming a bottleneck, similar to the rationale behind SO_REUSEPORT.
However, a bind() call with a new descriptor will take over the port from the listener descriptor until the connect() call is made. That provides a window of opportunity, albeit briefly, for ingress datagrams to get delivered to the new descriptor queue.
This window is also a problem for UDP servers wanting to employ DTLS. It's recoverable if the clients retry, but not having to would be preferable.
connect() on UDP does not provide connection demultiplexing.
connect() does two things:
Sets a default address for transmit functions that don't accept a destination address (send(), write(), etc)
Sets a filter on incoming datagrams.
It's important to note that the incoming filter simply discards datagrams that do not match. It does not forward them elsewhere. If there are multiple UDP sockets bound to the same address, some OSes will pick one (maybe random, maybe last created) for each datagram (demultiplexing is totally broken) and some will deliver all datagrams to all of them (demultiplexing succeeds but is incredibly inefficient). Both of these are "the wrong thing". Even an OS that lets you pick between the two behaviors via a socket option is still doing things differently from the way you wanted. The time between bind() and connect() is just the smallest piece of this puzzle of unwanted behavior.
To handle UDP with multiple peers, use a single socket in connectionless mode. To have multiple threads processing received packets in parallel, you can either
call recvfrom on multiple threads which process the data (this works because datagram sockets preserve message boundaries, you'd never do this with a stream socket such as TCP), or
call recvfrom on a single thread, which doesn't do any processing, just queues the message to the thread responsible for processing it.
Even if you had an OS that gave you an option for dispatching incoming UDP based on designated peer addresses (connection emulation), doing that dispatching inside the OS is still not going to be any more efficient than doing it in the server application, and a user-space dispatcher tuned for your traffic patterns is probably going to perform substantially better than a one-size-fits-all dispatcher provided by the OS.
For example, a DNS (DHCP) server is going to transact with a lot of different hosts, nearly all running on port 53 (67-68) at the remote end. So hashing based on the remote port would be useless, you need to hash on the host. Conversely, a cache server supporting a web application server cluster is going to transact with a handful of hosts, and a large number of different ports. Here hashing on remote port will be better.
Do the connection association yourself, don't use socket connection emulation.
The issue you described is the one I encountered some time ago doing TCP-like listen/accept mechanism for UDP.
In my case the solution (which turned out to be bad as I will describe later) was to create one UDP socket to receive any incoming datagrams and when one arrives making this particular socket connected to sender (via recvfrom() with MSG_PEEK and connect()) and returning it to new thread. Moreover, new not connected UDP socket was created for next incoming datagrams. This way the new thread (and dedicated socket) did recv() on the socket and was handling only this particular channel from now on, while the main one was waiting for new datagrams coming from other peers.
Everything had worked well until the incoming datagram rate was higher. The problem was that while the main socket was transitioning to connected state, it was buffering not one but a few more datagrams (coming from many peers) and thus thread created to handle the particular sender was reading in effect a few more datagrams not intended to it.
I could not find solution (e.g. creating new connected socket (instead connecting the main one) and pass the received datagram on main socket to its receive buffer for futher recv()). Eventually, I ended up with N threads, each one having one "listening" socket (with use of SO_REUSEPORT) with datagram scattering done on OS level.
I don't quite understand exactly how a few of the features are shared when a TcpListener and TcpClient communicate.
Let's say the following code is run (for now ignore synchronisation):
Server:
Dim server As New TcpListener(localAddr, port)
server.Start()
Dim client As TcpClient = server.AcceptTcpClient()
Client:
Dim client As New TcpClient
client.Connect(hostAddr, port)
And the connection is successfully established. Now there are two TcpClient instances — one on server side and one on client side. However, they share the same network stream through TcpClient.GetStream().
I'm slightly confused — does the client pass itself and all of its properties to the server when server.AcceptTcpClient() is called?
What about any changes to either of the TcpClient instances after this? When the connection shuts down I call this on both sides:
client.GetStream.Close()
client.Close()
But I get an exception with TcpClient.GetStream.Close() on the client which executes this code the latest because it tells me that the client is already closed (this happens when the above code isn't perfectly synchronised on both sides).
What about the .SendBufferSize and .ReceiveBufferSize properties? Do I need to set this on both sides of the connection?
Hope someone can clear up my confusion with an explanation of how exactly the TcpClient/Listener classes work during the communication — so far I haven't been able to find documentation explaining what exactly happens.
The TCP protocol does not know what a TcpClient is. This is a .NET concept. TCP does not reference .NET concepts at all. For that reason no objects will be sent across the wire.
The only thing that is sent is the bytes you explicitly write.
Each side has it's own isolated objects. Both sides use their own TcpClient object which acts like a handle to the TCP connection.
client.GetStream.Close()
client.Close()
This is not the proper shutdown sequence. The first line is redundant to the second and incomplete. Close should never be called. The best way to do it is to wrap the client in using. The second best way is to call Dispose on the client. The Close methods in the BCL are historic accidents and should be ignored. They do the same thing that Dispose does in all cases that I ever looked at.
Don't touch the buffer sizes. They control how much memory the kernel uses to buffer data on your end of the connection. The kernel is capable of managing this by itself.
Also don't look at the buffer sizes in your code. They are meaningless. Also don't use the DataAvailable property because if it returns false/0 this does not mean that no data can be read.
The Connected property is not necessarily synchronized on both sides. If the network goes down there can be no synchronization. Never look at the Connected property. If it says true the next nanosecond it could be false. So it's not possible to make decisions based on that property. You do not need to test anything. Just Read/Write and handle the exceptions by aborting.
Regarding packets, you are not sending packets when you Write. TCP has a boundaryless stream of bytes. The kernel packetizes your data internally. You do not need to split data into specific sizes. Just use a fairly big buffer size such as 8K (or more on fast networks). The write size is only about saving CPU time by being less chatty (assuming nagling is enabled).
To establish WebRTC connections the ICE protocol is used with a signaling server which must send messages in both directions. I wonder why after the initiator sent its offer and candidates to the other participant, the participant needed to send back its answer and candidates using the signaling channel in the other direction. Cannot the participant open the connection to the initiator using candidates from both sides and send back its answer using the open connection?
I started reading ICE RFC and the only relevant part I found is in section 5.2 where the initiator must take the controlling role and nominates candidate pairs. But it does not explain why the other could not initiate connection.
To give some background, I am trying to build a webapp for which I want users to establish WebRTC connections without using a signaling server. I thought of having the app to generate a URL including the offer and candidates and providing this URL to other participants through other medium like instant messaging. The issue I got is that the participant need to send back its answer and candidates using the same medium, which is not practical. In the end I will go for a signalling server but I wonder the technical reason.
Yes, you can do that if caller is behind public IP or Full Cone NAT(in this case, router connection mapping needs not to be timed out).
You can able full fill above conditions rarely.
What's the problem with other NAT types?
For example , PRC(port restricted cone) NAT won't allow you to receive a packet from a IP:Port , if you didn't send any packet to that IP:Port before. So callee will never able to send you a packet.
So if callee sends her candidates list to you . you can send some dummy data(with low TTL) to her IP:Port to fool your PRC NAT (now it allow incoming packets from callee's IP:Port as it sends a packet to that IP:Port before).
To know more about different types of NAT:
https://en.wikipedia.org/wiki/Network_address_translation
http://think-like-a-computer.com/2011/09/16/types-of-nat/
At the click of a button a simple message is to be sent from one process to another, and the message would be just a simple command with string arguments, totaling something like 50 characters (definitely much less than 1k), like
DisplaySomeInfo("param1", "param2")
and satisfying the following:
must be simple (i.e. no full-blown messaging system)
must run on Windows, should run on Unix
the picture below shows my actual requirements, but it should work with any major programming language / runtime
performance is not crucial, the whole send/receive cycle should not take more than a millisecond though for such a simple message
"guaranteed delivery" etc. NOT necessary
sender does not care if a/the receiver got the message - or if there is a receiver in the first place
no encryption/authentication/authorization necessary
In wikipedia there is a long list of ways to do IPC, but the simplest one seems to be the non-connected socket option.
Are there any better ways (easier to implement/maintain/debug, more future proof, ...) to implement it than simply sending / receiving UDP packets (one message per packet, TTL=0), say coupled with a JSON (de)serializer?
For these requirements, simply sending UDP unicast packets to some port on localhost seems ideal. The only problem is that a port number has to be chosen and fixed. But if that is not an issue, simply sending and receiving UDP packets is as easy as it gets, without the need for third party libraries.
Bare-bones example for sending a UDP packet from a .Net app:
public static void SendUdpPacket(int destinationPort, string payload)
{
IPEndPoint endPoint =
new IPEndPoint(IPAddress.Parse("127.0.0.1"), destinationPort);
byte[] buffer = Encoding.UTF8.GetBytes(payload);
Socket socket =
new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.SendTo(buffer, endPoint);
socket.Close();
}
The only open question is how to encode multiple parameters. But for that there are gazillions of ways, built into many languages is HTTP URL encoding.
I have 3 network interfaces on pc and want to make sure that when I do udp socket send, it sends via a specific network interface ( I have the ip address to use when sending data).
Here is the code.
udp::socket send_socket(io_service,boost::asio::ip::udp::v6());
udp::endpoint local_end_point(boost::asio::ip::address::from_string(LocalIpAddress),1111);
// send_socket.bind(local_end_point);
send_socket.send_to(const_buffer,senderEndpoint,0,error);
The above code works but I dont have control over what network interface the data would be sent through. If I uncomment the send_socket.bind line, I stop receiving any data on the other end.
Is there any other way to bind a socket to a particular network interface?
The bind() function ties your socket to a specific network interface for sending AND receiving. If you stop receiving data on the other end it's likely because the other end is not routable via the NIC that you've specified in the bind call.