I am using sockets in python to send files, and I am doing a packet capture while sending these files. However, I find that each packet is 1434 bytes instead of the 1500 bytes (MTU is set at 1500 bytes on my system).
I have attached some screenshots of the packet capture. I need to send the packet at 1500 bytes rather than the 1434 bytes, can some one tell me what's going on?
TCP sends whatever-sized segments it wants to send over the wire; you can't control that from the socket layer. Perhaps the remote machine is only offering a window big enough to send 1314 bytes of data (1314 + 14 bytes of Ethernet header + 20 bytes of IP header without options + 20 bytes of TCP header without options = 1368), or perhaps the congestion window isn't open wide enough to send more data.
Furthermore, you shouldn't have to control that. TCP provides a sequenced byte stream; packet boundaries are NOT significant in TCP!
Related
I want to simulate a simple client/server application streaming 2-3 4k 25 fps videos within 5G network using OMNeT++ stack. After capturing all incoming video flows with opencv and encoding with h264 codec I have roughly 20 kilobytes for each frame encoded as bytes -- uint8_t. Of course capturing for each of flow happens in a separate thread.
Now I want to send it to some clients over 5G using UDP protocol. If you look almost at every open source implementation of video streaming, the process of transmission is presented very simple:
uint_8* buffer; // encoded frame
int size; // buffer size
send(clientSocket, buffer, size, 0); // send to client
and on the client side a loop with an appropriate recv pulls. The same basically happens in every OMNeT++ simualtion.
Of course a UDP packet with around 20Kb payload will be fragmented on its way to pass good old IPv4 1500 bytes MTU at the backhaul part of a standard 5G architecture. So here comes my question: could I benefit something if I try to reduce my maximal UDP payload to, let's say, 1280 bytes to avoid IP fragmentation and to fit IPv6 minimum reassembly buffer size?
I'm afraid that if I just cluelessly send the encoded frame as in the code above, some fragmented packets may be lost and my decoder (h264 as well) on the client side can fail to decode the frame. Well however the same could happen if I send mine fragmented 1280 bytes packets... so here the question is pretty general, considering the fact that it happens in OMNeT++ simulation but with real video files: is there any advantage of controlling the packet size before sending or you just can cluelessly send any less than 64 Kb UDP datagram and just chill?
So I am seeding on BitTorrent and there appears to be two-fold communication showing up on Wireshark. From peers around the world I receive UDP packets with 20 bytes of data. In response my BitTorrent sends UDP packets with around 1438 bytes of data.
This uTorrent protocol suggested here does not seem to have anything as small as the 28 byte (20 bytes data 8 bytes header) UDP packets, likewise this link isn't helpful.
What is the formal communication mechanism or protocol at play here? Is it possible to analyze those 1438-byte packets or in more detail in order to get a snippet of the file being sent? Or the structure of the 20 bytes of data being sent from my peers?
This uTorrent protocol suggested here does not seem to have anything as small as the 28 byte (20 bytes data 8 bytes header) UDP packets
The µTP header is 20 bytes. So those most likely are ACK messages. Wireshark should support decoding those packets, at least if you captured a connection from the beginning.
a single block of secret information in 10 consecutive images of size 10 Kbytes generated every 100 ms. In order for Hermes to operate normally, all 10 images should be received within 5 seconds.
You should use TCP:
You need to receive all 10 images, which is 100K in size. This will not fit in a single IPv4 UDP datagram, so using UDP means you will have to send and keep track of at least 2 datagrams. Most source recommend much small UDP packets, so that means reassembling many smaller packets. This is not necessary in TCP.
With UDP, you will need to take care of sending acknowledgements and retransmitting one or more datagrams if the acknowledgement is not received. TCP takes care of this for you.
If your application cannot tolerate data loss, it is almost always better to start with TCP.
Is there any benifit to combining several UDP packets into one as opposed to sending them all one right after the other? I know that if the large packet gets courrupted then i loose all of them, but is there possibly some upside to sending them all in one? such as a lower chance of the large one being lost?
That would be at the discretion of the sending application.
Note that your large packet is limited by the MTU of the underlying network. e.g. the theoretical size of a UDP packet is 64k, but an ethernet frame is only ~1500 bytes. So I suspect this is not a practical feature.
Generally networking channels will be limited on the rate of packets that can be sent per second. Thus if you want to send millions of messages per second you generally want to combine into a smaller number of packets to run without major packet loss.
As an over generalisation, Windows doesn't like > 10,000 packets per second for UDP, but you can saturate a gigabit network with large MTU packets.
Is there any benifit to combining several UDP packets into one as opposed to sending them all one right after the other?
One can save on UDP header which is 8 bytes per datagram hence reducing the amount of data sent over the wire. Just make sure you don't send more then MTU sans IP and UDP header sizes to avoid fragmenting on IP layer.
Also, the standard POSIX socket API requires one send/sendto/sendmsg() system call to send or receive one datagram, so by sending fewer datagrams one does fewer system call reducing the overall latency (an order of a few microseconds per call). Linux kernels starting from 3.0 provide sendmsg() and recvmmsg() functions to send and receive multiple datagrams in one system call.
I know that if the large packet gets courrupted then i loose all of the
True. However, if the protocol can't cope with UDP datagram loss at all it may not matter that much - as soon as one datagram is lost it's broken anyway.
It is important for situations where packet size is small (less than 100 byte). The IP/UDP header is at least 28 bytes.
Imagine you have streaming connection to a server, each packet contains 50 bytes and your software sends packets with rate 1000 packet per second.
The actual payload is 1000 * 50 bytes = 50000 bytes. Headers overhead 1000 * 28 = 28000 bytes Total bytes : 50000 + 28000 = 87000 ==> 87 KBps
Imagine you can combine each 3 UDP packets into one packet:
Headers overhead 1000 / 3 * 28 = 9333 Total bytes : 50000 + 9333 ===> 60 KBps
This -in some applications- saves good portion of the bandwidth.
I want to know, how much overhead WebRTC produces when sending data over datachannels.
I know that Websockets have 2 - 14 Bytes overhead for each frame. Does WebRTC use more Overhead? I cannot find some useful information on the web. Its clear for me, that Datachannels can not be used for now. How much overhead is used with Mediastreams?
Thanks
At the application layer, you can think of DataChannel as sending and
receiving over SCTP. In the PPID (Payload Protocol Identifier) field of the
SCTP header, Datachannel sets value 0x51 for indicating that it's sending UTF-8
data and 0x52 for binary data.
Yes, you are right. RTCDataChannel uses SCTP over DTLS and UDP. DTLS is used for
security. However, SCTP has problems traversing most NAT/Firewall setups.
Hence, to overcome that, SCTP is tunneled through UDP. So the overall overhead
to send data would be overhead of:
SCTP + DTLS + UDP + IP
and that is:
28 bytes + 20-40 bytes + 8 bytes + 20 - 40 bytes
So, the overhead would be rougly about 120 bytes. The maximum size of the SCTP
packet that a WebRTC client can send is 1280 bytes. So at max, you can send
roughly 1160 bytes of data per SCTP packet.
WebRTC uses RTP to send its media. RTP runs over UDP.
Besides the usual IP and UDP headers, there are two additional headers:
The RTP header itself starts from 12 bytes and can grow from there, depending on what gets used.
The payload header - the header that is used for each data packet of the specific codec being used. This one depends on the codec itself.
RTP is designed to have as little overhead as possible over its payload due to the basic reasoning that you want to achieve better media quality, which means dedicating as many bits as possible to the media itself.
Here's a screenshot of 2 peer.js instances (babylon.js front end) sending exactly 3 bytes every 16ms (~60 per second).
The profiler shows 30,000 bits / second:
30,000 bits / 8 bits per byte / 60 per second = 62.5 bytes, so after the 3 bytes I'm sending it's ~59.5 bytes according to the profiler.
I'm not sure if something is not counted on the incoming, because it is only profiling half that, 15k bits / second