Short radio link with a data source attached with a needed throughput of 1280 Kbps over IPv6 with a UDP Stop-and-wait protocol, no other clients or noticeable noise sources in the area. How on earth can I calculate what the best packet size is to minimise overhead?
UPDATE
I thought it would be an idea to show my working so far:
IPv6 has a 40 byte header, so including ACK responses, that's 80 bytes overhead per packet.
To meet the throughput requirement, 1280 K/p packets need to be sent a second, where p is the packet payload size.
So by my reckoning that means that the total overhead is (1280 K/p)*(80), and throwing that into Wolfram gives a function with no minima, so no 'optimal' value.
I did a lot more math trying to shoehorn bit error rate calculations into there but came up against the same thing; if there's no minima, how do I choose the optimal value?
Your best bet is to use a simulation framework for networks. This is a hard problem, and doesn't have an easy answer.
NS2 or SimPy can help you devise a discrete event simulation to find optimal conditions, if you know your model in terms of packet loss.
Always work with the largest packet size available on the network, then in deployment configure the network MTU for the most reliable setting.
Consider latency requirements, how is the payload being generated, do you need to wait for sufficient data before sending a packet or can you immediately send?
The radio channel is already optimized for noise as the low packet level, you will usually have other demands of the implementation such as power requirements: sending in heavy batches or light continuous load.
Related
I want to send unidirectional streaming data over a WebRTC datachannel, and is looking of the best configuration options (high BW, low latency/jitter) and others' experience with expected bitrates in this kind of application.
My test program sends chunks of 2k, with a bufferedAmountLowThreshold event callback of 2k and calls send again until bufferedAmount exceeds 16k. Using this in Chrome, I achieve ~135Mbit/s on LAN and ~20Mbit/s from/to a remote connection, that has 100Mbit/s WAN connection on both ends.
What is the limiting factor here?
How can I see if the data is truly going peer to peer directly, or whether a TURN server is used?
My ultimate application will use the google-webrtc library on Android - I'm only using JS for prototyping. Can I set options to speed up bitrate in the library, that I cannot do in official JS APIs?
There are many variables that impact throughput and it also highly depends on how you've measured it. But I'll list a couple of things I have adjusted to increase the throughput of WebRTC data channels.
Disclaimer: I have not done these adjustments for libwebrtc but for my own WebRTC data channel library called RAWRTC, which btw also compiles for Android. However, both use the same SCTP library underneath, both use some OpenSSL-ish library and UDP sockets, so all of this should be appliable to libwebrtc.
Note that WebRTC data channel implementations using usrsctp are usually CPU bound when executed on the same machine, so keep that in mind when testing. With RAWRTC's default settings, I'm able to achieve ~520 Mbit/s on my i7 5820k. From my own tests, both Chrom(e|ium) and Firefox were able to achieve ~350 Mbit/s with default settings.
Alright, so let's dive into adjustments...
UDP Send/Receive Buffer Size
The default send/receive buffer of UDP sockets in Linux is quite small by default. If you can, you may want to adjust it.
DTLS Cipher Suites
Most Android devices have ARM processors without hardware AES support. ChaCha20 usually performs better in software and thus you may want to prefer it.
(This is what RAWRTC negotiates by default, so I have not included it in the end results.)
SCTP Send/Receive Buffer Size
The default send/receive window size of usrsctp, the SCTP stack used by libwebrtc, is 256 KiB which is way too small to achieve high throughput with moderate delay. The theoretical maximum throughput is limited by mbits = (window / (rtt_ms / 1000)) / 131072. So, with the default window of window=262144 and a fairly moderate RTT of rtt_ms=20, you will end up with a theoretical maximum of 100 Mbit/s.
The practical maximum is below that... actually, way lower than the theoretical maximum (see my test results). This may be a bug in the usrsctp stack (see sctplab/usrsctp#245).
The buffer size has been increased in Firefox (see bug 1051685) but not in libwebrtc used by Chrom(e|ium).
Release Builds
Optimisation level 3 makes a difference (duh!).
Message Size
You probably want to send 256 KiB sized messages.
Unless you need to support Chrome < ??? (sorry, I currently don't know where it landed...), then the maximum message size is 64 KiB (see issue 7774).
Unless you also need to support Firefox < 56, in which case the maximum message size is 16 KiB (see bug 979417).
It also depends on how much you send before you pause sending (i.e. the buffer's high water mark), and when you continue sending after the buffer has been drained (i.e. the buffer's low water mark). My tests have shown that targeting a high water mark of 1 MiB and setting a low water mark of 256 KiB results in adequate throughput.
This reduces the amount of API calls and can increase throughput.
End Results
Using optimisation level 3 with default settings on RAWRTC brought me up to ~600 Mbit/s.
Based on that, increasing the SCTP and UDP buffer sizes to 4 MiB brought me up further to ~700 Mbit/s, with one CPU core at 100% load.
However, I believe there is still room for improvements but it's unlikely to be low-hanging.
How can I see if the data is truly going peer to peer directly, or whether a TURN server is used?
Open about:webrtc in Firefox or chrome://webrtc-internals in Chrom(e|ium) and look for the chosen ICE candidate pair. Or use Wireshark.
I am working with hardware which support USB3.0 interface. According to USB standard spec it support 5Gbps data rates.
Now I want to know what is best way of effective utilization of USB bandwidth.
What's the recommended maximum packet size and frame sizes?
In my scenario I need to send huge amount of data (60GB) as fast as possible. How do form these data into packets.
As per USB spec we can send 1frame/mille second, so to get 5Gbps data rate what is the number of packets/frame required?
Give some brief explanation on USB3.0 protocol.
Thank you.
The only possible reason that I could think of is the low overhead ie fixed header size of only 2 bytes minimum, leading to low packet size. Are there other factors in the design of the protocol?
EDIT:- I am sorry, I made a mental typo (?), as #Shashi pointed out, I actually meant high latency, low bandwidth.
MQTT is designed for devices with little memory footprint, low network bandwidth etc. Devices, for example sensors, energy meters, pace makers etc are ideal use cases for MQTT. Low latency means high speed. For low latency you require different protocol, like Reliable Multicast running over Gigabit Ethernet or InfiniBand networks.
One of the key factors is, that the TCP connection a MQTT client establishes is reused all the time. That means you don't have to establish a new connection all the time as it's the case with classic HTTP. Also, as you already suspected, the very low packet size is key here, typical MQTT messages don't have much overhead over the raw TCP packet.
To save more bandwidth on unreliable networks, the persistent session feature of MQTT allows clients to subscribe only once and on reconnect the subscriptions are retained for the client. For subscribing clients this can drastically reduce the overhead as the subscription message is only sent once.
Another reason, it seems is the Last Will and Testament feature, which is a useful to have feature in high latency network, low bandwidth and unreliable networks.
It seems that UDP uses a fairly simple checksum algorithm and mostly it would produce a desired result and detect errors.
However, as far as I can tell, the header for UDP is the same when using Jumbo frames and the checksum is the same. It's only 16 bits.
Would this increase the likelihood that errors would seep in and that the protocols that use UDP (especially those that use large packets) should not rely on the UDP checksum at all?
As you pointed out, UDP uses a 16-bit checksum regardless of how large the packet is. UDP checksums are unable to detect errors if the change does not alter the sum of all the 16-bit words. This is more likely to occur as the packet size increases.
However, UDP normally runs on top of ethernet, which provides a 32-bit CRC, which is much more robust than the UDP checksum. Thus, most people rely on the etherframe to guarantee data integrity.
Large UDP packets will be fragmented into multiple IP packets, each in its own ethernet packet with a 32-bit CRC. Thus, sending large UDP packets is not a concern from a data integrity standpoint, although IP fragmentation can increase packet loss and re-ordering.
Not very reliable, and below is a scenario when it fails:
1011 + 0100 = 1111, which is the checksum;
the information got corrupted into below,
1100 + 0011 = 1111, but their sum stay the same.
Checksum algorithm wouldn't detect it; it happens less often in 16-bit, but still a big problem.
Large frames will need to be fragmented, and each fragment will have a checksum associated to it. You are correct that the size of the header doesn't change, but the amount of packets sent will change, thus increasing the amount of checksum bits sent. I hope this helps.
I have a very low speed data connection over serial (RS485):
9600 baud
actual data transmission rate is about 25% of that.
The serial line is going through an area of extremely high EMR. Peak fluctuations can reach 3000 KV.
I am not in the position (yet) to force a change in the physical medium, but could easily offer to put in a simple robust forward error correction scheme. The scheme needs to be easy to implement on a PIC18 series micro.
Ideas?
This site claims to implement Reed-Solomon on the PIC18. I've never used it myself, but perhaps it could be a helpful reference?
Search for CRC algorithm used in MODBUS ASCII protocol.
I develop with PIC18 devices and currently use the MCC18 and PICC18 compilers. I noticed a few weeks ago that the peripheral headers for PICC18 incorrectly map the Busy2USART() macro to the TRMT bit instead of the TRMT2 bit. This caused me major headaches for short time before I discovered the problem. Example, a simple transmission:
putc2USART(*p_value++);
while Busy2USART();
putc2USART(*p_value);
When the Busy2USART() macro was incorrectly mapped to the TRMT bit, I was never waiting for bytes to leave the shift register because I was monitoring the wrong bit. Before I realized the inaccurate header file, the only way I was able to successfully transmit a byte over 485 was to wait 1 ms between bytes. My baud rate was 91912 and the delays between bytes killed my throughput.
I also suggest implementing a means of collision detection and checksums. Checksums are cheap, even on a PIC18. If you are able to listen to your own transmissions, do so, it will allow you to be aware of collisions that may result from duplicate addresses on the same loop and incorrect timings.