How to detect network disconnects (on an RTCPeerConnection) as soon as possible or the resulting frozen video? - webrtc

Question

I am using RTCPeerconnections to submit video and audio in a web-RTC-based video-messenger. I am able to detect network disconnects after approximately 7 seconds - yet, this is 7 seconds in which the user is staring at a frozen video and starts to randomly click buttons in the app. I would like to improve the user experience by shortening this timespan - e.g. by informing the user about a network issue if the video freezes for more than 1 second.
Status Quo: I am currently detecting respective situations by listening to the onconnectionstatechange event of the RTCPeerConnection. Yet, the event is only fired approximately 7 seconds after the disconnect. I determined the ~7 seconds by connecting two machines via normal WiFi, using a hardware switch on one of the laptops to switch off the wireless (such switches exist on some older Lenovo models / guarantee an immediate disconnect) and wait for the other machine to detect the event.
Consideration: The root cause being the interruption of the underlying network connection, it would be ideal to detect the changed network status as early as possible (even if its just transport delays). This said, the disturbance faced by the user ultimately stems from the video that instantly freezes when interrupting the network. If there was no way to detect the connection issue earlier, it could be an option to detect the frozen video instead. Is any of these two things possible (ideally event-driven, so that I don't need to poll things every second)?
Here's a very simple code snippet describing my current disconnect detection:
myRTCPeerConnection.onconnectionstatechange = (event: Event) => {
let newCS = myRTCPeerConnection.connectionState;
if (newCS == "disconnected" || newCS == "failed" || newCS == "closed") {
//do something on disconnect - e.g. show messages to user and start reconnect
}
}

Answer 1

(ice)connectionstatechange is the right event in general.
If you want more granularity you'll need to poll getStats and looks for stats like framesReceived. But there is no guaranteed frame rate sent from the other side (e.g. in screensharing you go below 1/s).
While the actual ICE statistics like requestsSent seem more useful they happen much less frequently, only once per second and you can loose a packet or it comes late.
In general this is a question of how reliable the detection of the network failure is. If it is too aggressive you end up with a poor UX showing a warning too often.
You might not end up that is significantly better than at the cost of introducing complexity that you need to maintain.

Answer 2

Thanks Philipp for your insights - this pointed me into the right direction.
I'm now looking into using getStats to identify any freezes. At first sight, polling the framesPerSecond value seems most promising to me. The good thing: it reacts instantly upon disconnect - and - it still works when the underlying video stream is paused (i'm allowing the user to pause video submission / implemented it by setting all video tracks to enabled = false). I.e. even if the video tracks are disabled on the sending side, the receiving side still continues to receive the agreed frames per second.
As the usage of the getStats function appears weak on documentation at the time of this being written / there's rarely a simple examples for its usage, please find my code extract below:
peerRTCPC
.getReceivers()
.forEach(
(
receiver: RTCRtpReceiver,
index: number,
array: RTCRtpReceiver[]
) => {
if (receiver.track.kind == "video") {
receiver.getStats().then((myStatsReport: RTCStatsReport) => {
myStatsReport.forEach(
(statValue: any, key: string, parent: RTCStatsReport) => {
if (statValue.type == "inbound-rtp") {
console.log(
"The PC stats returned the framesPerSecond value " +
statValue["framesPerSecond"] +
" while the full inbound-rtp stats reflect as " +
JSON.stringify(statValue)
);
}
}
);
});
}
}
);
Note that upon disconnect, the framesPerSecond do not necessarily go to zero, even though the webRTCInternals screen suggests the same. I am seeing undefined when a disconnect happens.
Runtime impact of polling this at high frequency / across larger numbers of connections probably needs to be looked at more closely. Yet, this seems like a good step into the right direction unless doing it way to frequently.