Multi party video chat- Detect the peer speaker - webrtc

Question

I have implemented the multiparty video chat (audio and video enabled) and it is working fine. How can I figure out which peer is speaking and highlight a green icon besides that user.
The onStreamAdded gets called only when a new stream is added but how do I track
who is speaking currently.
Regards
Raghav

Answer 1

Take a look at the Otalk hark github project.
Hark is a tiny browser/commonJS module that listens to an audio stream, and emits events indicating whether the user is speaking or not. Hark uses the webaudio API to FFT (get the power of) the audio in the audio stream. If the power is above a threshold, it's determined to be speech.
https://github.com/otalk/hark

Answer 2

Hi you can use below logic to show active user on page.
Typescript:-
class AudioListenerBase {
private audio_progress: number = 0;
private audioContext = new AudioContext();
private analyser: AnalyserNode;
private microphone: MediaStreamAudioSourceNode;
private javascriptNode: ScriptProcessorNode;
public remotesElement: any;
constructor(
private zone: NgZone,
private cd: ChangeDetectorRef,
private stream: any,
private audioProgressCallBack: any
) {
this.analyser = this.audioContext.createAnalyser();
this.microphone = this.audioContext.createMediaStreamSource(stream);
this.javascriptNode = this.audioContext.createScriptProcessor(2048, 1, 1);
this.analyser.smoothingTimeConstant = 0.8;
this.analyser.fftSize = 1024;
this.microphone.connect(this.analyser);
this.analyser.connect(this.javascriptNode);
this.javascriptNode.connect(this.audioContext.destination);
this.javascriptNode.onaudioprocess = (() => {
var array = new Uint8Array(this.analyser.frequencyBinCount);
this.analyser.getByteFrequencyData(array);
var values = 0;
var length = array.length;
for (var i = 0; i < length; i++) {
values += (array[i]);
}
var average = (values / length) * 10;
if (this.audio_progress - average > 5 || average - this.audio_progress > 5)
this.zone.run(() => {
this.audio_progress = average;
this.cd.detectChanges();
audioProgressCallBack(this.audio_progress, this.remotesElement)
});
});
return this;
}
}
usage on component :-
this.myAudioListener = new AudioListenerBase(this.zone, this.changeDetectorRef, stream, (val, remotesElement) => {
this.audio_progress = val;
});
On component Html:
<div> <p>{{item.username}}</p> <p style="font-size:10px">{{item.audio_progress>20?'speaking..':''}}</p></div>