Implement Live Broadcast by Agora.io - react-native

I am trying to implement Live Broadcast by Agora.io in React Native mobile application. I have previously implemented video call successfully. I have gone through documentation, compare and contrast video call to live broadcast ( both web sdk). I could only find a difference in mode of the client which corresponds to channelProfile in react-native sdk. In documentation it says there are three different modes: Communication, Live Broadcast and Gaming. When I implemented video call I assigned 1 for the value of channelProfile, it worked fine, quality was good enough. However, when I assign 2 for channelProfile to indicate it is a Live Broadcast, the quality goes heavily down. Do I make anything wrong while implementation of Live Broadcast? How can I improve the quality of Live Broadcast?
For consideration, I put my code below:
const config = {
appid: 'MY APP ID',
channelProfile: this.props.navigation.getParam('channelProfile', 2),
clientRole: this.props.navigation.getParam('clientRole', 1),
videoEncoderConfig: {
width: 360,
height: 480,
bitrate: 1,
frameRate: FPS30,
orientationMode: Adaptative,
},
audioProfile: AudioProfileDefault,
audioScenario: AudioScenarioDefault
}
RtcEngine.on('userJoined', (data) => {
console.warn("user joined", data);
const { peerIds } = this.state;
if (peerIds.indexOf(data.uid) === -1) {
this.setState({
peerIds: [...this.state.peerIds, data.uid]
})
}
})
RtcEngine.on('error', (error) => {
console.warn("error", error);
})
RtcEngine.init(config);

In Agora's SDK there used to be three channel-modes, but recently the gaming SDK has been combined with the native SDKs so there are only two channel-modes, communication and broadcast.
Each mode optimizes for different qualities within the channel and within the streams. For broadcasting the documentation mentions that when using default bitrate that broadcast mode uses twice the bitrate of communication.
If you are having quality issues you should consider changing your bitrate, currently your code is setting the bitrate to 1 which is very low. Agora provides a list of suggested resolution profiles, fps, and bit rates.
Agora Video Bitrate documentation: https://docs.agora.io/en/Interactive%20Broadcast/API%20Reference/oc/Classes/AgoraVideoEncoderConfiguration.html#//api/name/bitrate

Related

How can I use MediaRecorder API of browser in react native

I just want to use media recorder API to record the stream which I'm using in WebRtc for react-native. Actually I want to save the stream to a file on the backend server. and hence i need to send the data of buffers through the sockets once the media recorder's function (ondataavailable) is available. I've implemented it in the Web (using React.js)
This is the code I'm using to send the recorded chunks to the backend server
const mediaRecorder = new MediaRecorder(_stream.current, recorderOptions);
mediaRecorder.ondataavailable = (event) => {
if (event.data && event.data.size > 0) {
socket.emit("message1", event.data);
}
};
Or any other solution would be highly recommended. Thanks

Set grid layout in video call UI (web SDK)

I want to achieve grid layout in my video call application which I am building with Agora's web SDK.
I was browsing the docs , but I couldn't get an help on how to achieve grid layout in video conferencing.
The best fit and grid layouts are only available in cloud recording APIs.
Any previous reference or github repo where it is implemented would also work.
Thanks for the help!
The Agora Web SDK provides a library for video streaming, it does not enforce a UI. Building the UI is your task. That being said, Agora makes its very easy to add video chat to your application.
In your case you can build a grid layout using CSS grid or any framework of your choosing. To connect Agora to your Grid layout you would use the stream-published event to create a new grid element, and subscribe to the new stream. Once the subscribe() promise resolves, use the video track's .play() method to play the video on a specific DOM element
client.on("user-published", async (user, mediaType) => {
// Initiate the subscription
await client.subscribe(user, mediaType);
// If the subscribed track is an audio track
if (mediaType === "audio") {
const audioTrack = user.audioTrack;
// Play the audio
audioTrack.play();
} else {
const videoTrack = user.videoTrack;
// Play the video the given DOM_ELEMENT
videoTrack.play(DOM_ELEMENT);
}
});

TokBox/Vonage allowing audio capture support when screensharing

Screen Capture API, specifically getDisplayMedia(), currently supports screensharing and sharing the audio playing in your device (e.g: youtube) at the same time. Docs. Is this currently supported using TokBox/Vonage Video API? Has someone been able to achieve this?
I guess there could be some workaround using getDisplayMedia and passing the audio source when publishing, e.g: OT.initPublisher({ audioSource: newDisplayMediaAudioTrack }), but doesn't seem like a clean solution.
Thanks,
Manik here from the Vonage Client SDK team.
Although this feature does not exist in the Video Client SDK just yet, you can accomplish the sharing of audio with screen by creating a publisher like so:
let publisher;
try {
const stream = await navigator.mediaDevices.getDisplayMedia({video: true, audio: true });
const audioTrack = stream.getAudioTracks()[0];
const videoTrack = stream.getVideoTracks()[0];
publisher = OT.initPublisher({audioSource: audioTrack, videoSource: videoTrack});
} catch (e) {
// handle error
}
If you share a tab, but the tab doesn't play audio (static pdf or ppt) then the screen flickers. To avoid this, specify frameRate constraint for the video stream. see - https://gist.github.com/rktalusani/ca854ca8621c20488bea6e62ad04e341

How to set the remote notification sound for iOS in react native app?

Im using react-native-firebase and firebase cloud messaging for remote notifications (v5.6.0), and I have my firebase cloud function's notification payload set. Im trying to set the notification sound for iOS/android. what I have currently is:
exports.sendChatNotification = functions.firestore.document('messages/{document}').onCreate((event) => {
const writeData = event.data();
const { receiverToken, senderName, content } = writeData;
const payload = {
notification: {
title: 'Chat Message',
body: senderName + ' has sent you a message',
sound: 'Complete'
}
};
return admin.messaging().sendToDevice(receiverToken, payload).then((msgDevicesResponse) => {
console.log(JSON.stringify(msgDevicesResponse));
});
});
Currently experiencing this notification sound playing the default for iOS, and am unable to per-select an existing iOS tone, such as "Complete". I understand from reading firebase docs that the 'sound' property of the notification payload only works for the android platform.
Since im using react-native-firebase, I looked at their documentation for Cloud Messaging, and found this Notification object that has iOS and android properties: https://rnfirebase.io/reference/messaging/notification
both platform properties have a sub-property sound, but Im unsure of how to integrate this, considering my notifications are being set in a firebase cloud function, which uses firebase.admin
My goal is to set the iOS notification sound from the os's list of tones. How can this be done?

Is it possible broadcast audio with screensharing with WebRTC

is it possible broadcast audio with screensharing with WebRTC?
Simple calling getUserMedia with audio: true fails by permission denied error.
Is there any workeround which could be used to broadcast audio also?
Will be audio implemented beside screensharing?
Thanks.
Refer this demo: Share screen and audio/video from single peer connection!
Multiple streams are captured and attached to a single peer connection. AFAIK, audio alongwith chromeMediaSource:screen is "still" not permitted.
Updated at April 21, 2016
Now you can capture audio+screen using single getUserMedia request both on Firefox and Chrome.
However Chrome merely supports audio+tab i.e. you can NOT capture full-screen along with audio.
Audio+Tab means any chrome tab along with microphone.
Updated at Jan 09, 2017
You can capture both audio and screen streams by making two parallel (UNIQUE) getUserMedia requests.
Now you can use addTrack method to add audio tracks into screen stream:
var audioStream = captureUsingGetUserMedia();
var screenStream = captureUsingGetUserMedia();
var audioTrack = audioStream.getAudioTracks()[0];
// add audio tracks into screen stream
screenStream.addTrack( audioTrack );
Now screenStream has both audio and video tracks.
nativeRTCPeerConnection.addStream( screenStream );
nativeRTCPeerConnection.createOffer(success, failure, options);
As of May 2020
To share the audio track of the screen share you can use getDisplayMedia instead of getUserMedia. Docs.
navigator.mediaDevices.getDisplayMedia({audio: true, video: true})
This is currently only supported in Chrome / Edge and it is only supported when using the "Chrome Tab" sharing option. You'll see a checkmark for Share audio in the dialog box.
In Firefox, you can use getUserMedia to grab a screenshare/etc and mic audio in the same request, and can attach it to a PeerConnection. You can combine it with other streams -- multiple audio or video tracks in a single PeerConnection in Firefox requires Firefox 38 or later. Currently 38 is Developer Edition (formerly termed Aurora). 38 should go to release in around 9 weeks or so.
yes you can record audio and screen record on chrome with two requests.
getScreenId(function (error, sourceId, screen_constraints) {
capture screen
navigator.getUserMedia = navigator.mozGetUserMedia || navigator.webkitGetUserMedia;
navigator.getUserMedia(screen_constraints, function (stream) {
navigator.getUserMedia({audio: true}, function (audioStream) {
stream.addTrack(audioStream.getAudioTracks()[0]);
var mediaRecorder = new MediaStreamRecorder(stream);
mediaRecorder.mimeType = 'video/mp4'
mediaRecorder.stream = stream;
document.querySelector('video').src = URL.createObjectURL(stream);
var video = document.getElementById('screen-video')
if (video) {
video.src = URL.createObjectURL(stream);
video.width = 360;
video.height = 300;
}
}, function (error) {
alert(error);
});
}, function (error) {
alert(error);
});
});