I am able to make a direct call between a Circuit WebClient and the example SDK app at https://output.jsbin.com/posoko.
When running the SDK example on a PC with a second camera (USB), the switching between the built-in camera and the USB camera works fine. But trying the same on my Android device (Samsung Galaxy S6) the switching does not work.
My code uses navigator.mediaDevices.enumerateDevices() to get the cameras and then uses the Circuit SDK function setMediaDevices to switch to the other camera.
async function switchCam() {
let availDevices = await navigator.mediaDevices.enumerateDevices();
availDevices = availDevices.filter(si => si.kind === 'videoinput');
let newDevice = availDevices[1]; // secondary camera
await client.setMediaDevices({video: newDevice.deviceId})
}
Can somebody explain why this doesn’t work on an Android device?
We have seen Android devices that don't allow calling navigator.getUserMedia while a video track (and therefore a stream) is still active. I tried your example above with a Pixel 2 without any issues though.
If you remove the video track from the stream and stop the track before calling client.setMediaDevices, the switch should work.
async function switchCam() {
const stream = await client.getLocalAudioVideoStream();
const currTrack = stream.getVideoTracks()[0];
console.log(`Remove and stop current track: ${currTrack.label}`);
stream.removeTrack(currTrack);
currTrack.stop();
let availDevices = await navigator.mediaDevices.enumerateDevices();
availDevices = availDevices.filter(si => si.kind === 'videoinput');
let newDevice = availDevices[1]; // secondary camera
await client.setMediaDevices({video: newDevice.deviceId})
}
There is a complete switch camera example on JSBin at https://output.jsbin.com/wuniwec/
Related
I have tried two use cases to play webrtc playback on videojs.
After getting MediaStream from webrtc and add like the following:
player.src({src:webRTCAdaptor.remoteVideo.srcObject});
I'm getting (CODE:4 MEDIA_ERR_SRC_NOT_SUPPORTED) error.
If i do like following, I'm not getting any error but video does not play either.
var vid=player.tech().el();
vid.srcObject=webRTCAdaptor.remoteVideo.srcObject;
Calling player.play() doesn't change anything.
Does anybody has any insight about it?
You can rewrite the play function to achieve，like this
if (player) {
const videoDom = player.tech().el()
videoDom && (videoDom.srcObject = stream)
player.play = () => {
videoDom.play()
}
player.play()
}
I am using the JavaScript version of Microsoft Cognitive Services Speech SDK from https://github.com/Azure-Samples/cognitive-services-speech-sdk.
The audio is played by the browser when synthesizer.speakTextAsync is called. When the audio is too long I want to stop the audio play but I couldn't find any documentation on how to do that?
Any help is appreciated!
synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig,
SpeechSDK.AudioConfig.fromDefaultSpeakerOutput());
synthesizer.speakTextAsync(
inputText,
result => {
if (result) {
console.log(JSON.stringify(rssesult));
}
},
error => {
console.log(error);
}
);
Stopping audio playing is supported.
You need to create a SpeechSDK.SpeakerAudioDestination() object and use it to create audioConfig like this.
var player = new SpeechSDK.SpeakerAudioDestination();
var audioConfig = SpeechSDK.AudioConfig.fromSpeakerOutput(player);
var synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig, audioConfig);
synthesizer.speakTextAsync(
...
);
Then you can call player.pause() and player.resume() to pause and resume the playback.
You can find more info from the doc and sample.
I am taking a MediaStream and merging two separate tracks (video and audio) using a canvas and the WebAudio API. The MediaStream itself does not seem to fall out of sync, but after reading it into a MediaRecorder and buffering it into a video element the audio will always seem to play much earlier than the video Here's the code that seems to have the issue:
let stream = new MediaStream();
// Get the mixed sources drawn to the canvas
this.canvas.captureStream().getVideoTracks().forEach(track => {
stream.addTrack(track);
});
// Add mixed audio tracks to the stream
// https://stackoverflow.com/questions/42138545/webrtc-mix-local-and-remote-audio-steams-and-record
this.audioMixer.dest.stream.getAudioTracks().forEach(track => {
stream.addTrack(track);
});
// stream = stream;
let mediaRecorder = new MediaRecorder(stream, { mimeType: 'video/webm;codecs=opus,vp8' });
let mediaSource = new MediaSource();
let video = document.createElement('video');
video.src = URL.createObjectURL(mediaSource);
document.body.appendChild(video);
video.controls = true;
video.autoplay = true;
// Source open
mediaSource.onsourceopen = () => {
let sourceBuffer = mediaSource.addSourceBuffer(mediaRecorder.mimeType);
mediaRecorder.ondataavailable = (event) => {
if (event.data.size > 0) {
const reader = new FileReader();
reader.readAsArrayBuffer(event.data);
reader.onloadend = () => {
sourceBuffer.appendBuffer(reader.result);
console.log(mediaSource.sourceBuffers);
console.log(event.data);
}
}
}
mediaRecorder.start(1000);
}
AudioMixer.js
export default class AudioMixer {
constructor() {
// Initialize an audio context
this.audioContext = new AudioContext();
// Destination outputs one track of mixed audio
this.dest = this.audioContext.createMediaStreamDestination();
// Array of current streams in mixer
this.sources = [];
}
// Add an audio stream to the mixer
addStream(id, stream) {
// Get the audio tracks from the stream and add them to the mixer
let sources = stream.getAudioTracks().map(track => this.audioContext.createMediaStreamSource(new MediaStream([track])));
sources.forEach(source => {
// Add it to the current sources being mixed
this.sources.push(source);
source.connect(this.dest);
// Connect to analyser to update volume slider
let analyser = this.audioContext.createAnalyser();
source.connect(analyser);
...
});
}
// Remove all current sources from the mixer
flushAll() {
this.sources.forEach(source => {
source.disconnect(this.dest);
});
this.sources = [];
}
// Clean up the audio context for the mixer
cleanup() {
this.audioContext.close();
}
}
I assume it has to do with how the data is pushed into the MediaSource buffer but I'm not sure. What am I doing that de-syncs the stream?
A late reply to an old post, but it might help someone ...
I had exactly the same problem: I have a video stream, which should be supplemented by an audio stream. In the audio stream short sounds (AudioBuffer) are played from time to time. The whole thing is recorded via MediaRecorder.
Everything works fine on Chrome. But on Chrome for Android, all sounds were played back in quick succession. The "when" parameter for "play()" was ignored on Android. (audiocontext.currentTime continued to increase over time ... - that was not the point).
My solution is similar to Jacob's comment Sep 2 '18 at 7:41:
I created and connected a sine wave oscillator with inaudible 48,000 Hz, which played permanently in the audio stream during recording. Apparently this leads to the proper time progress.
An RTP endpoint that is emitting multiple related RTP streams that
require synchronization at the other endpoint(s) MUST use the same
RTCP CNAME for all streams that are to be synchronized. This
requires a short-term persistent RTCP CNAME that is common across
several RTP streams, and potentially across several related RTP
sessions. A common example of such use occurs when lip-syncing audio
and video streams in a multimedia session, where a single participant
has to use the same RTCP CNAME for its audio RTP session and for its
video RTP session. Another example might be to synchronize the
layers of a layered audio codec, where the same RTCP CNAME has to be
used for each layer.
https://datatracker.ietf.org/doc/html/rfc6222#page-2
There is a bug in Chrome, that plays buffered media stream audio with 44100KHz, even when it's encoded with 48000 (which leads to gaps and video desync). All other browsers seem to play it fine. You can choose to change codec to the one which supports 44.1KHz encoding or play a file from web link as a source (this way Chrome can play it correctly)
Initially, I had two different webpages:
One was to do Video Call and
Other was to do Screen Sharing
Now, I want to do both of them in one page.
Here is the scenario:
During Live call, a user wants to stop sharing his/her video and start sharing screen.
Afterwards, again he/she wishes to turn off screen sharing and start video sharing.
For clarity, here are some questions I want to ask:
On Caller Side:
1) How can I change my local stream from video to screen and vice versa?
2) Once it is done, how can I assign it to the local video element?
On Callee Side:
1) How do I handle if the current stream I am receiving is changed from video to screen?
2) How do I handle if the stream I am receiving has stopped? I mean, now I can receive neither video nor screen (just audio)
Kindly, help me in this regards. If there are any open source codes available, kindly share their links too.
Just for your reference, I was trying to handle it using following code. (i know this is naive and won't work)
function handleUserMedia(newStream){
var localvideo = document.getElementById("localvideo");
localvideo.src = URL.createObjectURL(newStream);
localStream = newStream;
sendMessage('got user media');
if (isInitiator) {
maybeStart();
}
}
function handleUserMediaError(error){
console.log(error);
}
var video_constraints = {video: true, audio: true};
var screen_constraints = {video: { mandatory: { chromeMediaSource: 'screen' } }};
getUserMedia(video_constraints, handleUserMedia, handleUserMediaError);
//getUserMedia(screen_constraints, handleUserMedia, handleUserMediaError);
$scope.btnLabel = 'Share Screen';
$scope.toggleSelected = function () {
$scope.selected = !$scope.selected;
if($scope.selected)
{
getUserMedia(screen_constraints, handleUserMedia, handleUserMediaError);
$scope.btnLabel = 'Share Video';
}
else
{
getUserMedia(video_constraints, handleUserMedia, handleUserMediaError);
$scope.btnLabel = 'Share Screen';
}
};
Check this demo:
https://www.webrtc-experiment.com/demos/switch-streams.html
and the relevant tutorial:
https://www.webrtc-experiment.com/docs/how-to-switch-streams.html
simply renegotiate peer connections on both users' side!
I want to use multiple USB camera with Web RTC.
For ex)
https://apprtc.appspot.com/?r=93443359
This application is web RTC sample.
I can connect to another machine, but I have to disconnect once to change the camera.
What I want to is,
1.Use two camera at the same time on the same screen.
2.(if 1 is not possible),I want to switch the camera without disconnecting current connection
Does anyone have information about how to use two camera on Web RTC?
call getUserMedia twice and change the camera input in between
You can use constraints to specify which camera to use and you can have both of them displayed in one page as well. To specify which camera to use take a look at the following snippet (only works on Chrome 30+):
getUserMedia({
video: {
mandatory: {
sourceId: webcamId,
...
}
},
successCallback,
failCallback);
The webcamId you can get by:
MediaStreamTrack.getSources(function(sources){
var cams = _.filter(sources, function(e){ //only return video elements
return e.kind === 'video';
});
var camIds = _.map(cams, function (e) { // return only ids
return e.id;
});
});
In the snippet above I've used underscore methods filter and map.
More information on:
WebRTC video sources
constraints