How to stream audio file with opentok? - webrtc

Question

In opentok, with OT.initPublisher, you only can pass a deviceId to the audioSource. Does someone know a method to stream an audio file ?
For example, I have done this:
navigator.getUserMedia({audio: true, video: false},
function(stream) {
var context = new AudioContext();
var microphone = context.createMediaStreamSource(stream);
var backgroundMusic = context.createMediaElementSource(document.getElementById("song"));
var mixedOutput = context.createMediaStreamDestination();
microphone.connect(mixedOutput);
backgroundMusic.connect(mixedOutput);
},
handleError);
Like this, I can have a stream with the voice and my music but how to put this stream to a publisher ? Is it possible or is there another way to do this ?

Answer 1

Update: There is now an official way to do this, using the videoSource and audioSource properties provided to OT.initPublisher, please see the documentation: https://tokbox.com/developer/sdks/js/reference/OT.html#initPublisher
This is an example of how to stream a canvas element as a video track: https://github.com/opentok/opentok-web-samples/tree/master/Publish-Canvas
You can apply the same technique to stream an audio track.
Old Answer:
It's not currently possible with the officially supported API but there is a way to do it.
Please see the TokBox blog post about Camera Filters: https://tokbox.com/blog/camera-filters-in-opentok-for-web/
In order to modify the stream before it reaches the OpenTok JS SDK we use the mockGetUserMedia function to intercept the stream:
https://github.com/aullman/opentok-camera-filters/blob/master/src/mock-get-user-media.js
You could invoke mockGetUserMedia with a function which does your audio mixing. Something like this:
mockGetUserMedia(function(originalStream) {
var context = new AudioContext();
var microphone = context.createMediaStreamSource(originalStream);
var backgroundMusic = context.createMediaElementSource(document.getElementById("song"));
var mixedOutput = context.createMediaStreamDestination();
microphone.connect(mixedOutput);
backgroundMusic.connect(mixedOutput);
var stream = mixedOutput.stream;
originalStream.getVideoTracks().map(function(track) {
stream.addTrack(track);
});
return stream;
});
Note: I have not tested this function but it should lead you in the right direction. Remember that this technique is error prone and not officially supported by TokBox.
We are currently working on a new feature which will enable this use case but I cannot give a time estimate of when it will be available.

Answer 2

Thank you for the help but we cannot make it work since this morning.
So we made a different file with this code which is implemented before the opentok library in our html :
function mockGetUserMedia(mockOnStreamAvailable) {
var oldGetUserMedia = void 0;
if (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia) {
oldGetUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
navigator.webkitGetUserMedia = navigator.getUserMedia = navigator.mozGetUserMedia = function getUserMedia(constraints, onStreamAvailable, onStreamAvailableError, onAccessDialogOpened, onAccessDialogClosed, onAccessDenied) {
return oldGetUserMedia.call(navigator, constraints, function (stream) {
onStreamAvailable(mockOnStreamAvailable(stream));
}, onStreamAvailableError, onAccessDialogOpened, onAccessDialogClosed, onAccessDenied);
};
} else {
console.warn('Could not find getUserMedia function to mock out');
}
};
mockGetUserMedia(function(stream) {
var context = new AudioContext();
var bgMusic = context.createMediaElementSource(document.getElementById("song"));
var microphone = context.createMediaStreamSource(stream);
var destination = context.createMediaStreamDestination();
bgMusic.connect(destination);
microphone.connect(destination);
var mixedStream = destination.stream;
stream.getVideoTracks().map(function(track) {
mixedStream.addTrack(track);
});
return mixedStream;
});
In our angular, we init the session, create a publisher and publish it but get the error :
Uncaught DOMException: Failed to execute 'createMediaElementSource' on 'BaseAudioContext': HTMLMediaElement already connected previously to a different MediaElementSourceNode.
This error, I think, is throw because the function is executed twice. When the js load, and when we publish.
I am not sure how to use this mockGetUserMedia function, do you know what is wrong with our code ?
EDIT
We made it work with some if condition. Thank you so much man, very appreciated.