Can I use the same WebRTC channel for audio/video and file transfer? - webrtc

I am a newbie to WebRTC. I am building an application that enables users to view each other's video stream, as well as exchange files. The audio/video part is implemented and working. The problem is I need to add the ability to exchange files now. I am using the below code to initialize the PeerConnection object
var connection = _getConnection(partnerId);
console.log("Initiate offer")
// Add our audio/video stream
connection.addStream(stream);
// Send an offer for a connection
connection.createOffer(function (desc) { _createOfferSuccess(connection, partnerId, desc) }, function (error) { console.log('Error creating session description: ' + error); });
_getConnection creates a new RTCPeerConnection object using
var connection = new RTCPeerConnection(iceconfig);
i.e., with no explicit constraints. It also initializes the different event handlers on it. Right after this, I attach the audio/video stream to this connection. I also cache these connections using the partner id, so I can use it later.
The question is, can I later recall the connection object from the cache, add a data channel to it using something like
connection.createDataChannel("DataChannel", dataChannelOptions);
And use it to share files, or do I have to create a new RTCPeerConnection object and attach the data channel to it?

You certainly do not have to create a another PeerConnection for file transfer alone. Existing PeerConnection can utilize RTCDatachannel with behaves like traditional websocket mechanism ( ie 2 way communication without a central server )
`var PC = new RTCPeerConnection();
//specifying options for my datachannel
var dataChannelOptions = {
ordered: false, // unguaranted sequence
maxRetransmitTime: 2000, // 2000 miliseconds is the maximum time to try and retrsanmit failed messages
maxRetransmits : 5 // 5 is the number of times to try to retransmit failed messages , other options are negotiated , id , protocol
};
// createing data channel using RTC datachannel API name DC1
var dataChannel = PC.createDataChannel("DC1", dataChannelOptions);
dataChannel.onerror = function (error) {
console.log("DC Error:", error);
};
dataChannel.onmessage = function (event) {
console.log("DC Message:", event.data);
};
dataChannel.onopen = function () {
dataChannel.send(" Sending 123 "); // you can add file here in either strings/blob/array bufers almost anyways
};
dataChannel.onclose = function () {
console.log("DC is Closed");
};
`
PS : while sending files over datachannel API , it is advisable to break down the files into small chunks beforehand . I suggest chunk size of almost 10 - 15 KB .

Related

Socket.io - Is there a way to save socketid to prevent a new one being generated

After a connection to the socket.io server a socket.id is given for the connection. When the socket connection has not been used after some time a new socket id is generated.
I have read a lot of tutorials that do a "hello world" connection that just gets you connected, but, there is not much literature on messaging peer-to-peer/group. The docs give a 3 line paragraph on rooms/namespaces and every question related to this is just given a link to the same 3 line paragraph.
I understand that you can create and object/array of chats(in this example). For this example, let's say it is an object. That Object looks something like this:
const connections = {
"randomSocketID1": {
recipient: "Mom",
messages: [Array of Objects]
//more information
}
}
I then send a message to randomSocketID1 --> 'Hello'. Then next day I want to send another message to "Mom". Is that socketID going to be the same OR AT LEAST will "randomSocketID1" be updated under the hood, to its updated ID(which sounds improbable)? Is the regeneration of the socketID a product of garbage collection or a socket/engine/websocket protocol?
thanks for any clarification
So I was still unable to find an actual answer to this and by the 0 responses i see that no one knows. So what I have done in order to make sure that user and socket id are maintained is whenever a user enters the component that connects to the socketio server an automatic 'update-user' is emitted and the back end then just finds the user and assigns it the value.
So I have something like this:
chat.component.ts:
ngOnInit(){
this.socket.emit('update-user', 'Ctfrancia');
}
then in the back end:
const users = {};
io.on('connection', socket => {
socket.on('update-user', user => {
if (user in users) users[user] = socket.id;
else users[user] = socket.id
});
});

Is there any way to add custom property to a transceiver in Unified plan

I'm having difficulty identifying which track belongs to which media source on the receiving end. In unified plan is there any way to define custom properties on a transceiver?
I'm having difficulty identifying which track belongs to which media source on the receiving end.
Use transceiver.mid or the stream.id of an associated stream to correlate tracks.
A transceiver has a mid, which is a unique id that is the same on both sides of the connection after initial negotiation. It is exposed here:
pc.ontrack = event => {
const track = event.track;
const mid = event.transceiver.mid;
}
Alternatively, use addTransceiver(track, {streams: [stream]}) or addTrack(track, stream) and use the stream.id:
pc.ontrack = event => {
const track = event.track;
const id = event.streams[0].id;
}
In unified plan is there any way to define custom properties on a transceiver?
Any JS object can have a property defined on it. But I suspect that's not what you mean.
mid and stream.ids are the only metadata negotiated over to the remote peer connection, and there's no official way to add custom ones.
Once a connection has been established, you can of course use a datachannel to send whatever data you want over.
How to hack custom metadata
OK there is a way, but I hesitate to show it, since you haven't said what you'd use it for. Please consider the above options before resorting to this. Use at your own risk!
You can add any number of stream.ids and replace them in the SPD with whatever you want:
const config = {sdpSemantics: "unified-plan"};
const pc1 = new RTCPeerConnection(config), pc2 = new RTCPeerConnection(config);
const stream = new MediaStream();
pc1.addTransceiver("video", {streams: [stream]});
pc1.msg = "Hello";
pc2.ontrack = event => {
pc2.msg = event.streams[0].id;
console.log(pc2.msg);
};
pc1.onicecandidate = e => pc2.addIceCandidate(e.candidate);
pc2.onicecandidate = e => pc1.addIceCandidate(e.candidate);
pc1.onnegotiationneeded = async e => {
await pc1.setLocalDescription(await pc1.createOffer());
let sdp = pc1.localDescription.sdp.replace(new RegExp(stream.id, 'g'), pc1.msg);
await pc2.setRemoteDescription({type: "offer", sdp});
await pc2.setLocalDescription(await pc2.createAnswer());
await pc1.setRemoteDescription(pc2.localDescription);
}
I'm not actually recommending this, just showing it can be done. Any message you put in here is subject to SDP parsing rules, so be careful.

Issues with dat project's hyperdb in browser with webrtc and signalhub

I'm trying to use hyperdb in browser with swarming via webrtc and signalhub. The code is pretty strait forward, but there is some issue with hyperdb replicate where the connecting is killed because of a sameKey check in hypercore. So, I'm thinking ... I'm not properly juggling my discovery keys and id keys so the peers know they should be sync'd. Here is some sample code, it is a bit of a mess but the relevant bits are the hyperdb initialization and the webrtc/signalhub stuff (I think) ... the key at the top is the discovery key of the other peer:
const crypto = require('crypto'),
sha = crypto.createHash('sha1'),
hyperdb = require('hyperdb'),
hyperdiscovery = require('hyperdiscovery'),
cms = require('random-access-idb')('cms'),
webrtc = require('webrtc-swarm'),
signalhub = require('signalhub'),
hyperdrive = require('hyperdrive'),
pump = require('pump');
// Discovery key of other peer/signalhub channel
var key = "cbffda913dabfe73cbd45f64466ffda845383965e66b2aef5f3b716ee6c06528";
const db = hyperdb(filename => {
return cms(filename);
}, { valueEncoding: 'utf-8' });
var DEFAULT_SIGNALHUBS = 'https://signalhub-jccqtwhdwc.now.sh';
db.on('ready', function () {
const swarm = webrtc(signalhub(key, DEFAULT_SIGNALHUBS));
swarm.on('peer', function (conn) {
console.log("PEER!!!!!!!");
const peer = db.replicate({
upload: true,
download: true
});
pump(conn, peer, conn)
});
});
I put up a working example here: https://github.com/joehand/hyperdb-web-example/blob/master/index.js
I think you are getting that error because you are not initializing the db with the key:
var db = hyperdb(storage, key)
Once you do that, you can get the discovery key. Generally, you don't need to be copying the discovery key around because that is always generated from the public key.
If that does not work, please include only the relevant code or a minimum example, so it is easier to debug =). Thanks!

How to get stream object from kurento utils when using kurento java tutorial samples

Using kurento tutorials java samples. I Want to handle stream events like onended etc on the webrtcpeer object. Following is my sample code from where i want to fetch the stream object.
var options = {
localVideo: video,
mediaConstraints: constraints,
onicecandidate: participant.onIceCandidate.bind(participant)
};
var peer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendonly(options, function(error) {
if (error) {
return console.error(error);
}
this.generateOffer(participant.offerToReceiveVideo.bind(participant));
});
I want to handle events in a way similar to as mentions in this question
How should I proceed? Please Help
You can bind to those events two ways
Passing a onstreamended listener in the options bag
var options = {
localVideo: video,
mediaConstraints: constraints,
onicecandidate: participant.onIceCandidate.bind(participant),
onstreamended: myOnStreamEnded,
};
Accessing directly the RTCPeerConnection object wrapped inside the WebRtcPeer, and binding to events directly.
var rtcPeerConnection = peer.peerConnection
The latter gives you full access to the peer connection object, so you can work as if you would with that object.

Closing stream after using BitmapEncoder with WinJS Metro app

In a Windows 8 Metro application written in JS I open a file, get the stream, write some image data to it using the 'promise - .then' pattern. It works fine - the file is successfully saved to the file system, except after using the BitmapEncoder to flush the stream to the file, the stream is still open. ie; I can't access the file until I kill the application, but the 'stream' variable is out of scope for me to reference, so I can't close() it. Is there something comparable to the C# using statement that could be used?
...then(function (file) {
return file.openAsync(Windows.Storage.FileAccessMode.readWrite);
})
.then(function (stream) {
//Create imageencoder object
return Imaging.BitmapEncoder.createAsync(Imaging.BitmapEncoder.pngEncoderId, stream);
})
.then(function (encoder) {
//Set the pixel data in the encoder ('canvasImage.data' is an existing image stream)
encoder.setPixelData(Imaging.BitmapPixelFormat.rgba8, Imaging.BitmapAlphaMode.straight, canvasImage.width, canvasImage.height, 96, 96, canvasImage.data);
//Go do the encoding
return encoder.flushAsync();
//file saved successfully,
//but stream is still open and the stream variable is out of scope.
};
This simple imaging sample from Microsoft might help. Copied below.
It looks like, in your case, you need to declare the stream before the chain of then calls, make sure you don't name-collide with your parameter to your function accepting the stream (note the part where they do _stream = stream), and add a then call to close the stream.
function scenario2GetImageRotationAsync(file) {
var accessMode = Windows.Storage.FileAccessMode.read;
// Keep data in-scope across multiple asynchronous methods
var stream;
var exifRotation;
return file.openAsync(accessMode).then(function (_stream) {
stream = _stream;
return Imaging.BitmapDecoder.createAsync(stream);
}).then(function (decoder) {
// irrelevant stuff to this question
}).then(function () {
if (stream) {
stream.close();
}
return exifRotation;
});
}