In a scenario where an endpoint adds multiple video streams to a peer connection, the onaddstream event handler is invoked multiple times on the peer end.
Is there any means by which an application (on the peer end) can determine between the different video streams (within onaddstream handler)? By identifying the stream, each stream can be associated with a different video element.
RTCPeerConnection has a couple of methods:
getRemoteStreams
getLocalStreams
These methods return an array of MediaStream's associated with the remote/local end of the connection. Every MediaStream has an id, so you will be able to identify them.
There is also getStreamById, but it is deprecated, replaced by the two above.
Related
I'm building a video calling app using WebRTC which allows one peer to call another by selecting someone in the lobby. When peer A sends a call request, the other peer B can accept. At this point, WebRTC signaling starts:
Both peers get their local media using MediaDevices.getUserMedia()
Both peers create an RTCPeerConnection and attach event listeners
Both peers calls RTCPeerConnection.addTrack() to add their local media
One peer A (the impolite user) creates an offer, calls RTCPeerConnection.setLocalDescription() to set that offer as the local description, and sends it to the WebSocket server, which forwards it to the other peer B.
The other peer B receives this offer and adds calls RTCPeerConnection.setRemoteDescription() to record it as the remote description
The other peer B then creates an answer and transmits it again to the first peer A.
(Steps based on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity)
This flow is almost working well. In 1 out of 10 calls, I receive no video/audio from one of the peers (while both peers have working local video). In such a case, I have noticed that the answer SDP contains a=recvonly while this should be a=sendrecv under normal circumstances.
I have further determined that by the time the other peer receives the offer and needs to reply with an answer, the localMedia of this side has sometimes not yet been added, because MediaDevices.getUserMedia can take a while to complete. I have also confirmed this order of operations by logging and observing that the offer sometimes arrives before local tracks were added.
I'm assuming that I shouldn't send an answer before the local media has been added?
I'm thinking of two ways to fix this, but I am not sure which option is best, if any:
Create the RTCPeerConnection only after MediaDevices.getUserMedia() completes. In the meantime, when receiving an offer, there is no peer connection yet, so we save offers in a buffer to process them later once the RTCPeerConnection is created.
When receiving an offer, and there are no localMedia tracks yet, hold off on creating the answer until the localMedia tracks have been added.
I am having difficulties to decide which solution (or another) matches best with the "Perfect Negotiation" pattern.
Thanks in advance!
Yes, it is good to add the stream before creating an offer if you do it 'statically', but the best way to do it is to do it in the onnegotiationneeded event because the addtrack event triggers an onnegotiationneeded event. So you should add the stream and then use the createoffer inside onnegotiationneeded. As far as the answer you can do it before with no issues, but remember that a well-established connection will let you add/remove tracks with no problems (even after the SDP has been set). You didn't post any code but remember that you also MUST exchange ice candidates.
The last piece of advice, remember that all of the above IS asynchronous! so you should use promises, and await until the description is set, only THEN create an offer/answer.
Hopefully, this will help
The WebRTC RTCPeerConnection interface has a createDataChannel method and an ondatachannel event handler. How do these interact? How do I create a single data channel that can be used to send/receive data between two peers?
Also, the RTCDataChannelInit constructor has a negotiated field, which by default is set to false and says it results in the channel being announced in-band. What happens if it's set to true?
Firstly: to create any data channel, the peers need to exchange an SDP offer/answer that negotiates the properties of the SCTP connection used by all data channels. This doesn't happen by default; you must call createDataChannel before calling createOffer for the offer to contain this SCTP information (an "m=application" section in the SDP).
If you don't do this, the data channel state will be stuck forever at connecting.
With that out of the way, there are two ways to negotiate a data channel between two peers:
In-band negotiation
This is what occurs by default, if the negotiated field is not set to true. One peer calls createDataChannel, and the other connects to the ondatachannel EventHandler. How this works:
Peer A calls createDataChannel.
Normal offer/answer exchange occurs.
Once the SCTP connection is up, a message is sent in-band from Peer A to Peer B to tell it about the data channel's existence.
On Peer B, the ondatachannel EventHandler is invoked with a new data channel, created from the in-band message. It has the same properties as the data channel created by Peer A, and now these data channels can be used to send data bidirectionally.
The advantage of this approach is that data channels can be created dynamically at any time, without the application needing to do additional signaling.
Out-of-band negotiation
Data channels can also be negotiated out-of-band. With this approach, instead of calling createDataChannel on one side and listening for ondatachannel on the other side, the application just calls createDataChannel on both sides.
Peer A calls createDataChannel({negotiated: true, id: 0})
Peer B also calls createDataChannel({negotiated: true, id: 0}).
Normal offer/answer exchange occurs.
Once the SCTP connection is up, the channels will instantly be usable (readyState will change to open). They're matched up by the ID, which is the underlying SCTP stream ID.
The advantage of this approach is that, since no message needs to be sent in-band to create the data channel on Peer B, the channel is usable sooner. This also makes the application code simpler, since you don't even need to bother with ondatachannel.
So, for applications that only use a fixed number of data channels, this approach is recommended.
Note that the ID you choose is not just an arbitrary value. It represents an underlying 0-based SCTP stream ID. And these IDs can only go as high as the number of SCTP streams negotiated by the WebRTC implementations. So, if you use an ID that's too high, your data channel won't work.
What about native applications?
If you're using the native webrtc library instead of the JS API, it works the same way; things just have different names.
C++:
PeerConnectionObserver::OnDataChannel
DataChannelInit::negotiated
DataChannelInit::id
Java:
PeerConnection.Observer.onDataChannel
DataChannel.Init.negotiated
DataChannel.Init.id
Obj-C:
RTCPeerConnectionDelegate::didOpenDataChannel
RTCDataChannelConfiguration::isNegotiated
RTCDataChannelConfiguration::channelId
Here's a very in-depth article about the particulars of peer-to-peer...
https://blog.sessionstack.com/how-javascript-works-webrtc-and-the-mechanics-of-peer-to-peer-connectivity-87cc56c1d0ab
Primary sources...
https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection
https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API
https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel
The mother load of peer-to-peer projects...
https://github.com/kgryte/awesome-peer-to-peer
As far as I can tell there are 7 events dispatched by a NoFlo port:
attach,
connect,
begingroup,
data,
endgroup,
disconnect,
detach
To me some of these events sound very similar such as attach + connect, and disconnect + detach. What is the difference?
What does begingroup and endgroup mean?
When do these events get emitted and when are they generally used?
I've seen the documentation at: http://noflojs.org/documentation/components/#portevents
Would my assumption be correct to assume that attach and detach are for handling NoFlo UI cases eg changing the state of the components look?
Another assumption would be that connect gets fired every time before data is sent? Then data gets fired. Then disconnect? Seems a bit odd to me...
I'm completely in the dark when it comes to groups.
attach and detach happen when the NoFlo Network attaches (or removes) a socket to the port. So usually they happen at network start-up time, before IIPs get sent.
The exception to this is when you're live-editing the graph with a tool like Flowhub. In that situation attach/detach can happen whenever you connect or remove wires.
Most components don't need to care about the attachment events.
connect happens before the upstream connection sends data, and disconnect when the upstream connection says that it has sent everything it is intending to send. So in effect they're beginning of transmission and end of transmission events. An upstream component may choose to connect again after a disconnect if it has a new batch of data to send.
data is the event for actual payload-containing packets.
begingroup and endgroup are the "bracket IPs" containing metadata about the data being sent. They can be used for creating tree structures with packet data.
For example, filesystem/ReadFile will send the file contents as a data packet, but the filename is sent via a bracket IP using a begingroup/endgroup packets around the actual file contents.
The noflo-groups library provides lots of components for utilizing group information for synchronization, routing, etc.
I am trying to create an application which requires a user to send his local video stream to multiple peers using webRTC. As far as I've seen I am responsible for managing several PeerConnection objects because a PeerConnection can only connect to a single peer at a time. What I want to know is if it is possible to create a connection and send my local stream to a peer without him sendig his local stream to me using webRTC.
Simply don't call peer.addStream for broadcast-viewers to make it oneway streaming!
You can disable audio/video media lines in the session description by setting OfferToReceiveAudio and OfferToReceiveVideo to false.
3-Way handshake isn't drafted by RTCWebb IETF WG, yet.
Because browser needs to take care of a lot stuff simultaneously like multi-tracks and multi-media lines; where each m-line should point out a unique peer.
How do I send my local video stream to multiple remote peers? Do I need to instantiate one PeerConnection per remote peer? Or can the same PeerConnection be used for all remote peers at the same time?
According to user dom on #webrtc on irc.w3.org, each PeerConnection is associated with a single remote peer. The developer is responsible for sharing the same stream instance with multiple PeerConnections:
<Cow_woC> Can a single PeerConnection connect to multiple remote peers, or only a single one at a time? If I want to stream the same video to multiple remote peers, what am I supposed to do?
<dom> Cow_woC, you need to manage several PeerConnection objects
<dom> and plug your video stream to each of them
<Cow_woC> dom: How do I share the camera feed with multiple PeerConnections? Is getUserMedia() allowed to return the same resource (and share it) multiple times?
<Cow_woC> dom: Or am I responsible for keeping the reference around and passing it to multiple PeerConnections?
<dom> the latter