I am using Simple-peer to build a webrtc application. To establish connection, we need to first send the offer and receive the answer. After that onicecandidate event gets triggered generating the candidate, we are required to send the candidate data to remote peer. The remote peer will than run addicecandidate and send back the remote candidate data which need to be added on localpeer using addicecandidate and connection gets established.
I want to understand how simple-peer is handling transfer of candidate data. The SDP data related to OFFER and ANSWER is required to be transferred using server in between, in one of the example socket-io has been used. But how the candidate data is getting transferred?
In simple-peer the signal from peer.on('signal', data=>{}) contains all the webrtc signaling data. If you print out the value of the signal you'll see it contains sdp, offer and answer all labeled to identify which is which.
Related
I'm building a video calling app using WebRTC which allows one peer to call another by selecting someone in the lobby. When peer A sends a call request, the other peer B can accept. At this point, WebRTC signaling starts:
Both peers get their local media using MediaDevices.getUserMedia()
Both peers create an RTCPeerConnection and attach event listeners
Both peers calls RTCPeerConnection.addTrack() to add their local media
One peer A (the impolite user) creates an offer, calls RTCPeerConnection.setLocalDescription() to set that offer as the local description, and sends it to the WebSocket server, which forwards it to the other peer B.
The other peer B receives this offer and adds calls RTCPeerConnection.setRemoteDescription() to record it as the remote description
The other peer B then creates an answer and transmits it again to the first peer A.
(Steps based on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity)
This flow is almost working well. In 1 out of 10 calls, I receive no video/audio from one of the peers (while both peers have working local video). In such a case, I have noticed that the answer SDP contains a=recvonly while this should be a=sendrecv under normal circumstances.
I have further determined that by the time the other peer receives the offer and needs to reply with an answer, the localMedia of this side has sometimes not yet been added, because MediaDevices.getUserMedia can take a while to complete. I have also confirmed this order of operations by logging and observing that the offer sometimes arrives before local tracks were added.
I'm assuming that I shouldn't send an answer before the local media has been added?
I'm thinking of two ways to fix this, but I am not sure which option is best, if any:
Create the RTCPeerConnection only after MediaDevices.getUserMedia() completes. In the meantime, when receiving an offer, there is no peer connection yet, so we save offers in a buffer to process them later once the RTCPeerConnection is created.
When receiving an offer, and there are no localMedia tracks yet, hold off on creating the answer until the localMedia tracks have been added.
I am having difficulties to decide which solution (or another) matches best with the "Perfect Negotiation" pattern.
Thanks in advance!
Yes, it is good to add the stream before creating an offer if you do it 'statically', but the best way to do it is to do it in the onnegotiationneeded event because the addtrack event triggers an onnegotiationneeded event. So you should add the stream and then use the createoffer inside onnegotiationneeded. As far as the answer you can do it before with no issues, but remember that a well-established connection will let you add/remove tracks with no problems (even after the SDP has been set). You didn't post any code but remember that you also MUST exchange ice candidates.
The last piece of advice, remember that all of the above IS asynchronous! so you should use promises, and await until the description is set, only THEN create an offer/answer.
Hopefully, this will help
I am working on setting up group calls involving up to 8 peers using WebRTC.
Let's say a peer needs to set up 7 RTCPeerConnections to join a group call. Instead of relying on onicecandidate event for every single RTCPeerConnection, I was wondering if I can track the client's icecandidates in a central location and reuse it for each new RTCPeerConnection. (e.g. Signaling Server will keep track of a peer's full ICE candidates, and share them with other peers as soon as they need them).
I am unsure what the average number of 'icecandidates' each client will have, but with ice trickle process, it seems that many duplicate http or websocket calls will need to be made to a Signaling Server in oder to exchange ice candidates between any 2 peers.
So I was wondering if I could just "accumulate" ice candidates locally and reuse them when new RTCPeerConnection will need to be made with new peer.
You can not. ICE candidates are associated with the peerconnection and its ice username fragment and password.
There is a feature called ice forking that would allow what you ask for but it is not implemented yet. https://bugs.chromium.org/p/webrtc/issues/detail?id=11252#c3 has some details.
The WebRTC RTCPeerConnection interface has a createDataChannel method and an ondatachannel event handler. How do these interact? How do I create a single data channel that can be used to send/receive data between two peers?
Also, the RTCDataChannelInit constructor has a negotiated field, which by default is set to false and says it results in the channel being announced in-band. What happens if it's set to true?
Firstly: to create any data channel, the peers need to exchange an SDP offer/answer that negotiates the properties of the SCTP connection used by all data channels. This doesn't happen by default; you must call createDataChannel before calling createOffer for the offer to contain this SCTP information (an "m=application" section in the SDP).
If you don't do this, the data channel state will be stuck forever at connecting.
With that out of the way, there are two ways to negotiate a data channel between two peers:
In-band negotiation
This is what occurs by default, if the negotiated field is not set to true. One peer calls createDataChannel, and the other connects to the ondatachannel EventHandler. How this works:
Peer A calls createDataChannel.
Normal offer/answer exchange occurs.
Once the SCTP connection is up, a message is sent in-band from Peer A to Peer B to tell it about the data channel's existence.
On Peer B, the ondatachannel EventHandler is invoked with a new data channel, created from the in-band message. It has the same properties as the data channel created by Peer A, and now these data channels can be used to send data bidirectionally.
The advantage of this approach is that data channels can be created dynamically at any time, without the application needing to do additional signaling.
Out-of-band negotiation
Data channels can also be negotiated out-of-band. With this approach, instead of calling createDataChannel on one side and listening for ondatachannel on the other side, the application just calls createDataChannel on both sides.
Peer A calls createDataChannel({negotiated: true, id: 0})
Peer B also calls createDataChannel({negotiated: true, id: 0}).
Normal offer/answer exchange occurs.
Once the SCTP connection is up, the channels will instantly be usable (readyState will change to open). They're matched up by the ID, which is the underlying SCTP stream ID.
The advantage of this approach is that, since no message needs to be sent in-band to create the data channel on Peer B, the channel is usable sooner. This also makes the application code simpler, since you don't even need to bother with ondatachannel.
So, for applications that only use a fixed number of data channels, this approach is recommended.
Note that the ID you choose is not just an arbitrary value. It represents an underlying 0-based SCTP stream ID. And these IDs can only go as high as the number of SCTP streams negotiated by the WebRTC implementations. So, if you use an ID that's too high, your data channel won't work.
What about native applications?
If you're using the native webrtc library instead of the JS API, it works the same way; things just have different names.
C++:
PeerConnectionObserver::OnDataChannel
DataChannelInit::negotiated
DataChannelInit::id
Java:
PeerConnection.Observer.onDataChannel
DataChannel.Init.negotiated
DataChannel.Init.id
Obj-C:
RTCPeerConnectionDelegate::didOpenDataChannel
RTCDataChannelConfiguration::isNegotiated
RTCDataChannelConfiguration::channelId
Here's a very in-depth article about the particulars of peer-to-peer...
https://blog.sessionstack.com/how-javascript-works-webrtc-and-the-mechanics-of-peer-to-peer-connectivity-87cc56c1d0ab
Primary sources...
https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection
https://developer.mozilla.org/en-US/docs/Web/API/Media_Streams_API
https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel
The mother load of peer-to-peer projects...
https://github.com/kgryte/awesome-peer-to-peer
In WebRTC, how are the ICE candidates selected and used? If a new candidate is added, it will be used and if yes, when?
Newly gathered candidates needs to be advertised to other end using signalling then it will be paired with the remote candidates for both ends and restart the ICE processing for the new pairs. If any of the pairs gets succeeded and the priority value set by ICE is higher than the existing selected candidates then offerer end will select this new pair as favoured and selected candidate and start using this new pair of selected candidates. Some time it may require to send updated offer using the selected candidates through signalling.
Sending updated offer/answer depends on the design or implementation of the client or ICE implementation. Basically for anything changes related with the ICE attributes from initial offer/answer requires updated offer. As per RFC when ICE candidates are selected then offerer end shall include "a=remote-candidates" with the selected candidates and exchange the updated offer/answer. If client doesn't requires explicitly the updated offer as the indication of remote-candidates then its fine not to send any updated offer, you can try by not sending this and see what happens. It's should be harmless if you send updated offer all the time as well as long as the other client can parse this and recognises the attributes.
In FF or Chrome browser default implementation of WebRTC (Trickle ICE) you don't need to get worried about updated candidates, as client end will receive events when an ICE candidate is available to retrieve so you will just retrieve that and send to other end. For re-Invite case, call createOffer once the ICE state goes to "completed" state on the Controlling (Offerer) side.
To establish WebRTC connections the ICE protocol is used with a signaling server which must send messages in both directions. I wonder why after the initiator sent its offer and candidates to the other participant, the participant needed to send back its answer and candidates using the signaling channel in the other direction. Cannot the participant open the connection to the initiator using candidates from both sides and send back its answer using the open connection?
I started reading ICE RFC and the only relevant part I found is in section 5.2 where the initiator must take the controlling role and nominates candidate pairs. But it does not explain why the other could not initiate connection.
To give some background, I am trying to build a webapp for which I want users to establish WebRTC connections without using a signaling server. I thought of having the app to generate a URL including the offer and candidates and providing this URL to other participants through other medium like instant messaging. The issue I got is that the participant need to send back its answer and candidates using the same medium, which is not practical. In the end I will go for a signalling server but I wonder the technical reason.
Yes, you can do that if caller is behind public IP or Full Cone NAT(in this case, router connection mapping needs not to be timed out).
You can able full fill above conditions rarely.
What's the problem with other NAT types?
For example , PRC(port restricted cone) NAT won't allow you to receive a packet from a IP:Port , if you didn't send any packet to that IP:Port before. So callee will never able to send you a packet.
So if callee sends her candidates list to you . you can send some dummy data(with low TTL) to her IP:Port to fool your PRC NAT (now it allow incoming packets from callee's IP:Port as it sends a packet to that IP:Port before).
To know more about different types of NAT:
https://en.wikipedia.org/wiki/Network_address_translation
http://think-like-a-computer.com/2011/09/16/types-of-nat/