In WebRTC, how are the ICE candidates selected and used? If a new candidate is added, it will be used and if yes, when?
Newly gathered candidates needs to be advertised to other end using signalling then it will be paired with the remote candidates for both ends and restart the ICE processing for the new pairs. If any of the pairs gets succeeded and the priority value set by ICE is higher than the existing selected candidates then offerer end will select this new pair as favoured and selected candidate and start using this new pair of selected candidates. Some time it may require to send updated offer using the selected candidates through signalling.
Sending updated offer/answer depends on the design or implementation of the client or ICE implementation. Basically for anything changes related with the ICE attributes from initial offer/answer requires updated offer. As per RFC when ICE candidates are selected then offerer end shall include "a=remote-candidates" with the selected candidates and exchange the updated offer/answer. If client doesn't requires explicitly the updated offer as the indication of remote-candidates then its fine not to send any updated offer, you can try by not sending this and see what happens. It's should be harmless if you send updated offer all the time as well as long as the other client can parse this and recognises the attributes.
In FF or Chrome browser default implementation of WebRTC (Trickle ICE) you don't need to get worried about updated candidates, as client end will receive events when an ICE candidate is available to retrieve so you will just retrieve that and send to other end. For re-Invite case, call createOffer once the ICE state goes to "completed" state on the Controlling (Offerer) side.
Related
I'm building a video calling app using WebRTC which allows one peer to call another by selecting someone in the lobby. When peer A sends a call request, the other peer B can accept. At this point, WebRTC signaling starts:
Both peers get their local media using MediaDevices.getUserMedia()
Both peers create an RTCPeerConnection and attach event listeners
Both peers calls RTCPeerConnection.addTrack() to add their local media
One peer A (the impolite user) creates an offer, calls RTCPeerConnection.setLocalDescription() to set that offer as the local description, and sends it to the WebSocket server, which forwards it to the other peer B.
The other peer B receives this offer and adds calls RTCPeerConnection.setRemoteDescription() to record it as the remote description
The other peer B then creates an answer and transmits it again to the first peer A.
(Steps based on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity)
This flow is almost working well. In 1 out of 10 calls, I receive no video/audio from one of the peers (while both peers have working local video). In such a case, I have noticed that the answer SDP contains a=recvonly while this should be a=sendrecv under normal circumstances.
I have further determined that by the time the other peer receives the offer and needs to reply with an answer, the localMedia of this side has sometimes not yet been added, because MediaDevices.getUserMedia can take a while to complete. I have also confirmed this order of operations by logging and observing that the offer sometimes arrives before local tracks were added.
I'm assuming that I shouldn't send an answer before the local media has been added?
I'm thinking of two ways to fix this, but I am not sure which option is best, if any:
Create the RTCPeerConnection only after MediaDevices.getUserMedia() completes. In the meantime, when receiving an offer, there is no peer connection yet, so we save offers in a buffer to process them later once the RTCPeerConnection is created.
When receiving an offer, and there are no localMedia tracks yet, hold off on creating the answer until the localMedia tracks have been added.
I am having difficulties to decide which solution (or another) matches best with the "Perfect Negotiation" pattern.
Thanks in advance!
Yes, it is good to add the stream before creating an offer if you do it 'statically', but the best way to do it is to do it in the onnegotiationneeded event because the addtrack event triggers an onnegotiationneeded event. So you should add the stream and then use the createoffer inside onnegotiationneeded. As far as the answer you can do it before with no issues, but remember that a well-established connection will let you add/remove tracks with no problems (even after the SDP has been set). You didn't post any code but remember that you also MUST exchange ice candidates.
The last piece of advice, remember that all of the above IS asynchronous! so you should use promises, and await until the description is set, only THEN create an offer/answer.
Hopefully, this will help
I am working on setting up group calls involving up to 8 peers using WebRTC.
Let's say a peer needs to set up 7 RTCPeerConnections to join a group call. Instead of relying on onicecandidate event for every single RTCPeerConnection, I was wondering if I can track the client's icecandidates in a central location and reuse it for each new RTCPeerConnection. (e.g. Signaling Server will keep track of a peer's full ICE candidates, and share them with other peers as soon as they need them).
I am unsure what the average number of 'icecandidates' each client will have, but with ice trickle process, it seems that many duplicate http or websocket calls will need to be made to a Signaling Server in oder to exchange ice candidates between any 2 peers.
So I was wondering if I could just "accumulate" ice candidates locally and reuse them when new RTCPeerConnection will need to be made with new peer.
You can not. ICE candidates are associated with the peerconnection and its ice username fragment and password.
There is a feature called ice forking that would allow what you ask for but it is not implemented yet. https://bugs.chromium.org/p/webrtc/issues/detail?id=11252#c3 has some details.
I am using Simple-peer to build a webrtc application. To establish connection, we need to first send the offer and receive the answer. After that onicecandidate event gets triggered generating the candidate, we are required to send the candidate data to remote peer. The remote peer will than run addicecandidate and send back the remote candidate data which need to be added on localpeer using addicecandidate and connection gets established.
I want to understand how simple-peer is handling transfer of candidate data. The SDP data related to OFFER and ANSWER is required to be transferred using server in between, in one of the example socket-io has been used. But how the candidate data is getting transferred?
In simple-peer the signal from peer.on('signal', data=>{}) contains all the webrtc signaling data. If you print out the value of the signal you'll see it contains sdp, offer and answer all labeled to identify which is which.
I'm trying to understand a concept of WebRTC. As I found in some descriptions (for example here http://www.innoarchitech.com/content/images/2015/02/webrtc-complete-diagram.png), there is such a way of making a connection:
Call STUN, to get your IP:port address.
Get some channel from TURN - with that channel you can send info to other peer.
Send to other peer ICE candidates.
Accept ICE candidates with other peers- start a call.
The question is, what do we need ICE candidates for? We know our IP, we can send it to TURN therefore to other peer, and on TURN we have a nice connection with other peer- so we don't have to scary about NATs. Why except that we are sending ICE candidates (why many?), and why we need to use them?
We have 3 main concepts here:
ICE
TURN
STUN
The ICE negotiation is not that simple...
To execute ICE, UAs have to identify all address candidates, transport addresses. Transport addresses are a combination of IP address and port for a particular transport protocol. There are three types of candidates:
Host Candidate – transport address associated with a UA’s local interface
Relayed Candidate – transport address associated with a TURN server (can only be obtained from a TURN server)
Server Reflexive Candidate – translated address on the public side of the NAT (obtained from either a STUN server or a TURN server)
After UA1 has gathered all of its candidates, it arranges them in order of priority from highest to lowest and sends them to UA2 in attributes in an SDP offer message. UA2 performs the same candidate gathering and sends a SDP response with it’s list of candidates. Each UA takes the two lists of candidates and pairs them up to make candidate pairs. Each UA gathers these into check lists and schedules connectivity checks, STUN request/response transaction, to see which pairs work. Figure 3 shows the components of the candidate pairs that make up the UA check list.
ICE assigns one of the agents as the “Controlling Agent” and the other as the “Controlled Agent”. The controlling agent used the valid candidate pairs to nominate a pair to use for the media. There are two nomination methods that can be used:
Regular Nomination The checks continue until there is at least one valid candidate pair. The controlling agent picks from the valid pairs and sends a second STUN request on that pair with a flag to tell the peer that this is the one that is nominated for use.
Aggressive Nomination The nomination flag is sent with every STUN request, once the first check succeeds ICE processing for that media stream is finished and a second STUN request is not needed.
Each candidate pair in the check list has a state associated with it. The state is assigned by the UA once the check list has been computed. There are five possible states:
Frozen This pair can only be checked after being put in the waiting state. To enter the waiting state some other check must succeed first.
Waiting As soon as this is the highest priority pair in the check list a check will be performed.
In-Progress A check has been sent for this pair and the transaction is in progress
Succeeded Successful result from pair check.
Failed Failed result from pair check.
The link below includes more information and diagrams of the ICE flow.
Reference:
http://www.vocal.com/networking/ice-interactive-connectivity-establishment/
RFC https://www.rfc-editor.org/rfc/rfc5245#page-9
TURN is typically used only as a fallback when a direct peer-to-peer connection cannot be established. The latter is the hard part, and is what ICE is for.
Always using TURN, is an option, but a bit of an edge-case.
Related WebRTC, ICE protocol gives the which pair of addresses will work for direct media transfer between the pairs.
Let A and B are two endpoints
To choose which address will work for direct communication between A and B, Person A first gather candidates, encode candidate attribute, encode the SDP offer message, and send it to another endpoint.
When B get offer message from A,then person B gather candidates, encode the SDP answer message with its own list of candidates and send it to person A.
At this end of this process, each agent has a complete list of local candidates and Remote candidates. Its pairs them up, resulting in CANDIDATE PAIRS. To see, which pair work, each agent performs the connectivity checks using STUN req/resp.
How many connectivity checks are performed, to nominate valid candidate pair?
What are the remaining ICE connectivity checks are performed regarding webRTC call?
To develop ICE module for webRTC call, I have to follow each step in RFC5245 or any thing else?
How many connectivity checks are performed, to nominate valid
candidate pair?
The number of candidate pairs are the number of connectivity checks done by each side.
What are the remaining ICE connectivity checks are performed regarding
webRTC call?
There are no extra ICE connectivity checks for webRTC.
To develop ICE module for webRTC call, I have to follow each step in
RFC5245 or any thing else?
You have to implement or use existing implementation of DTLS protocol, RFC5763 and RFC5764. DTLS implementation can be found on OpenSSL library.
All these seems a lot of work but if you use openssl then its easy enough.