I am seeing these iceCandidates:
[{"candidate":"","sdpMid":"audio","sdpMLineIndex":0,"usernameFragment":"f18ab8e6"}]
[{"candidate":"","sdpMid":"0","sdpMLineIndex":0,"usernameFragment":"b06cfb12"}]
Namely, candidate field is empty string. It's not that empty cadidate which means, no more candidates are comming, this is normal iceCandidate object except it doesn't have the candidate field. How is that possible?
This is valid and emitted by Firefox, Chrome does not implement it yet (but should be silently ignoring it when fed into addIceCandidate. It means the ICE engine has finished gathering all candidates it needs for this sdpMid/sdpMLineIndex.
Related
My question is about webrtc negotiation.
There is a contradiction in many online tutorials and what is described in MDN.
In MDN, it says link
At the end of each generation of candidates, an end-of-candidates
notification is sent in the form of an RTCIceCandidate whose candidate
property is an empty string. This candidate should still be added to
the connection using addIceCandidate() method, as usual, in order to
deliver that notification to the remote peer.
When there are no more candidates at all to be expected during the
current negotiation exchange, an end-of-candidates notification is
sent by delivering a RTCIceCandidate whose candidate property is null.
This message does not need to be sent to the remote peer. It's a
legacy notification of a state which can be detected instead by
watching for the iceGatheringState to change to complete, by watching
for the icegatheringstatechange event.
However, in the tutorial here, they introduce the following code
function handleICECandidateEvent(event) {
if (event.candidate) {
sendToServer({
type: "new-ice-candidate",
target: targetUsername,
candidate: event.candidate
});
}
}
If candidate is an empty string, it will be evaluated falsy and not be sent via sendToServer.
More interestingly, even in a same article here
They have following sample code
rtcPeerConnection.onicecandidate = (event) => {
if (event.candidate) {
sendCandidateToRemotePeer(event.candidate)
}
}
But right below this snippet, they say
When an ICE negotiation session runs out of candidates to propose for
a given RTCIceTransport, it has completed gathering for a generation
of candidates. That this has occurred is indicated by an icecandidate
event whose candidate string is empty ("").
You should deliver this to the remote peer just like any standard candidate, as described under Sharing a new candidate above. This
ensures that the remote peer is given the end-of-candidates
notification as well.
Actually, I read many online tutorials but I have never seen anywhere where they handle the empty string candidate.
The old spec did not require sending an empty candidate, but the new spec require send and addIceCandidate() an empty candidate.
Since Chrome is still an old specification, an empty candidate will cause an error when addedIceCandidate(), so I will not send it.
Is there any standard that defines the behavior when malformed UDP packets(like UDP packets with empty payload) is received in RTPStream?
I don't think there is. Malformed packets are largely left ignored, as the assumption is that all is well with the world.
In most cases, there's one of three different behaviors you should be expecting - depending on how things got implemented:
Browser crash. In such a case, just file a bug with the browser vendor
Ignore. The browser will ignore the packet altogether and move on
Disconnect. The browser may decide to disconnect the peer connection due to that (highly unlikely, but not unheard of)
Just don't send malformed packets on a media connection and expect consistent behavior.
I'm trying to understand a concept of WebRTC. As I found in some descriptions (for example here http://www.innoarchitech.com/content/images/2015/02/webrtc-complete-diagram.png), there is such a way of making a connection:
Call STUN, to get your IP:port address.
Get some channel from TURN - with that channel you can send info to other peer.
Send to other peer ICE candidates.
Accept ICE candidates with other peers- start a call.
The question is, what do we need ICE candidates for? We know our IP, we can send it to TURN therefore to other peer, and on TURN we have a nice connection with other peer- so we don't have to scary about NATs. Why except that we are sending ICE candidates (why many?), and why we need to use them?
We have 3 main concepts here:
ICE
TURN
STUN
The ICE negotiation is not that simple...
To execute ICE, UAs have to identify all address candidates, transport addresses. Transport addresses are a combination of IP address and port for a particular transport protocol. There are three types of candidates:
Host Candidate – transport address associated with a UA’s local interface
Relayed Candidate – transport address associated with a TURN server (can only be obtained from a TURN server)
Server Reflexive Candidate – translated address on the public side of the NAT (obtained from either a STUN server or a TURN server)
After UA1 has gathered all of its candidates, it arranges them in order of priority from highest to lowest and sends them to UA2 in attributes in an SDP offer message. UA2 performs the same candidate gathering and sends a SDP response with it’s list of candidates. Each UA takes the two lists of candidates and pairs them up to make candidate pairs. Each UA gathers these into check lists and schedules connectivity checks, STUN request/response transaction, to see which pairs work. Figure 3 shows the components of the candidate pairs that make up the UA check list.
ICE assigns one of the agents as the “Controlling Agent” and the other as the “Controlled Agent”. The controlling agent used the valid candidate pairs to nominate a pair to use for the media. There are two nomination methods that can be used:
Regular Nomination The checks continue until there is at least one valid candidate pair. The controlling agent picks from the valid pairs and sends a second STUN request on that pair with a flag to tell the peer that this is the one that is nominated for use.
Aggressive Nomination The nomination flag is sent with every STUN request, once the first check succeeds ICE processing for that media stream is finished and a second STUN request is not needed.
Each candidate pair in the check list has a state associated with it. The state is assigned by the UA once the check list has been computed. There are five possible states:
Frozen This pair can only be checked after being put in the waiting state. To enter the waiting state some other check must succeed first.
Waiting As soon as this is the highest priority pair in the check list a check will be performed.
In-Progress A check has been sent for this pair and the transaction is in progress
Succeeded Successful result from pair check.
Failed Failed result from pair check.
The link below includes more information and diagrams of the ICE flow.
Reference:
http://www.vocal.com/networking/ice-interactive-connectivity-establishment/
RFC https://www.rfc-editor.org/rfc/rfc5245#page-9
TURN is typically used only as a fallback when a direct peer-to-peer connection cannot be established. The latter is the hard part, and is what ICE is for.
Always using TURN, is an option, but a bit of an edge-case.
In WebRTC, how are the ICE candidates selected and used? If a new candidate is added, it will be used and if yes, when?
Newly gathered candidates needs to be advertised to other end using signalling then it will be paired with the remote candidates for both ends and restart the ICE processing for the new pairs. If any of the pairs gets succeeded and the priority value set by ICE is higher than the existing selected candidates then offerer end will select this new pair as favoured and selected candidate and start using this new pair of selected candidates. Some time it may require to send updated offer using the selected candidates through signalling.
Sending updated offer/answer depends on the design or implementation of the client or ICE implementation. Basically for anything changes related with the ICE attributes from initial offer/answer requires updated offer. As per RFC when ICE candidates are selected then offerer end shall include "a=remote-candidates" with the selected candidates and exchange the updated offer/answer. If client doesn't requires explicitly the updated offer as the indication of remote-candidates then its fine not to send any updated offer, you can try by not sending this and see what happens. It's should be harmless if you send updated offer all the time as well as long as the other client can parse this and recognises the attributes.
In FF or Chrome browser default implementation of WebRTC (Trickle ICE) you don't need to get worried about updated candidates, as client end will receive events when an ICE candidate is available to retrieve so you will just retrieve that and send to other end. For re-Invite case, call createOffer once the ICE state goes to "completed" state on the Controlling (Offerer) side.
I newly wrote a simple chat application, but I didn't really understand the background of ICE Candidates.
When the peer create a connection they get ICE Candidates and they exchange them and set
them finally to the peerconnection.
So my question is, where do the ICE Candidates come from and how are they used and are they all really used ?
I have noticed that my colleague got less candidates when he executes the application on his machine, what could be the reason for different amount of Candidates ?
the answer from #Ichigo is correct, but it is a litte bit bigger. Every ICE contains 'a node' of your network, until it has reached the outside. By this you send these ICE's to the other peer, so they know through what connection points they can reach you.
See it as a large building: one is in the building, and needs to tell the other (who is not familiar) how to walk through it. Same here, if I have a lot of network devices, the incoming connection somehow needs to find the right way to my computer.
By providing all nodes, the RTC connection finds the shortest route itself. So when you would connect to the computer next to you, which is connected to the same router/switch/whatever, it uses all ICE's and determine the shortest, and that is directly through that point. That your collegue got less ICE candidates has to do with the ammount of devices it has to go through.
Please note that every network adapter inside your computer which has an IP adress (I have a vEthernet switch from hyper-v) it also creates an ICE for it.
ICE stands for Interactive Connectivity Establishment , its a techniques used in NAT( network address translator ) for establishing communication for VOIP, peer-peer, instant-messaging, and other kind of interactive media.
Typically ice candidate provides the information about the ipaddress and port from where the data is going to be exchanged.
It's format is something like follows
a=candidate:1 1 UDP 2130706431 192.168.1.102 1816 typ host
here UDP specifies the protocol to be used, the typ host specifies which type of ice candidates it is, host means the candidates is generated within the firewall.
If you use wireshark to monitor the traffic then you can see the ports that are used for data transfer are same as the one present in ice-candidates.
Another type is relay , which denotes this candidates can be used when communication is to be done outside the firewall.
It may contain more information depending on browser you are using.
Many time i have seen 8-12 ice-candidates are generated by browser.
Ichigo has a good answer, but doesn't emphasise how each candidate is used. I think MarijnS95's answer is plain wrong:
Every ICE contains 'a node' of your network, until it has reached the outside
By providing all nodes, the RTC connection finds the shortest route itself.
First, he means ICE candidate, but that part is fine. Maybe I'm misinterpreting him, but by saying 'until it has reached the outside', he makes it seem like a client (the initiating peer) is the inner most layer of an onion, and suggests the ICE candidate helps you peel the layers until you get to the 'internet', where can get to the responding peer, perhaps peeling another onion to get to it. This is just not true. If an initiating peer fails to reach a responding peer through the transport address, it discards this candidate and will try a different candidate. It does not store any nodes anywhere in the candidate. The ICE candidates are generated before any communication with the responding peer. An ice candidate does not help you peel the proverbial NAT onion. Also regarding the second quote I made from his answer, he makes it seem like ICE is used in a shortest path algorithm, where 'shortest' does not show up in the ICE RFC at all.
From RFC8445 terminology list:
ICE allows the agents to discover enough information
about their topologies to potentially find one or more paths by which
they can establish a data session.
The purpose of ICE is to discover which pairs of addresses will work. The way that ICE does this is to systematically try all possible pairs (in a carefully sorted order) until it finds one or more that work.
Candidate, Candidate Information: A transport address that is a
potential point of contact for receipt of data. Candidates also
have properties -- their type (server reflexive, relayed, or
host), priority, foundation, and base.
Transport Address: The combination of an IP address and the
transport protocol (such as UDP or TCP) port.
So there you have it, (ICE) Candidate was defined (an IP address and port that could potentially be an address that receives data, which might not work), and the selection process was explained (the first transport address pair that works). Note, it is not a list of nodes or onion peels.
Different users may have different ice candidates because of the process of "gathering candidates". There are different types of candidates, and some are obtained from the local interface. If you have an extra virtual interface on your device, then an extra ICE will be generated (I did not test this!). If you want to know how ICE candidates are 'gathered', read the 2.1. Gathering Candidates