After a WebRTC session has been established, I would like to stop sending the stream to the peer at some point and then resume sending later.
If I call removeStream, it does indeed stop sending data. If I then call addStream (with the previously removed stream), it resumes. Great!
However, is this safe? Or do I need to re-negotiate/exchange the SDP after the removal and/or after the re-add? I've seen it mentioned in several places that the SDP needs to be re-negotiated after such changes, but I'm wondering if it is ok in this simple case where the same stream is being removed and re-added?
PS: Just in case anyone wants to suggest it: I don't want to change the enabled state of the track since I still need the local stream playing, even while it is not being sent to the peer.
It will only work in Chrome, and is non-spec, so it's neither web compatible nor future-proof.
The spec has pivoted from streams to tracks, sporting addTrack and removeTrack instead of addStream and removeStream. As a result the latter isn't even implemented in Firefox.
Unfortunately, because Chrome hasn't caught up, this means renegotiation currently works differently in different browsers. It is possible to do however with some effort.
The new model has a cleaner separation between streams and what's sent in RTCPeerConnection.
Instead of renegotiating, setting track.enabled = false is a good idea. You can still play a local view of your video by cloning the track.
Related
Browser: Chrome
I am trying to debug a webRTC application which works fine on three out of four fronts! I cannot get video from the receiver to the caller. I can get video and audio from the caller to the receiver and audio from the receiver to the caller. The problem is that the receiver does not fire a video (sdpMid="video") ICE candidate. While desperately trying to solve this problem, I tried to use pc.CreateAnswer before setting pc.remoteDescription and it gave the error quoted in the title.
My question is to understand the reason behind this. An answer SDP would just be the SDP based upon the getUserMedia settings/constraints. So, why do we have to wait for setting remoteDescription. I thought that a createAnswer would start firing the gathering of ICE candidates and this can be done earlier without waiting to set remoteDescription. That is not the case. Why?
Offers and answers aren't independent, they're part of an inherently asymmetric exchange.
An answer is a direct response to a specific offer (hence the name "answer"). Therefore the peer cannot answer before it has an offer, which you set with setRemoteDescription.
An offer contains specific limitations, or envelope (like m-lines), that an answer has to abide by/answer to/stay within. Another way to say it is that the answer is an iteration of the offer.
For instance, an offer created with offer options offerToReceiveVideo: false can only be answered with recvonly for video (meaning receive video from offerer to answerer only), never sendrecv.
I need to develop a custom WebRTC peer (I need to establish audio or/and data connection between web-browser and non-browser). I however, struggle to find a proper, clear description of the handshake phase.
Answers to questions such as How to create data channel in WebRTC peer connection? are not entirely helpful, as they are not too detailed. Specifically, they say nothing about SDP contents.
Can anyone explain this or recommend any good documentation?
Here is a page with some graphs showing how the signaling process works. Basically, you set some client side stuff first:
PeerConnectionFactory; to generate PeerConnections,
PeerConnection; one for every connection to another peer you want (usually 1),
MediaStream; to hook up the audio and video from your client device.
Then you generate an SDP offer
peerConnection.createOffer();
on the caller side and send it to the callee. The callee sets this offer
peerConnection.setRemoteDescription(insert-the-offer-here);
and generates an SDP answer
peerConnection.createAnswer();
and sends it back to the caller. The caller receives this answer and sets it.
peerConnection.setRemoteDescription(insert-the-answer-here);
Both the caller and callee get a call to
onAddStream() {...} //needs to be implemented in your code
The callee when the caller's offer is set and the caller when the callee's answer is set. This callback signals the beginning of the connection.
You can also use ICE (STUN/TURN) to avoid firewall and NAT issues, but this is optional. Although in production code, you probably want to implement it anyway.
Note: Webrtc documentation is scarce and subject to change, take everything you read about webrtc (at least anything written as of now) with a grain of salt...
I am thinking of changing my application from using a single PeerConnection for transferring media both ways to one PeerConnection for upstream and one for downstream for a single call between two peer.
The advantages I foresee:
Less worry about signalling state of PeerConnection when changing offering media from video+audio to audio and vice-versa
Might be easier to plug an Media Servers like kurento into the application ( in case of multi user call, lesser upload bandwidth required by user).
(not sure of this one) single responsibility principle, each PeerConnection has single role.
the major reason I want to do this change is, I am noticing that if peer(peer1) offers only audio but other peer(peer2) answers with both video+audio, peer1 recieves only the audio for some reason, but if peer1 had been an answerer, it is able to recieve both MediaTracks without any problem. Not sure if it is a bug in my app or browser( got same result in firefox and chrome). I was able to make a workaround by maintaining states, changing offerer based on state and stuff, but having problems with both peers changing state (nearly) simultaneously. Thought above proposal would be simpler solution and I can get rid of maintaining states.
Other than the obvious disadvantages of extra overhead of more ICE candidate requests( n STUN n TURN), maintaining extra PeerConnections, any other issue possible following this design?
Nothing prevents you from doing that, but I suspect there's a simpler solution to your problem which you kind of buried:
the major reason I want to do this change is, I am noticing that if peer(peer1) offers only audio but other peer(peer2) answers with both video+audio, peer1 recieves only the audio for some reason,
Don't ask me why, but the default spec behavior when peer1 only offers audio, is to only request audio from the other side. To override this and leave yourself open to receiving video as well if the other side has it, use RTCOfferOptions:
peer1.createOffer({ offerToReceiveVideo: true }).then( ... )
(or if you're using the legacy non-promise API it's the third argument.)
The nice thing with this is that it is intent-based so you don't need to track any state. e.g. always using { offerToReceiveVideo: true, offerToReceiveAudio: true } may be right for you.
A resource issue would be that you are be utilizing more ports as both sides of the connection have to complete the DTLS handshake(which is done peer-to-peer and not through the signalling server).
A design challenge is keeping track of two connections orthogonally. It could be hairy and will more readily show errors in the underlying webrtc implementation if the state is not handled properly(browser state errors, etc.).
I'm using the soundcloud API and so far it was working fine until I hit this track:
https://soundcloud.com/katyperryofficial/roar
I don't know what's wrong with this track but it really wouldn't play. I can get all info of it just not the stream part. I checked chrome network tab and it gives me this. It just cancels without any error:
Name Method Status Type Initiator Size Time
stream?consumer_key=### GET (canceled) Other 13B 1.02s
Any ideas? Have I missed something?
Soundcloud devs made some changes in their code, and i don't know why, they are switching back to rtmp protocol.
Even the response said that track is streamable it can't be streamed with a regular stream_url.
After some digging in dev tools, i've noticed that some tracks use rtmp protocol instead of http/https
Anyway, you can find the streams of the track on:
http://api.soundcloud.com/tracks/TrackID/streams?consumer_key=XXX
from here, you're on your own. from my research, only flash (why?) can play rtmp streams.
I'm working with GameKit.framework and I'm trying to create a reliable communication between two iPhones.
I'm sending packages with the GKMatchSendDataReliable mode.
The documentation says:
GKMatchSendDataReliable
The data is sent continuously until it is successfully received by the intended recipients or the connection times out.
Reliable transmissions are delivered in the order they were sent. Use this when you need to guarantee delivery.
Available in iOS 4.1 and later. Declared in GKMatch.h.
I have experienced some problems on a bad WiFi connection. The GameKit does not declare the connection lost, but some packages never arrive.
Can I count on a 100% reliable communication when using GKMatchSendDataReliable or is Apple just using fancy names for something they didn't implement?
My users also complain that some data may be accidentally lost during the game. I wrote a test app and figured out that GKMatchSendDataReliable is not really reliable. On weak internet connection (e.g. EDGE) some packets are regularly lost without any error from the Game Center API.
So the only option is to add an extra transport layer for truly reliable delivery.
I wrote a simple lib for this purpose: RoUTP. It saves all sent messages until acknowledgement for each received, resends lost and buffers received messages in case of broken sequence.
In my tests combination "RoUTP + GKMatchSendDataUnreliable" works even beter than "RoUTP + GKMatchSendDataReliable" (and of course better than pure GKMatchSendDataReliable which is not really reliable).
It nearly 100% reliable but maybe not what you need sometimes… For example you dropped out of network all the stuff that you send via GKMatchSendDataReliable will be sent in the order you've send them.
This is brilliant for turn-based games for example, but if fast reaction is necessary a dropout of the network would not just forget the missed packages he would get all the now late packages till he gets to realtime again.
The case GKMatchSendDataReliable doesn't send the data is a connection time out.
I think this would be also the case when you close the app