I am planning to use Puppeteer for WebRTC call. I hope it should be easy. I am not sure how do I collect statistics like WebRTC call is passed or failed, how many media packets (UDP packets exchanged), stun / turn pass fail, media parameters like jitter, delay etc.
Can somebody please help me to understand, using Puppeteer how can one collect WebRTC related statistics.
There is a well known WebRTC test engine based on selenium and selenium grid called KITE. For references, and quick start you can check the simple KITE-AppRTC-Test implementation to see how they are collecting the stats, and show them. You might want to run the demos as well because it seems to have the results you are looking for.
Among many other approaches one might be -
Collect WebRTC connection metrics by calling getStats API. What you see in chrome://webrtc-internals is a visual representation of this getStats API that collects getStats snapshots in regular interval, and showing them after some post-processing.
Collect getStats data from puppeteer page.evaluate, send it to server and then analyse the data realtime or at the end of call based on your use case.
There are quite good amount of opensource work done by WebRTC experts on how you can collect WebRTC data, send them to server and represent them
https://github.com/fippo/webrtc-externals
https://github.com/fippo/webrtc-dump-importer
https://github.com/fippo/dump-webrtc-event-log
Related
I am new to WebRTC technology.
I want to create a video chat / video conferencing with a transmitter and many followers (more 1000).
Example:
I read a lot of documentations :
https://medium.com/linagora-engineering/scalability-in-video-conferencing-part-1-276f52b4acac
https://webrtcglossary.com/sfu/
But I still don't know what is the best solution (in my case) between Selective Forwarding Unit (SFU) and Multiploint Control Unit (MCU).
Can you help me to understand?
I think the best way is MCU but I am not sure.
Second question:
Can you suggest some sources and links that can help me to set up such an architecture. Currently my project works perfectly in Peer To Peer (Mesh) but it is not the right solution. I have absolutely no idea how to set this up.
Thank you so much
It is possible to implement this using an SFU. The more peers are connected, the more you would need processing power to handle those new peers. This could be done by using more threads and/or forwarding requests to another machine.
With mediasoup it is possible have control over this. With this tool you have routers where peers can connect to to get the stream. A router works on a worker which has a limited amount of receiving peers (depending on cpu capacity). Now to allow more peers you can forward the stream to other routers which can expand the total capacity.
useful links:
https://mediasoup.org/documentation/v3/scalability/#one-to-many-broadcasting
https://mediasoup.org/documentation/v3/mediasoup/design/#architecture
https://mediasoup.discourse.group/t/scalability-in-mediasoup-example/793/2?u=dirvann
We’ve been using the Tokbox platform for several months now with a Javascript web-client as well as an Android phone client, where sessions and connections are managed by a Python server. While integration and bring-up went well on both ends (client and server), we continue to encounter problems with the in-session audio and video experience.
Sessions are always routed and always between two participants only, with much use of a collaborative editor.
The in-session experience is like a coin toss: we never know how it’s going to go, and that’s becoming a business threat.
Web-Client: A/V Resources
The most common problem is the acquisition of audio and/or video: at the beginning of a session, one or the other participants may have problems hearing or seeing the other. Allocating a new connection to establish new streams does not fix that, nor does restarting the browser.
Question: What’s the recommended way to detect possible resource locks (e.g. does another application hog the camera/microphone)?
Web-Client: Network
Bandwidth and packet loss are a challenge, for example this inspector graph:
Audio and video of both participants is all over the place, and while we can not control the network connections the web-client should be able to reliably give useful information.
Question: Other than continuous connection monitoring with getStats() and maybe the experimental navigator.connection property, how can the web-client monitor network connectivity?
Pre-Call Test
We recommend to customers to run a pre-call test and have implemented it on our site as well. However, results of that test often times do not reflect the in-session connectivity. Worse, a pre-call test may detect a low (no video) bandwidth while Skype works just fine.
Question: How can that be?
I'm a member of the TokBox development team. I remember you reported an issue with the Python SDK, thanks for that!
Web-Client: A/V Resources
Most acquisition issues are detected by the JS SDK and if they aren't then we'd really like to hear about it! Please report reproduction steps or affected session IDs to TokBox support (referencing this StackOverflow question): https://support.tokbox.com/hc/en-us/requests/new
Most acquisition errors appear as OT_HARDWARE_UNAVAILABLE or OT_MEDIA_ERR_ABORTED errors. Are you detecting and surfacing these errors to your users? There is also the special OT_CHROME_MICROPHONE_ACQUISITION_ERROR error which is due to a known issue with Chrome that has been mostly fixed since Chrome 63 (see https://bugs.chromium.org/p/webrtc/issues/detail?id=4799).
Web-Client: Network
Unfortunately this is one of the more difficult issues to troubleshoot. Yes, Subscriber#getStats() is the best tool we have at our disposal and is a wrapper around the native RTCPeerConnection#getStats() function. Unfortunately we don't have much control over the values returned by the native function and if you think our SDK is returning incorrect values when compared with values from RTCPeerConnection#getStats() then please let us know!
It would be worthwhile confirming whether the issue is reproducible in all browsers or only a particular one. If you have detailed data regarding the inaccuracy of the native RTCPeerConnection#getStats() function then we could work together to report it to the browser vendor(s).
Fortunately we have just released the new Publisher#getStats() function which lets you get the publisher side of the stats. This should help you narrow down the cause of a connectivity issue to either a publisher or subscriber side. Please let us know if this helps with tracking down these issues.
Pre-Call Test
Again, these tests are based on Subscriber#getStats() which in turn are based on RTCPeerConnection#getStats(), the accuracy of which is out of our hands, but we'd love any reproduction steps to either fix a bug in our client SDK or report a bug to the browser vendors.
Just to confirm though, when you say you've implemented a pre-call test in your site, did you use the official JavaScript network test module? https://github.com/opentok/opentok-network-test-js This is actually what's used by the TokBox pre-call test.
#Aiham, thanks for responding, I've been looking at the the new Publisher#getStats() you linked to (thank you!), so we too can give our users some way of visibly seeing the network conditions that might be affected the quality of their call (and who's causing it). However, it seems as though bytes / packets sent goes up sharply as the number of subscribers increases, even though we're in a routed session.
Am I wrong to expect the Publisher#getStats() statistics to stay fairly stable regardless of the number of subscribers then receiving that stream in a routed session? I expected the nature of a routed call to mean it's sent once to the OpenTok Media Servers, and the statistics would end there.
I have a client/server audio synthesizer where the server (java) dynamically generates an audio stream (Ogg/Vorbis) to be rendered by the client using an HTML5 audio element. Users can tweak various parameters and the server immediately alters the output accordingly. Unfortunately the audio element buffers (prefetches) very aggressively so changes made by the user won't be heard until minutes later, literally.
Trying to disable preload has no effect, and apparently this setting is only 'advisory' so there's no guarantee that it's behavior would be consistent across browsers.
I've been reading everything that I can find on WebRTC and the evolving WebAudio API and it seems like all of the pieces I need are there but I don't know if it's possible to connect them up the way I'd like to.
I looked at RTCPeerConnection, it does provide low latency but it brings in a lot of baggage that I don't want or need (STUN, ICE, offer/answer, etc) and currently it seems to only support a limited set of codecs, mostly geared towards voice. Also since the server side is in java I think I'd have to do a lot of work to teach it to 'speak' the various protocols and formats involved.
AudioContext.decodeAudioData works great for a static sample, but not for a stream since it doesn't process the incoming data until it's consumed the entire stream.
What I want is the exact functionality of the audio tag (i.e. HTMLAudioElement) without any buffering. If I could somehow create a MediaStream object that uses the server URL for its input then I could create a MediaStreamAudioSourceNode and send that output to context.destination. This is not very different than what AudioContext.decodeAudioData already does, except that method creates a static buffer, not a stream.
I would like to keep the Ogg/Vorbis compression and eventually use other codecs, but one thing that I may try next is to send raw PCM and build audio buffers on the fly, just as if they were being generated programatically by javascript code. But again, I think all of the parts already exist, and if there's any way to leverage that I would be most thrilled to know about it!
Thanks in advance,
Joe
How are you getting on ? Did you resolve this question ? I am solving a similar challenge. On the browser side I'm using web audio API which has nice ways to render streaming input audio data, and nodejs on the server side using web sockets as the middleware to send the browser streaming PCM buffers.
I'm subscribing to live data with the Bloomberg API. Occasionally, it hangs on the call to session.Cancel(correlationID)
Anyone know why?
Where can I find documentation on the API?
I assume that you are talking about the .NET or Java API. In either case you should be able to find documentation (pdfs) by running WAPI on a Bloomberg terminal.
The Bloomberg API can be run in two modes - synchronous and asynchronous. So if you've taken some code example using WAPI and it happens to have been synchronous, you will face delays in your application.
The mode differs in the way data is accessed, for e.g.
the COM API in asynchronous mode would first send out the request using one procedure and another procedure is called back to execute when the data is fetched and ready, hence enabling the user to continue interacting with the GUI.
The synchronous mode would handle data requests and fetching in the same function with the same thread causing the app to hang. It won't make a big difference for the single value return types, but some large data sets could cause delays depending on your leased-line or internet bandwidth.
Is your question referring to the Bloomberg's Excel Add-In or its API Library releases to access live dataI? In either case, unless the data is not widely available to the public, and unless you have a special subscription arrangement from Bloomberg or other data feeds that can be sourced through the terminal, you are going to run into limits on the amount of live data that you are able to gather in any single interval.
To answer your second question, you can access Documentation for Bloomberg's Developers API here. And you can find Documentation and Resources for Bloomberg's API Libraries / Releases here.
The canonical example here is Twitter's API. I understand conceptually how the REST API works, essentially its just a query to their server for your particular request in which you then receive a response (JSON, XML, etc), great.
However I'm not exactly sure how a streaming API works behind the scenes. I understand how to consume it. For example with Twitter listen for a response. From the response listen for data and in which the tweets come in chunks. Build up the chunks in a string buffer and wait for a line feed which signifies end of Tweet. But what are they doing to make this work?
Let's say I had a bunch of data and I wanted to setup a streaming API locally for other people on the net to consume (just like Twitter). How is this done, what technologies? Is this something Node JS could handle? I'm just trying to wrap my head around what they are doing to make this thing work.
Twitter's stream API is that it's essentially a long-running request that's left open, data is pushed into it as and when it becomes available.
The repercussion of that is that the server will have to be able to deal with lots of concurrent open HTTP connections (one per client). A lot of existing servers don't manage that well, for example Java servlet engines assign one Thread per request which can (a) get quite expensive and (b) quickly hits the normal max-threads setting and prevents subsequent connections.
As you guessed the Node.js model fits the idea of a streaming connection much better than say a servlet model does. Both requests and responses are exposed as streams in Node.js, but don't occupy an entire thread or process, which means that you could continue pushing data into the stream for as long as it remained open without tying up excessive resources (although this is subjective). In theory you could have a lot of concurrent open responses connected to a single process and only write to each one when necessary.
If you haven't looked at it already the HTTP docs for Node.js might be useful.
I'd also take a look at technoweenie's Twitter client to see what the consumer end of that API looks like with Node.js, the stream() function in particular.