I note one feature for Logitech webcams - if the RightLight option is turned on in the webcam properties, then output frame rate decreases in two times (1920x1080 with 30 fps -> ~15 fps) for rendering. For test I used DirectShow and the Logitech webcams: c270, c310 and c920.
Any ideas - how to force a webcam to work with 30 fps with RightLight?
RightLight halves the framerate as part of how it functions. You are better off adding another light and turning off Rightlight.
Related
I have been working on echo cancellation on cross-platform smartphones (iOS & Android) clusters in a room. Since the phones in the cluster don't have fixed orientation with respect to the common form which is usually a circular or linear array of phones. The phones are placed randomly in the room.
All phones play the audio in synchronization with accuracy up to 10ms. But the problem is there is an echo (some kind of screeching sound) that doesn't get canceled. I have seen aec dump files and it seems that echoes are longer than the typical window of delay which the webrtc aecm caters with a default filter length of 128ms i.e.
For Android:
low latency phones : 50ms -> [50 ms to 170 ms]
High latency phones: 150ms ->[150ms to 270ms]
For iOS:
Recording latency: 50ms
Playing latency: 50ms
I have rectified webrtc code to use HW AEC in combination with WebRTC AECM but the problem is the echoes are outside these windows.
So how do I increase the filter length?
I am using the objective-c framework for WebRTC for building a screensharing app. The video is captured using CGDisplayStream. I have a working demo but at 2580x1080 I get only 3-4 fps. My googAvgEncodeMs is around 100-300ms (should be >10ms ideally) which explains why the screensharing is far from being fluid (30fsp+). I also switched between codecs (h264/vp8/vp9) but with all of them I get the same slow experience. The contentType in webRTC is set to screen (values: [screen,realtime]).
The cpu usage of my mac is then between 80-100%. My guess is that there is some major optimisation (qpMax, hardware-acceleration etc...) in the c++ code of the codecs that I have missed. Unfortunately my knowledge on codecs is limited.
Also interesting: Even when I lower the resolution to 320x240 the googAvgEncodeMs is still in the range of 30-60ms.
I am running this on a MacBook Pro 15 inch from 2018. When running a random webrtc inside Chrome/Firefox etc I get smoother results than with the vanilla webrtc framework.
WebRTC uses software encoding and that is the real culprit. Also encoding 2580 x 1080 in software is not going to be practical. Try reducing H and V resolution in half and it will improve performance with some loss in quality. Also if you are doing screen sharing and video support is not critical, you can drop frame rate to 10 frames per second. Logical solution is to figure out how to incorporate h/w acceleration.
Is the bit rate of black screen shown when video is muted same as the original video's bit rate or is it significantly less because it is just a black screen?
It is significantly less. Since there is essentially no video information being sent to the remote party. How much depends on a lot of factors (connection quality etc).
I just did a quick test and the outgoing bit rate at 640x480 # 27fps was around 900 kbps to 1 mbps. Disabling the video track of the stream resulted in an outgoing bitrate of 30 kbps.
Please keep in mind that this was only a simple test I did. You can get this kind of information yourself by evaluating the reports in peerConnection.getStats
Some documentation and libraries for getStats
https://www.w3.org/TR/webrtc-stats
https://github.com/muaz-khan/getStats
https://www.callstats.io/2015/07/06/basics-webrtc-getstats-api/
Came across chrome://webrtc-internals, which has inbuilt tracking for bit rates and has other good features.
As seen in graph, bit rate before video was muted was ~150k which reduces to ~30k on muting the video.
I am currently creating an application which signals readiness to other devices using a high frequency sound.
(transmitter): A device will produce a short burst of sound of around 20khz.
(receiver): Another device will be listening for a sound at this frequency at a small distance from the transmitter(10m approx) The device recieves audio data from a microphone
The background noise will be fairly loud, varying from around 0 - 10khz(about human speech range), and would be produced by a small crowd of people.
I need the receiving device to be able to detect the 20khz sound, separated from the noise,
and know the time at which it was received.
Any help with an appropriate algorithm, a library, or even better, code in C or
Objc to detect this high frequency sound would be greatly appreciated.
20 kHz may be pushing it, as (a) most sound cards have low pass (anti aliassing) filters at 18 - 20 kHz and (b) most speakers and microphones tend to have a poor response at 20 kHz. You might want to consider say 15 kHz ?
The actual detection part should be easy - just implement a narrow band pass filter at the tone frequency, rectify the output and low pass filter (e.g. 10 Hz).
You may want to look into FFT (Fast Fourier Transform). This algorithm will allow you to analyse a waveform and convert it to the frequency spectrum for further analysis.
If this is for Mac OS or iOS, I'd start looking into Core Audio's Audio Units.
1 Here's Apple's Core Audio Overview.
2 Some AudioUnits for Mac OS
3 Or for iOS AudioUnit Hosting
A sound with that high frequency will not travel at all with the iphone speaker.
I'm looking into connecting multiple low resolution USB webcams to a single computer. What implications might this have on performance? How does, for example, four 320x240 cameras fare against a single 640x480 camera? I'm not well versed in the architecture of the USB interface, what are the performance caveats? By performance I mean how would it affect the time to read the image data from multiple cameras compared to a single one.
Each webcam is connected to a different USB port? If so, its good.
Even if its just 1 port with 4 connected webcams. I dont think 4 320x240 will have any problem either. USB 2.0 = 320Mbps. Streaming a 320x240 video wouldn't be over 1mbps. Worst case scenario, putting a 320x240 at 2mbps + 1mb of other data. That would be 12mbps bandwidth between your usb port and the device.
So from the above, the 1 USB port can handle 4 webcams connected by a splitter just as fast as 1 640x480 webcam.,
Processing these images depends on your computer speed and how you write your algorithm.
The maximum data rate of USB is way higher than what you will actually get.
Webcams will probably use isochronous transfer, which under USB 2 can only get about 40% (if I recall correctly) of the bus time, and this also has a good bit of overhead.
I don't know for sure, but I suspect that this is why usb webcam resolutions and data rates seem to have hit a ceiling several years ago. They may start to increase again with the use of USB 3.
I'd suggest that you attach each of your cameras to it's own USB 2 port, as the 40% is shared among all isochronous connections.
One of those connections sharing bandwidth with a keyboard or even a usb mass storage device should be ok, because they would only use parts of the remainder of the bandwidth.
Wrong.
First, USB 2.0 is 480mbps theoretical, and you should be able to get up to about 80% of that with a direct connection.
Second, to calculate the bandwidth used by a camera, image bit depth must be taken into account, therefore:
BW = hresolution() * vresolution() * imagebitdepth(bit) * framerate(frame/s) (in bit/sec)
imagebitdepth can be, for webcams, 8, 16, 24, or 32 bits (ranging from Y800 monochrome to RGBA/RGBT colour full, check spec)
Therefore, a typical webcam # 640*480 resolution, 30fps, 16bit RGB bit-masked RGBA image bit depth will require 147.456 Mbps, and consequently, one of similar spec but # 320*240 resolution would require 36.864 Mbps, as opposed to the major BS stated by Shawn above with his 1mbps which then is also inconsistent with just about all of his other, also wrong data.
Simulatenous operation is nevertheless largely driver dependent, it is up to the manufacturer to take the otherwise minimal effort and expose unique device IDs to DirectShow.