Flickering and failing video streaming with uvc-gadget and g_webcam - camera

Am using this commit of uvc-gadget together with g_webcam as of 4.4.143 for Rockchip. This version of uvc-gadget only transmits a static mjpeg image (and is much better written than earlier sources of uvc-gadget).
Observing interesting behavior on host laptop, which is receiving the stream from gadget with guvcview: after a while frames start to flicker like an old TV (V4L2_CORE: (jpeg decoder) error while decoding frame), and then eventually the stream breaks down on host: V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable. Underneath host continues polling ([75290.124695] uvcvideo: uvc_v4l2_poll), there is no error neither in host's dmesg, nor in uvc-gadget on device. In fact, after re-opening guvcview streaming works again without uvc-gadget restart, but soon crashes in the same way.
I'm using stock USB3.0 cable, which is both for streaming and powering the device. AFAIK, there is no source of noise that may result in such kind of strange flickering on physical level.
Additionally, I've noticed with smaller USB packet sizes going down from 1024 to 256, stream survives for longer (up to 50,000 frames or so), but still finally crashes.
Any idea what's going on here?
UPDATE
After I switched from MJPEG-compressed to uncompressed stream, there is no longer flickering, but still always loss of contact after several seconds: V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable

Related

WebRTC video and photo at same time

I'm working on an application that transmits video in low quality using webrtc. Periodically I want to send from same camera single frame in high resolution.
When I try to acquire another stream using getUserMedia I get same low quality one and when I try to pass some constraints to force higher resolution then then operation fails with overconstrained error (even though normally when there is no other stream it works fine).
Is it even possible to have at the same time many streams with different parameters from same device? Or is it possible acquire high resolution image without requesting for a new stream?

Fast image stream using objective C

I've been working on a project which involves a network connection between a server on a Mac, and a client on an iOS device.
The client side requires a live view of of the server's screen.
As of now, my solution has been
getting snapshots of the Mac's screen using CGDisplayCreateImage( ). Timing is done with an NSTimer set to fire every 1/30.0 second.
compressing and generating GIF images using CGImageDestination
sending the GIF images (which are stored in NSData) over Bonjour to the client.
client receives the data, and initializes a new UIImage from it, passing it into an image view.
But I've been encountering a few problems with this method.
The latency is horrible. Each image took many seconds to load, and seemed to never even reach the client.
Each image NSData object's size was around 170000 bytes long. This seems WAY too high to even send over network fast enough. But I have no idea how to compress the images further.
Thank you.

Debugging an intermittently unresponsive USB device

My app communicates with a simple USB device as follows:
The app sends commands (2 or 3 bytes each) to the USB device by using WriteFile (kernel32.dll).
For each command that is send, the USB device sends a short response, which the PC receives using ReadFile (kernel32.dll).
Reading and writing is done asynchronously, using GetOverlappedResult to check the status of an operation.
Testing on 2 out of 3 PCs, the app and device function perfectly: all responses are received 100% reliably.
Under identical tests on the third PC, approximately 50% of the ReadFile requests do not return any data - the status remains as pending (ERROR_IO_INCOMPLETE) forever.
In other words, approximately for every 2 commands sent, one response is received.
Because the device functions perfectly with the other PCs, it lead me to believe that the problem might be occuring inside Windows, in the underlying code which is called by ReadFile (I presume some lower level USB driver code).
Question:
Please could you advise what debugging tool is most useful to investigate this? With my current knowledge, the internal workings of ReadFile are quite opaque.
The PC which is experiencing the issue is running Windows 8.0
You could try DebugView. Run as admin.
Go to "Capture", enable "Capture Kernel", enable "Enable Verbose Kernel Output".
This might help to investigate errors on Kernel level, if any occured.

alsa - managing non-blocking stream

Working with a usb audio device (its a HID with multiple channels) that constantly outputs data.
What I'm hoping to achieve is to ignore the audio until a signal comes in from the device. At that point I would start monitoring the feed. A second signal from the device would indicate that I can go back to ignoring the data. I have opened said device in non-blocking mode so it won't interfere with other USB signals coming in.
This is working fine except that when I start reading the data (via snd_pcm_readi) I get an EPIPE error indicating a buffer overrun. This can be fixed by calling snd_pcm_prepare every time but I'm hoping there is a way to let the buffer empty while Im ignoring it.
I've looked at snd_pcm_drain and snd_pcm_drop but these stop the PCM and I'd rather keep it open.
Suggestions?
To ignore buffer overruns, change the PCM device's software parameters: set the stop threshold to the same value as the boundary.
With that configuration, overruns will not cause the device to stop, but will let it continue to fill the buffer.
(Other errors will still stop the device; it would be hard to continue when a USB device is unplugged ...)
When an overrun has happened, the buffer will contain more data than actually can fit into it, i.e., snd_pcm_avail will report more available frames than the buffer size.
When you want to actually start recording, you should call snd_pcm_forward to throw away all those invalid frames.

USB CDC device stalling

I'm writing a simple virtual serial port device to report an older serial port. By this point I'm able to enumerate the device and send/receive characters.
After a varying number of bulk-out transmissions from the host to the device the endpoint appears to give up and stop transferring data. On the PC side I receive a write error, and judging from a USBlyzer trace the music stops on a stall (USBD_STATUS_STALL_PID). However my code never intentionally issues a STALL condition on that endpoint and the status flag for having generated one never gets set though.
Given the short amount of time elapsed (<300 µs) between issuing the request and the STALL it would appear to be an invalid response of some sort, and not a time-out. On the device side the output endpoint is ready to go, with data in the buffer and proper DATA0/1 synchronization, but nothing further ever happens.
Note that the device appears to work fine even for long periods of time until I start sending "large" quantities of data. As near as I can tell the device enumeration/configuration also appears to complete successfully. Oh, and the bulk-in endpoint continues to work just fine after this.
For the record I'm using the standard Windows usbser.sys driver and an XMega128A4U µP. I'm also seeing the same behaviour across multiple Windows Vista and 7 machines.
Any ideas what I'm doing wrong or what further tests I might run to narrow things down?
USBlyzer log,
USB CDC stack,
test project
For the record this eventually turned out to be an oscillator problem. (Apparently the FLL's reference is always 1,024 Hz even when the 1,000 Hz USB frames are chosen. The slight clock error meant that a packet occasionally got rejected if it happened to contain one too many 1-bits in a row.)
I guess the moral of the story is to check the basics before assuming you've got a problem with the higher-level protocol. Also in retrospect a hardware USB analyzer would have been a worthwhile investment, the software alternatives mostly seems to spit out a generic error code or nothing at all when something goes awry.
Stalling the out-endpoint may happen on an overflow of the output buffer on the host side. Are you sure that the device does fetch the data it receives via out-endpoint - and if so does it fetch the data at least as fast as data is sent to the device?
Note that the device appears to work fine even for long periods of
time until I start sending "large" quantities of data.
This seems to be a hint for an overflow of the output-buffer.