I'm transcoding videos to HLS and am able to validate the stream via:
$mediastreamvalidator <url-to-master-playlist.m3u8>
Mediastreamvalidator gives no errors and all segments seem ok.
However on many videos the progress bar contains a small white/unloaded segment which never changes despite the video segments fully loading. Does this have to do with my encoding properties?
Here is the mediastreamvalidator output for one of the errant segments:
Processed 3 out of 3 segments
Average segment duration: 3.686918
Total segment bitrates (all discontinuities): average: 2293.51 kb/s, max: 2562.78 kb/s
Playlist max bitrate: 2374.000000 kb/s
Audio Group ID: AUDIO
Discontinuity: sequence: 0, parsed segment count: 3 of 3, duration: 11.061 sec, average: 2293.51 kb/s, max: 2562.78 kb/s
Track ID: 1
Audio Codec: AAC-LC
Audio sample rate: 44100 Hz
Audio channel layout: Stereo (L R)
Track ID: 2
Video Codec: avc1
Video profile: Main
Video level: 3.1
Video resolution: 480x360
Video average IDR interval: 3.409137, Standard deviation: 0.000005
Video frame rate: 26.400
Related
I am trying to set the fps of the coral mipi camera to 60fps. From the datasheet, it says it can run 720p at 60fps so i know the camera is capable.
I have set the resolution using the following command which works.
v4l2-ctl --device=/dev/video1 --set-fmt-video=width=1280,height=720
but when I set the fps to 60, the max limit seem to be 30.
mendel#mocha-calf:~$ v4l2-ctl --device=/dev/video1 -p 60
Frame rate set to 30.000 fps
There are no option to set exposure or gain. Would I have to rebuild the driver to be able to have these options?
Regards
Paul
60 FPS not supported. Please see the supported formats below.
mendel#tuned-rabbit:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920
And for changing/setting exposure and gain, you may need to make modifications in the current driver code and rebuild the kernel image. Driver code can be found at:
https://coral.googlesource.com/linux-imx/+/refs/heads/master/drivers/media/platform/mxc/capture/ov5645_mipi_v2.c#2774
And instructions to build the kernel image can be seen at : https://coral.googlesource.com/docs/+/refs/heads/master/GettingStarted.md
My source video file (1h 30min movie) is playable in both PotPlayer and VLC: h264, 8-bit color and 7755kb/s bitrate.
The NVEnc command I'm using is this:
.\nvencc\NVEncC64.exe --avhw -i "input.mkv" --codec hevc --preset quality --bframes 4 --ref 7 --cu-max 32 --cu-min 8 --output-depth 10 --audio-copy --sub-copy -o "output.mkv"
Encoding works fine (I believe):
NVEncC (x64) 5.26 (r1786) by rigaya, Jan 31 2021 09:23:04 (VC 1928/Win/avx2)
OS Version Windows 10 x64 (19042)
CPU AMD Ryzen 5 1600 Six-Core Processor [3.79GHz] (6C/12T)
GPU #0: GeForce GTX 1660 (1408 cores, 1830 MHz)[PCIe3x16][457.51]
NVENC / CUDA NVENC API 11.0, CUDA 11.1, schedule mode: auto
Input Buffers CUDA, 21 frames
Input Info avcuvid: H.264/AVC, 1920x800, 24000/1001 fps
AVSync vfr
Vpp Filters cspconv(nv12 -> p010)
Output Info H.265/HEVC main10 # Level auto
1920x800p 1:1 23.976fps (24000/1001fps)
avwriter: hevc, eac3, subtitle#1 => matroska
Encoder Preset quality
Rate Control CQP I:20 P:23 B:25
Lookahead off
GOP length 240 frames
B frames 4 frames [ref mode: disabled]
Ref frames 7 frames, MultiRef L0:auto L1:auto
AQ off
CU max / min 32 / 8
Others mv:auto
encoded 142592 frames, 219.97 fps, 1549.90 kbps, 1098.83 MB
encode time 0:10:48, CPU: 8.7%, GPU: 5.2%, VE: 98.3%, VD: 21.5%, GPUClock: 1966MHz, VEClock: 1816MHz
frame type IDR 595
frame type I 595, avgQP 20.00, total size 39.44 MB
frame type P 28519, avgQP 23.00, total size 471.93 MB
frame type B 113478, avgQP 25.00, total size 587.45 MB
but when I try to play it in either PotPlayer or VLC it says there is no video track or it just doesn't play at all.
MediaInfo also doesn't show any video, audio, or subtitle tracks either, just the name of the file and the file size. Am I missing something?
Switching --avhw to --avsw solved the problem.
I tested if my ros2 node was subscribing exactly to /camera/depth/image_rect_raw topic from reassense ROS2 node.I attatched a realsense camera to TX2 board and configured as 15 fps.
However, I expected that the rate of subscribing /camera/depth/image_rect_raw should be similar to 15 hz but it is different as below. Why is there the rate difference between publising and subscribing image topic? Is it possible to match the subscribing rate to publishing rate?
$ ros2 topic hz /camera/depth/image_rect_raw
average rate: 10.798
min: 0.040s max: 0.144s std dev: 0.03146s window: 13
average rate: 8.610
min: 0.040s max: 0.357s std dev: 0.06849s window: 22
average rate: 8.085
min: 0.040s max: 0.357s std dev: 0.07445s window: 30
average rate: 9.498
min: 0.015s max: 0.357s std dev: 0.06742s window: 45
average rate: 9.552
min: 0.015s max: 0.415s std dev: 0.07555s window: 55
average rate: 9.265
min: 0.015s max: 0.415s std dev: 0.07543s window: 63
average rate: 8.510
min: 0.015s max: 0.415s std dev: 0.08619s window: 68
average rate: 7.940
min: 0.015s max: 0.480s std dev: 0.09757s window: 73
average rate: 7.539
min: 0.015s max: 0.480s std dev: 0.10456s window: 77
average rate: 7.750
min: 0.015s max: 0.480s std dev: 0.09972s window: 87
It is likely that the difference is due to the transport delay of putting the image into the network. The significance of this delay depends on whether your subscriber is running on the Jetson or if it is on a separate device on the same physical network. Regardless, I would suggest changing the default QoS policies to get better performance for video streaming such as setting RELIABILITY=BEST_EFFORT. That said the biggest improvement (if you are streaming over a network) will likely come from using the image_transport_plugins to compress the images before they are published. Although these are CPU based (theora, etc.) they will likely help.
Another thing to consider is using the compression hardware accelerators that are built into the Jetson though that will require some more work until the maintainers of image_transport_plugins or another enterprising developer gets this working.
I am running IP Webcam on Android which provides an mpjpeg video stream. I have to limit the capture frame rate to 5fps to save on battery.
However ffmpeg will still detect the input stream to be 25 fps, which causes it to be saved in the wrong speed causing timestamps and audio to be desynchronized.
Input #0, mpjpeg, from 'https://***:***#smarthome:8080/video':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
Input #1, ogg, from 'https://***:***#smarthome:8080/audio.opus':
Duration: N/A, start: 0.006500, bitrate: N/A
Stream #1:0: Audio: opus, 48000 Hz, mono, fltp
Metadata:
ENCODER : Lavf58.12.100
[stream_segment,ssegment # 0x19b49a0] Opening '/mnt/nas/SecurityCamera/2020-06-20_14-26-04.mkv' for writing
Output #0, stream_segment,ssegment, to '/mnt/nas/SecurityCamera/%Y-%m-%d_%H-%M-%S.mkv':
Metadata:
encoder : Lavf58.20.100
Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 tbr, 1k tbn, 25 tbc
Stream #0:1: Audio: opus, 48000 Hz, mono, fltp
Metadata:
ENCODER : Lavf58.12.100
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[ogg # 0x1753e00] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
frame= 8 fps=7.9 q=-1.0 size=N/A time=00:00:00.49 bitrate=N/A speed=0.489x
As you can see it detects Input #0 to be 25 tbr, 25 tbn, 25 tbc which results in an expected 25 fps, however the fps (shown high right now but it slowly approaching 5fps) is way below 25 which causes the speed to be <1x.
I have tried to use -r 5 -i ... and -vsync 2 and different values for -enc_time_base none of which had any impact. From https://trac.ffmpeg.org/ticket/403 I've learned that -r only works on inputs with unknown fps. But my input doesn't have unknown fps, it has the wrong fps.
Is there any way to force overwrite the input fps so that I can get a proper speed of 1x and synchronized timestamps and audio?
My code does not work with the phone recorded videos ( MP4 ) .
Have tested other videos mp4 and it works normally.
Does he not support the video phone ?
Video Features :
Format : MPEG - 4
Format profile: Base Media
Codec ID : isom
File size: 29.8 MiB
Duration : 21mn 2s
Overall bit rate: 198 Kbps
Writing application : Lavf52.84.0
Video
ID: 1
Format : MPEG - 4 Visual
Format profile: Simple # L1
Format settings , BVOP : No
Format settings , QPEL On
Format settings , GMC : No warppoints
Format settings , Matrix : Default ( H.263 )
Codec ID: 20
Duration : 21mn 2s
Bit rate: 97.5 Kbps
Width : 176 pixels
Height : 144 pixels
Display aspect ratio : 1.222
Frame rate mode: Constant
Frame rate: 29.970 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type: Progressive
Compression mode: Lossy
Bits / ( Pixel * Frame) : 0.128
Stream size: 14.7 MiB ( 49 % )
Writing library : Lavc52.97.0
audio
ID: 2
Format : MPEG Audio
Format version: Version 1
Format profile: Layer 3
Mode: Joint stereo
Mode extension: MS Stereo
Codec ID: 6B
Duration : 21mn 2s
Bit rate mode: Constant
Bit rate: 96.0 Kbps
Channel ( s ) : 2 channels
Sampling rate: 44.1 KHz
Compression mode: Lossy
Stream size: 14.4 MiB ( 48 % )
Writing library : LAME3.98.4
Your video format is MPEG-4 Visual, which chrome doesn't support.
http://code.google.com/p/chromium/issues/detail?id=54036
It's confusing, but that's not the same as a more standard MP4 file which uses the h.264 codec. Your info should look something like this:
General
Complete name : filename.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
...
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Baseline#L3.0
Format settings, CABAC : No
Format settings, ReFrames : 2 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
...
Try using something like HandBrake or Zencoder to transcode the video.