gstreamer with "rtpvrawpay" fails due to client being unable to prepare media - camera

I have been playing gstreamer to stream video via RTSP and I need help !
At first I wanted to stream video by encoding the data to h264 format so I used x264enc encoder followed by rtph264pay for the payload.
I tried with the following command and it works perfectly:
examples$ ./test-launch --gst-debug=3 '( v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96 )'
But I also want to try sending raw video data over the network without the encoder so I tried with the following command but it fails.
examples$ ./test-launch --gst-debug=3 '( v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=30/1 ! rtpvrawpay name=pay0 pt=96 )'
The log is here:
examples$ ./test-launch --gst-debug=3 '( v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=30/1 ! rtpvrawpay name=pay0 pt=96 )'
stream ready at rtsp://127.0.0.1:8554/test
0:00:04.246791630 77643 0x7f6464016cc0 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:04.246814474 77643 0x7f6464016c60 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:<appsrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:04.247156839 77643 0x7f6464016d20 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:04.247209181 77643 0x7f6464016d20 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
0:00:04.246846899 77643 0x7f646c003400 WARN v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:04.247422399 77643 0x7f646c003400 WARN v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:04.249446256 77643 0x7f646c003400 WARN rtspmedia rtsp-media.c:1834:default_handle_message: 0x7f646c034160: got error Internal data stream error. (gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4))
0:00:04.249557496 77643 0x561f080aaf60 WARN rtspmedia rtsp-media.c:2127:wait_preroll: failed to preroll pipeline
0:00:04.249603261 77643 0x561f080aaf60 WARN rtspmedia rtsp-media.c:2384:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:04.254686872 77643 0x561f080aaf60 ERROR rtspclient rtsp-client.c:678:find_media: client 0x561f080b58b0: can't prepare media
0:00:04.254996068 77643 0x561f080aaf60 ERROR rtspclient rtsp-client.c:2210:handle_describe_request: client 0x561f080b58b0: no media
0:00:04.271635553 77643 0x7f6464016c00 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:<appsrc3:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:04.271635725 77643 0x7f646c003f60 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:<appsrc2:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:04.271760281 77643 0x561f080aaf60 WARN v4l2src gstv4l2src.c:695:gst_v4l2src_query:<v4l2src1> Can't give latency since framerate isn't fixated !
0:00:04.272014291 77643 0x7f646c003ea0 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src1> error: Internal data stream error.
0:00:04.272047911 77643 0x7f646c003ea0 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src1> error: streaming stopped, reason not-negotiated (-4)
0:00:04.272645141 77643 0x561f080aaf60 WARN rtspmedia rtsp-media.c:1834:default_handle_message: 0x7f646c0348e0: got error Internal data stream error. (gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin1/GstV4l2Src:v4l2src1:
streaming stopped, reason not-negotiated (-4))
0:00:04.272711585 77643 0x7f646c003400 WARN rtspmedia rtsp-media.c:2127:wait_preroll: failed to preroll pipeline
0:00:04.272742948 77643 0x7f646c003400 WARN rtspmedia rtsp-media.c:2384:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:04.277308540 77643 0x7f646c003400 ERROR rtspclient rtsp-client.c:678:find_media: client 0x561f080b59a0: can't prepare media
0:00:04.277484803 77643 0x7f646c003400 ERROR rtspclient rtsp-client.c:1993:handle_setup_request: client 0x561f080b59a0: media '/test' not found
The camera I am using is a USB camera and from
$ v4l2-ctl -d /dev/video0 --list-formats
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YUYV' (YUYV 4:2:2)
[1]: 'MJPG' (Motion-JPEG, compressed)
Why can't I stream raw image data over RTSP ?
Thanks,

These are formats rtpvrawpay accepts as input:
format: { (string)RGB, (string)RGBA, (string)BGR, (string)BGRA, (string)AYUV, (string)UYVY, (string)I420, (string)Y41B, (string)UYVP }
While your camera seems to list YUYV only which is not one of the supported formats. You will probably have to insert a videoconvert in this pipeline too.
YUYV -> UYVY conversion should just be re-arranging the data, so that should be a lossless transformation.

Related

gstreamer how to select the camera's format type

I have gstreamer installed on my computer and I want to use H264 format. From this command I know the camera supports it:
# v4l2-ctl -d /dev/video2 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Index : 1
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUYV 4:2:2
Index : 2
Type : Video Capture
Pixel Format: 'H264' (compressed)
Name : H.264
The command I would like to use for gstreamer is in the following:
./test-launch --gst-debug=3 '( v4l2src device=/dev/video2 ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconvert ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96 )'
Where do I set to use H264 ?
From the command I see the encoder is already set as x264enc and payloader is h264pay. Why do I need to encode to x264 since the data from the camera is already h264 ?
With this command I have this error:
# ./test-launch --gst-debug=3 '( v4l2src d
evice=/dev/video2 ! video/x-raw,width=640,height=480,framerate=30/1 ! videoconve
rt ! x264enc tune=zerolatency ! rtph264pay name=pay0 pt=96 )'
stream ready at rtsp://127.0.0.1:8554/test
0:00:56.085259093 4891 0x55ae286140 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "x264enc"!
0:00:56.085377739 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "x264enc"
0:00:56.085404562 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no sink [source=#0x7f740197c0]
0:00:56.088847062 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#0x7f74038210]
0:00:56.092296385 4891 0x55ae286140 WARN rtspmediafactory rtsp-media-factory.c:1427:default_create_element: recoverable parsing error: no element "x264enc"
0:00:56.123282895 4891 0x7f6c029990 FIXME default gstutils.c:3902:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:01:04.027984559 4891 0x7f6c029a80 WARN v4l2bufferpool gstv4l2bufferpool.c:754:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:01:04.240517944 4891 0x7f6c029a80 WARN basesrc gstbasesrc.c:2939:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:01:04.240787267 4891 0x7f6c029a80 WARN basesrc gstbasesrc.c:2939:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-linked (-1)
0:01:04.242162319 4891 0x7f7400f4a0 WARN rtspmedia rtsp-media.c:2433:default_handle_message: 0x7f7404b180: got error Internal data stream error. (../../../../gstreamer-1.12.2/libs/gst/base/gstbasesrc.c(2939): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-linked (-1))
0:01:04.242804976 4891 0x55ae286140 WARN rtspmedia rtsp-media.c:2728:wait_preroll: failed to preroll pipeline
0:01:04.242955184 4891 0x55ae286140 WARN rtspmedia rtsp-media.c:3032:gst_rtsp_media_prepare: failed to preroll pipeline
0:01:04.402276538 4891 0x55ae286140 ERROR rtspclient rtsp-client.c:977:find_media: client 0x55ae2c5170: can't prepare media
0:01:04.403620496 4891 0x55ae286140 ERROR rtspclient rtsp-client.c:2696:handle_describe_request: client 0x55ae2c5170: no media
0:01:04.405440236 4891 0x55ae286140 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "x264enc"!
0:01:04.405544819 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "x264enc"
0:01:04.405635132 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no sink [source=#0x7f740197c0]
0:01:04.405832684 4891 0x55ae286140 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#0x7f74038500]
0:01:04.411951382 4891 0x55ae286140 WARN rtspmediafactory rtsp-media-factory.c:1427:default_create_element: recoverable parsing error: no element "x264enc"
0:01:04.432344715 4891 0x7f6c0299e0 FIXME default gstutils.c:3902:gst_pad_create_stream_id_internal:<appsrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:01:05.191722944 4891 0x7f6c02a000 WARN v4l2 gstv4l2object.c:2186:gst_v4l2_object_add_interlace_mode:0x7f7405b090 Failed to determine interlace mode
0:01:05.208055340 4891 0x7f6c02a000 WARN v4l2 gstv4l2object.c:2834:gst_v4l2_object_get_nearest_size:<v4l2src1> Unable to try format: No such device
0:01:05.208426434 4891 0x7f6c02a000 WARN
You tell v4l2source via the caps you give downstream. Currently you have:
video/x-raw,width=640,height=480,framerate=30/1
which tells it to use a raw and therefore uncompressed format. Instead try something like this:
video/x-h264,width=640,height=480,framerate=30/1
instead.

How to fix " error message: 'Received RST_STREAM with error code 8'" in compiling tensorflow using bazel

I am trying to compile the latest tensorflow version (2.0) using bazel (0.24.1):
~/bin/bazel build -c opt --verbose_failures --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma //tensorflow/tools/pip_package:build_pip_package
I receive the following error messages:
Server terminated abruptly (error code: 1, error message: 'Received RST_STREAM with error code 8', log file: '/home/zhang/.cache/bazel/_bazel_zhang/72e7c60ab7642c4e9dd9ba0bb5e0d8db/server/jvm.out')
In the jvm.out file, it looks like:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/home/zhang/.cache/bazel/_bazel_zhang/install/7da6a92c096ada842b8d48c251312343/_embedded_binaries/A-server.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Please help me out. Thanks!

libav :0:: cabac decode of qscale diff failed at 4 26? While UDP Streaming in Gstreamer

I would like to stream the webcam data over UDP in mpegts format.
At Sending End::
gst-launch-1.0 -v v4l2src device=/dev/video1 ! videoconvert ! video/x-raw,width=720,height=576,framerate=25/1,format=I420 ! videoparse width=720 height=576 framerate=25/1 ! x264enc bitrate=2048 ref=4 key-int-max=20 byte-stream=true tune=zerolatency speed-preset=3 sliced-threads=true threads=4 ! video/x-h264,stream-format=byte-stream,profile=main ! queue ! mpegtsmux ! rtpmp2tpay ! udpsink host=192.168.2.149 port=8888 sync=true async=false qos=true qos-dscp=46
At Receiving End::
GST_DEBUG=4 gst-launch-1.0 -v udpsrc port=8888 caps=application/x-rtp,media=video,encoding-name=MP2T buffer-size=524288 ! rtpmp2tdepay ! tsdemux name=demuxer demuxer. ! queue max-size-buffers=0 max-size-time=0 ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink qos=true sync=true async=false
For few seconds, video is coming good, but after that it is getting disturbed, and not at all coming, struck at single frame.
And when i used GST_DEBUG=4, i observed below error:
0:00.000000000, position 0:00:10.397114020, duration 99:99:99.999999999
0:00:10.396195300 536 0x7f1328001300 ERROR libav :0:: cabac decode of qscale diff failed at 4 26
0:00:10.396212221 536 0x7f1328001300 ERROR libav :0:: error while decoding MB 4 26, bytestream -1
0:00:10.396255836 536 0xe03320 INFO libav :0:: concealing 1215 DC, 1215 AC, 1215 MV errors
But, if I stream the Data to localhost, then it is perfectly working.
After setting cabac=false on x264enc,
0:00.000000000, position 0:00:14.796602703, duration 99:99:99.999999999
0:00:15.020440888 15449 0x7f69dc4fe2c0 ERROR libav :0:: out of range intra chroma pred mode at 18 14
0:00:15.020464168 15449 0x7f69dc4fe2c0 ERROR libav :0:: error while decoding MB 18 14
0:00:15.020694828 15449 0x2513320 INFO libav :0:: concealing 1215 DC, 1215 AC, 1215 MV errors
What is the problem here? is this because of network bandwidth or any other else? And how to solve this problem?

How to make maximize performance of Gstreamer

I'm trying to stream video of camera and audio of mic using Gstreamer.
Use Odroid C1+ as a server and PC as a client.
When I use lan cable, it's perfact. But it's performance significantly decrease when I use WiFi.
I used
$ gst-launch-1.0 v4l2src ! gdppay ! tcpserversink host=localhost port=5000 alsasrc device="hw:1,0" ! gdppay ! tcpserversink host=localhost port=6000
$ gst-launch-1.0 tcpclientsrc host=localhost port=5000 ! gdpdepay ! autovideosink sync=false tcpclientsrc host=localhost port=6000 ! gdpdepay ! autoaudiosink sync=false
and
$ gst-launch-1.0 v4l2src ! videoconvert ! x264enc tune=zerolatency ! rtph264pay ! gdppay ! tcpserversink host=localhost port=5000 sync=false alsasrc device='hw:0,0' ! gdppay ! tcpserversink host=localhost port=6000 sync=false
$ gst-launch-1.0 tcpclientsrc host=localhost port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false tcpclientsrc host=localhost port=6000 ! gdpdepay ! autoaudiosink sync=false
How can I maximize the performance of the streaming of Gstreamer,
and also synchronize video and audio?
Update
$ gst-launch-1.0 v4l2src ! vaapih264enc ! rtph264pay ! gdppay ! tcpserversink host=localhost port=5000
Setting pipeline to PAUSED ...
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
libva info: va_openDriver() returns -1
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.so
libva info: va_openDriver() returns -1
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstVaapiEncodeH264:vaapiencodeh264-0: Could not initialize supporting library.
Additional debug info:
gstvideoencoder.c(1559): gst_video_encoder_change_state (): /GstPipeline:pipeline0/GstVaapiEncodeH264:vaapiencodeh264-0:
Failed to open encoder
Setting pipeline to NULL ...
Freeing pipeline ...
I ran this at Ubuntu16.04, VMware on i5-6400 CPU and no GPU PC for test.

Gstreamer ffdec_h264 missing

I am running this script to view cameras on network:
gst-launch udpsrc port=1234 ! "application/x-rtp, payload=127" ! rtph264depay ! ffdec_h264 ! xvimagesink sync=false
I am getting this error:
WARNING: erroneous pipeline: no element "ffdec_h264"
I am getting error with ffdec_h264. I have all the packages from g-streamer but I don't know which one I am missing.
when I run gst-inspect | grep 264
I get this output:
h264parse: legacyh264parse: H264Parse
x264: x264enc: x264enc
videoparsersbad: h264parse: H.264 parser
typefindfunctions: video/x-h264: h264, x264, 264
rtp: rtph264pay: RTP H264 payloader
rtp: rtph264depay: RTP H264 depayloader
Which shows I don't have this ffdec_h264
which package I am missing?
This might depend on your OS/distribution and GStreamer version.
Over here (Debian jessie, GStreamer 0.10.36) gst-inspect ffdec_h264 gives the following output:
Factory Details:
Long name: FFmpeg H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
Class: Codec/Decoder/Video
Description: FFmpeg h264 decoder
Author(s): Wim Taymans <wim.taymans#gmail.com>, Ronald Bultje <rbultje#ronald.bitfreak.net>, Edwar$
Rank: primary (256)
Plugin Details:
Name: ffmpeg
Description: All FFmpeg codecs and formats (system install)
Filename: /usr/lib/x86_64-linux-gnu/gstreamer-0.10/libgstffmpeg.so
Version: 0.10.13
License: GPL
Source module: gst-ffmpeg
Binary package: FFmpeg
Origin URL: http://ffmpeg.org/
So on my system, ffdec_h264 is in the gst-ffmpeg module (which was installed using apt-get install gstreamer0.10-ffmpeg).
You have to enter this command and have an idea what h264 decoder you have with gstreamer, in other words what Gstreamer is calling it.
gst-inspect | grep "h264"