Stream webcam video with gstreamer 1.0 over UDP to PC - udp

Im trying to stream video from a Raspberry Pi (on Raspbian) to a Windows 7 PC like in this video: https://www.youtube.com/watch?v=lNvYanDLHZA
I have a Logitech C270 connected to the Raspberry Pi, and have managed to stream webcam video over TCP using:
gst-launch v4l2src device=/dev/video0 ! \
'video/x-raw-yuv,width=640,height=480' ! \
x264enc pass=qual quantizer=20 tune=zerolatency ! \
rtph264pay ! tcpsink host=$pi_ip port=5000
from my Pi. Receive this using VLC works, but with a 3 sec delay.
I want to do this over UDP to get a shorter delay (correct me if I'm wrong). But cannot for the life of me figure it out. I have tried following:
gst-launch-1.0 v4l2src device=/dev/video0 ! \
'video/x-raw-yuv,width=640,height=480' ! \
x264enc pass=qual quantizer=20 tune=zerolatency ! \
rtph264pay ! udpsink host=$pc_ip port=1234
and
gst-launch-1.0 udpsrc port=1234 ! \
"application/x-rtp, payload=127" ! \
rtph264depay ! ffdec_h264 ! fpsdisplaysink sync=false text-overlay=false
For the Pi and PC side, respectively (taken from
Webcam streaming using gstreamer over UDP)
but with no luck. (tried to change the video/x-raw-yuv to fit 1.0 version but still without luck)
Any hints would be highly appreciated!
Edit
With the raspi camera (not the webcam) the following works:
Windows batch script:
#echo off
cd C:\gstreamer\1.0\x86_64\bin
gst-launch-1.0 -e -v udpsrc port=5000 ! application/x-rtp, payload=96 !
rtpjitterbuffer ! rtph264depay ! avdec_h264 ! fpsdisplaysink sync=false
text-overlay=false
Raspberry Pi Bash Script:
#!/bin/bash
clear
raspivid -n -t 0 -rot 270 -w 960 -h 720 -fps 30 -b 6000000 -o - | gst-
launch-1.0 -e -vvvv fdsrc ! h264parse ! rtph264pay pt=96 config-interval=5 !
udpsink host=***YOUR_PC_IP*** port=5000
But I cannot figure out how to use to webcam instead of the raspberry pi camera (i.e. v4l2src instead of raspivid) in the same manner
Edit 2
The following works, but is very slow and has a huge delay:
RPi
gst-launch-1.0 -vv -e v4l2src device=/dev/video0 \
! videoscale \
! "video/x-raw,width=400,height=200,framerate=10/1" \
! x264enc pass=qual quantizer=20 tune=zerolatency \
! h264parse \
! rtph264pay config-interval=5 pt=96 \
! udpsink host=$myip port=$myport
PC:
gst-launch-1.0 -e -v udpsrc port=5001 ! ^
application/x-rtp, payload=96 ! ^
rtpjitterbuffer ! ^
rtph264depay ! ^
avdec_h264 ! ^
autovideosink sync=false text-overlay=false
I now suspect that (thanks to hint from #Mustafa Chelik) that the huge lag is due to the fact that the raspberry pi has to encode the webcam video, while the raspberry pi video is already encoded, not sure if this makes sense though?

Found hints to the solution from http://www.z25.org/static/rd/videostreaming_intro_plab/
The following worked very well for streaming video from Logitech c270 on raspberry pi to a windows 7 pc:
PC side:
gst-launch-1.0 -e -v udpsrc port=5001 ! ^
application/x-rtp, encoding-name=JPEG,payload=26 ! ^
rtpjpegdepay ! jpegdec ! ^
autovideosink
RPi side:
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! "image/jpeg,width=1280, height=720,framerate=30/1" \
! rtpjpegpay \
! udpsink host=$myip port=$myport
I suspect that it was the encoding of the webcam video to h264 that was too slow on the raspberry pi, however the webcamera already gave jpeg frames and thus no encoding was nescessary using "image/jpeg"

I have used for my webcamstream the MJPG-Streamer and get a 0,2 seconds delay.
http://wiki.ubuntuusers.de/MJPG-Streamer
And the advantage is that you can watch it with the webbrowser.

Related

Implement GStreamer pipeline using UDp socket #command line

I want to tranfer the .mp4 file from one terminal to another terminal using Gstreamer pipeline.
first I have test using videotestsrc
#server side gst-launch-1.0 -v videotestsrc pattern=ball ! video/x-raw,width=1280,height=720 ! jpegenc ! rtpjpegpay ! udpsink name=sink host=localhost port=34400 sync=false async=false
#client gst-launch-1.0 udpsrc port=34400 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, payload=(int)26" ! rtpjpegdepay ! jpegdec ! filesink location=a.mp4
but I want read .mp4 file from local directory so I have used following command
Server: $gst-launch-1.0 -v filesrc location =<file_path/>video_test.mp4 ! qtdemux ! video/x-h264 ! rtph264pay ! udpsink host=127.0.0.1 port=9001
#client: $gst-launch-1.0 -v udpsrc port=9001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! filesink location=a.mp4
but i am unable to capture # client side because a.mp4 file contains 0 bytes.
#server : gst-launch-1.0 -v filesrc location=//video_test.mp4 ! qtdemux name=demux demux. ! h264parse config-interval=10 ! queue ! mux. mpegtsmux name=mux ! rtpmp2tpay ! udpsink host=127.0.0.1 port=5002 demux. ! aacparse ! queue ! mux.
#client :gst-launch-1.0 -v udpsrc port=5002 caps="application/x-rtp" ! rtpmp2tdepay ! tsparse ! filesink location=x.mp4
Its working fine for me.

Stream UDP to Android Device using gStreamer Tutorial 3

Based on the gStreamer Tutorial 3 for Android i'm trying to implement a UDP connection.
On the server side i have a raspberry pi with the following line to start the server.
raspivid -t 0 -hf -n -h 480 -w 640 -fps 15 -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264sparse ! rtph264pay config-interval=10 pt=96! gdppay ! udpsink host=192.168.1.1 port=5000 
On the receiver side i have the tutorial 3 in which i changed the pipeline to the following line.
>data->pipeline = gst_parse_launch("udpsrc port=5000 caps=\"application/x-rtp, media=video, clock-rate=90000, payload=96\" ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink", &error);
I also included the following plugins:
>$(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_PLAYBACK) $(GSTREAMER_PLUGINS_CODECS) $(GSTREAMER_PLUGINS_CODECS_RESTRICTED) $(GSTREAMER_PLUGINS_NET) $(GSTREAMER_PLUGINS_SYS)
When i start the app there is a blackscreen with no video or audio.
In Logcat i get an error that H264 and AAC mapping is not possible.
Maybe i have to include sprop-parameter-sets?
And if i have to, how can i easily do that with the right syntax?
On the server side you use rtph264pay and gdppay. You should remove gdppay.

GStreamer v1.0 UDP Multicast streaming not properly decoded if client starts after server

I am trying to make UDP Multicast screen streaming using GStreamer. My screen casting server should run on Windows and my client should run on Linux.
If I start the client before the server, everything is fine.
The problem is when I start the client and the server had already been launched. The video is received, but it is terribly distorted.
This is the result.
Server:
gst-launch-1.0 -e gdiscreencapsrc ! queue ! video/x-raw, framerate=25/1 ! videoconvert ! \
x264enc noise-reduction=10000 tune=zerolatency bitrate=2500 speed-preset="fast" byte-stream=true threads=4 key-int-max=15 intra-refresh=true ! \
h264parse ! rtph264pay config-interval=1 \
! udpsink host=224.1.1.1 port=5000 auto-multicast=true
Client:
gst-launch-1.0 -v udpsrc multicast-group=224.1.1.1 auto-multicast=true port=5000 ! application/x-rtp ! rtph264depay ! h264parse ! queue ! decodebin ! videoconvert ! autovideosink caps='video/x-raw, format=RGB'
I have already tried using dx9screencapsrc, but the behaviour is the same.
The issue is fixed only if I replace gdiscreencapsrc with videotestsrc.
If I launch the server on Linux, using ximagesrc, I still have some issues, but the video is improving over time.
Any help would be appreciated!
Adding cabac=false to my x264enc element fixed the ussue.
gst-launch-1.0 -v gdiscreencapsrc ! queue ! video/x-raw,framerate=60/1 ! decodebin ! videoscale ! videoconvert ! \
x264enc cabac=false tune=zerolatency bitrate=4000 speed-preset="fast" ! \
h264parse ! rtph264pay config-interval=-1 \
! udpsink host=224.1.1.1 port=5000 auto-multicast=true sync=false

Why can I stream h264 encoded video from webcam to BOTH display and file, but NOT raw video?

I want to stream raw video from a Logitech C920 webcam and while both displaying and saving the video to file using GStreamer 1.0.
This works if I stream h264 encoded video from the camera (the camera provides hardware encoded h264), but it fails if I stream raw video from the camera. However, if I only display, or only save to file, streaming raw video works.
Why does it work with a h264 video stream but not with a raw video stream?
h264 encoded video stream from camera to BOTH display and file (WORKS):
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! video/x-h264,width=640,height=480,framerate=15/1 ! tee name=t \
t. ! queue ! h264parse ! avdec_h264 ! xvimagesink sync=false \
t. ! queue ! h264parse ! matroskamux \
! filesink location='h264_dual.mkv' sync=false
raw video stream from camera to ONLY display (WORKS):
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
! xvimagesink sync=false
raw video stream from camera to ONLY file (WORKS):
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
! videoconvert ! x264enc ! matroskamux \
! filesink location='raw_single.mkv' sync=false
raw video stream from camera to BOTH display and file (FAILS):
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
! tee name=t \
t. ! queue ! xvimagesink sync=false \
t. ! queue ! videoconvert ! x264enc ! h264parse ! matroskamux \
! filesink location='raw_dual.mkv' sync=false
The last command (raw video to both display and file) fails without any warnings or errors. The gst-launch terminal output is exactly the same as when only writing to file. The xvimage window appears and displays an image from the camera, but the image does not change (i.e. it is frozen). A zero bytes file appears too.
I have tried multiple versions of the above commands, but I think those are the minimal commands that can reproduce the problem.
Does anyone understand what I am doing wrong?
Streaming raw video from a webcam (not specific to C920) to both display and h.264 encoded file can be done. The x264enc property tune needs to be set to zerolatency.
h.264 example:
gst-launch-1.0 -v v4l2src device=/dev/video0 \
! video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 \
! tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \
videoconvert ! x264enc tune=zerolatency ! h264parse ! \
matroskamux ! filesink location='raw_dual.mkv' sync=false
Alternatively, one can skip h.264 altogether and encode to theora or vp8 instead.
theora example:
gst-launch-1.0 -v v4l2src device=/dev/video0 ! \
video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 ! \
tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \
videoconvert ! theoraenc ! theoraparse ! \
matroskamux ! filesink location='raw_dual.mkv' sync=false
vp8 example:
gst-launch-1.0 -v v4l2src device=/dev/video0 ! \
video/x-raw,format=YUY2,width=640,height=480,framerate=15/1 ! \
tee name=t t. ! queue ! xvimagesink sync=false t. ! queue ! \
videoconvert ! vp8enc ! \
matroskamux ! filesink location='raw_dual.mkv' sync=false
Thanks a lot to Jan Spurny and Tim.

GStreamer Videomixer Raspivid

I have searched all over the place and have not found anyone using the videomixer function from gstreamer with the raspberry pi's raspivid.
I am trying to duplicate the raspivid output and merge them side by side and then eventually send a stream over tcp. But for right now I am just looking for some help with getting the videomixing to work.
The resulting video should be 1280x568 for my specific application and I do not care that there is any angle between the videos to create a "3d effect" because it is not required for the specific application I'm making.
I am using gstreamer 1.2 so the function call is gst-launch-1.0 and I can not use ffmpeg b/c I believe it has depricated, so I assume I would just use videoconvert to achieve the same result.
Im not sure if I should be using h264parse instead of decodebin. So here is what Ive got so far:
gst-launch-1.0 fdsrc | raspivid -t 0 -h 568 -w 640 -fps 25 -hf -b 2000000 -o - ! decodebin ! queue ! videoconvert ! videobox border-alpha=0 right=-640 ! videomixer name=mix ! videoconvert ! autovideosink fdsrc | raspivid -t 0 -h 568 -w 640 -fps 25 -hf -b 2000000 -o - ! decodebin ! queue ! videoconvert ! videobox border-alpha=0 left=-640 ! mix.
Im trying to model this based off these two sources(the raspivid command in the first link works for me):
http://www.raspberry-projects.com/pi/pi-hardware/raspberry-pi-camera/streaming-video-using-gstreamer
http://www.technomancy.org/gstreamer/playing-two-videos-side-by-side/
I know I am probably so far off but I am having a lot of difficulty finding examples of how to do this, especially with the raspivid function. I would greatly appreciate any help. Thank You.
You can find a exemple, with some explaination how to use videomixer using videomixer
Example of using videomixer combining 3 videos
Note: UNIX paths are used in this example
gst-launch-1.0 -e \
videomixer name=mix background=0 \
sink_1::xpos=0 sink_1::ypos=0 \
sink_2::xpos=200 sink_2::ypos=0 \
sink_3::xpos=100 sink_3::ypos=100 \
! autovideosink \
uridecodebin uri='file:///data/big_buck_bunny_trailer-360p.mp4' \
! videoscale \
! video/x-raw,width=200,height=100 \
! mix.sink_1 \
uridecodebin uri='file:///data/sintel_trailer-480p.webm' \
! videoscale \
! video/x-raw,width=200,height=100 \
! mix.sink_2 \
uridecodebin uri='file:///data/the_daily_dweebs-720p.mp4' \
! videoscale \
! video/x-raw,width=200,height=100 \
! mix.sink_3