I'm trying to forward an RTSP stream to a web browser using WebRTC with GStreamer. It's mostly working, but I'm having a couple hiccups. The video stream has a clock on it so it's easy to see these issues:
Will occasionally freeze for a few seconds.
Less occasionally, it will appear to show a frame from a few seconds earlier.
I have viewed the RTSP stream directly in VLC and it plays perfectly, so I don't think there's an issue with the stream itself.
Here are a couple of pipelines I've been playing around with:
rtspsrc location=<rtspurl> is-live=true ! rtpjitterbuffer ! queue ! rtph264depay !
rtph264pay aggregate-mode=zero-latency config-interval=1 !
webrtcbin turn-server=<turnurl> bundle-policy=max-bundle name=webrtcbin
rtspsrc location=<rtspurl> is-live=true ! rtpjitterbuffer mode=0 ! queue ! parsebin !
rtph264pay aggregate-mode=zero-latency config-interval=-1 timestamp-offset=0 ! queue !
webrtcbin turn-server=<turnurl> bundle-policy=max-bundle name=webrtcbin
I've also tried just watching the stream with gstreamer. For instance:
gst-launch-1.0 rtspsrc location=<rtspurl> ! decodebin ! autovideosink
This does a lot better than the WebRTC pipelines. It still freezes, but not nearly as much. And I haven't noticed it showing an earlier video frame.
I've looked at all the example pipelines I can find as well as the different options for the elements in my pipelines, but I can't find anything that makes this better. Is there something I can try adding to my pipeline to take care of the issues?
you have to check network buffer size and network queue size, lower the buffer size if network speed is slow. One more thing is these players caches the small content to storage to ensure the continued playback without interruption, altering cache size can also help you.
Related
I am capturing video from webcam in my PC and in the fly convert it to FLV (using ffmpeg).
As a result I have a continuously growing .FLV file.
And now I would like to play it as a live stream.
I was trying VLC but it plays the file no longer than the duration read from file on initialization.
What player can I use for live playing FLV?
I am working on Ubuntu 16.04.
Thank you in advance for your answers!
You cannot play live FLV directly but there is a tricky protocol popular among Chinese live streaming platform called "http-flv" that would play live flv within http framework.
Why http-flv?
Latency for HLS / Dash is long. It is about 10 to 20+ seconds.
Http-flv reduces end-to-end the latency to ~5 seconds. It could be played on browsers with MSE support.
How it works?
FLV is a simple container that "supports" file-based progressive streaming because one could get partial byte range in a flv video and still play it ( for mp4, you would need meta like moov etc for playback. )
For file server, host a growing flv file and remove the HTTP response header "content length" so that when client request the file, it does not know the response body size. It would keep the connection and receive videos segments until connection ends.
On client side, use flv.js to fetch only the latest segments for a flv file and perform the playback.
A lot of other tricks that would make the pipeline work.
There are a lot of source online you could play around with. Here are some references:
https://github.com/Bilibili/flv.js/
https://github.com/winshining/nginx-http-flv-module
A blog about how to achieve this: https://www.yanxurui.cc/posts/server/2017-11-25-http-flv/
so I've been wanting to make a real-time live streaming application. Essentially, the application would send the microphone feed from an Xcode application to a website where it can be viewed in real-time. Is FFMPEG the best solution for this? How would I go about doing this? If that's too broad, then how do I use the FFMPEG framework in an OS X objective-c application?
To directly address your questions:
(1) Is FFMPEG the best solution for this?
It depends. When setting up a live streaming environment you will likely stumble over FFMPEG, VLC and gstreamer, which are the options you have to simply stream video/audio. Therefore, yes, FFMPEG can be used as part of the solution. Please look into the following question: DIY Video Streaming Server
(2) How would I go about doing this?
Your requirement is to make a live streaming application which sends the mic input onto the web. This includes the following steps:
(0) Your Xcode application will need to provide a method to start this process. You don't necessarily need to integrate a framework to achieve this.
(1) Streaming / Restreaming
Use FFMPEG or VLC to grab your device and stream it locally:
ffmpeg -i audioDevice -acodec libfaac -ar 44100 -ab 48k -f rtp rtp://host:port
(2) Segmenting for HTTP Live Streaming*
Use a segmenter such as: mediastreamsegmenter (Apple), livehttp (VLC) or segment (FFMPEG) to prepare your stream for web delivery:
vlc -vvv -I dummy <SOURCEADDRESS> --sout='#transcode{acodec=libfaac,ab=48}:std{access=livehttp{seglen=10,delsegs=false,numsegs=10,index=/path/to/your/index/prog_index.m3u8,index-url=YourUrl/fileSequence######.ts},mux=ts{use-key-frames},dst=/path/to/your/ts/files/fileSequence######.ts}'
*you could also simply use VLC to grab your audiodevice with qtsound (see this question) and prepare it for streaming with livehttp.
(3) HTML 5 Delivery
Publish your stream
<audio>
<source src="YOUR_PATH/playlist.m3u8" />
</audio>
(3) If that's too broad, then how do I use the FFMPEG framework in an OS X objective-c application?
Either use an external wrapper framework to access FFMPEG functionality and consult the tutorials to work with these frameworks or you could also approach this by using NSTask to wrap your command line arguments in Objective-C and simply start those tasks from your application - as in this question.
Another way would be to use VLCKit, which offers VLC functionality in a framework for Objective-C (VLCKit wiki). However when tackling streaming challenges I prefer to work with the actual commands instead of pushing another layer of framework in between, which may be missing some options.
I hope this points you in the right directions. There are multiple ways to solve this. It's a broad question, therefore this broad approach to answer your question.
In the gstreamer - streamingtest example
(https://janus.conf.meetecho.com/streamingtest.html)
a gstreamer pipe is sending to udpsink host=127.0.0.1 port=5004, which then is broadcasted via webRTC in Janus.
how is it possible to send a webcam-stream from another user through his browser getUserMedia() to Janus-Gateway for broadcasting?
Do i have to configure a pipe for it as well and how would that look like?
I have installed Janus and i am able to run all the Demos.
there is a rtp_forward request possible against the videoroom which would forward the rtp from a publisher in that room to the streaming plug-in or any other ip.
it was added here:
https://github.com/meetecho/janus-gateway/pull/255
instead of rtp_listen though, you should request rtp_forward and also pass in the secret.
(this solution needs a browser, but I marked it as right solution since it works for me this way and also scaling users is possible like this)
I am doing some research on how to do two things: Trim and stream H.264 video.
What does it take to trim a mpeg4 h.264 video to 30 seconds and downsize it to 480p. I am assuming I would need to find a 3rd party library that does H.264 encoding, doing a quick Google search and the only thing I find is VideoLan.org, but I cannot find their commercial license. Are there other options folks know of?
How is streaming of H.264 to a HTML5 work? I know that with Flash, one can have one file format that requires the whole file to be downloaded, then it will play. The other format allows streaming, but requires a Flash server. I am going to be using Apache to serve up the images on the Intranet, how does one go about streaming them on Apache?
1) You can use FFmpeg :
ffmpeg -i in.mp4 -s 720x480 -t 30 out.mp4
-s is to resize and -t is to dump only 30 seconds
2) For http streaming, if the moov atomc(contains the video headers and seek information), is present at the start of the video, the video will start playing as soon as it buffers up few seconds, it does not wait for the whole file to download. Forward seek is possible through ByteRange headers in http. To put moov atom in the beginning use qt-fastart . It comes with FFmpeg
qt-faststart in.mp4 out.mp4
I m publishing a stream on red5 using microphone on client side as3 code . but it not published good stream but the same thing i m doing on FMS it creates perfect stream
I need to be understand what is the issue during publish on red 5 .
Read Red5 documentation for that. And ofcourse there are differences between the performances of the two servers. However if you want to improve the quality of stream you can use FFMPEG or Xuggler with Red5 to encode streams.
Because you are not saying what your encoder is, it is hard to give a clear answer. If you are using Adobe's FMLE to create the stream that goes to your FMS server, it is the FMLE that explains why you have good video and audio encoding 'out-of-the-box'.
I have never tried to use FMLE with RED5, so I cannot tell you if it works, but doubtful it works out-of-the-box. It probably can work with a bit of tweaking on both client and server side.
To use your own encoder, what you do is capture two streams using ffmpeg, a great example on how to do that is on stackoverflow here.
Once you are capturing, you can use ffmpeg to send the combined audio and video streams to a file, or you can send it directly to your red 5 server. A simplified version of the ffmpeg command to show mapping two streams to give a single rtmp output is shown below. ffmpeg -i video_stream -i audio_stream -map 0:0 -map 1:0 -f flv rtmp://my.red5.server:1935/live/mystream