I am able to stream video with Kurento using WebRTC, I need to implement multi party audio conference using MCU feature of Kurento Media server. So audio coming from all clients are merged and send back that combined audio to all clients in efficient manner using WebRTC.
if it will works then we need only two connection(one for send and one for receive) other wise we need peer connection to all clients using WebRTC. It is not feasible to establish peer connection to all all clients.
Please suggest me any sample code which have implemented MCU for audio using Kurento Media Server or guide me to implement same using Kurento Media Server.
I'm afraid there's no code that allows that un Kurento. There is the Composite media element, but that is usually for audio AND video. It combines streams into a single stream matrix of the required size, usually more than 9 streams may have performance problems. If you only want to process audio, surely it could handle much more than 9 streams. To use only audio just connect AUDIO stream to the HubPort.
EDIT 1
The code to generate the media elements needed, and the correct way establish an audio-only connection is as follows.
WebRtcEndpoint webrtc = new WebRtcEndpoint.Builder(pipeline).build();
Composite composite = new Composite.Builder(pipeline).build();
HubPort hubport = new HubPort.Builder(composite).build();
webrtc.connect(hubport, MediaType.AUDIO);
Please note that the connection is from the WebRtcEndpoint to the HubPort. If you need it to be bidirectional, you'll need to connect that way also.
hubport.connect(webrtc, MediaType.AUDIO);
Related
On AWS, how do you play video from MediaLive through the UDP output group?
For my use case, I'm building a live stream pipeline that takes an MPEG-2 transport stream from MediaLive, processes it through a UDP server (configured as an output group), and consumed by a web client that plays on HTML5 video.
The problem is: the data is flowing, but the video isn't rendering.
Previously, my output group was set to AWS MediaPackage, but because I need the ability to read and update frames over live stream, I'm trying to feed through UDP.
Is setting the output group to UDP the right approach?
The documentation is a bit sparse here. I'm wondering if there are resources or examples where others were able to play video this way as oppose to HLS/DASH.
Thanks for your post. Yes the UDP or RTP output would be the right choice of output from MediaLive. Appropriate routing rules will need to be used on any intermediary routers or firewalls to ensure that the UDP traffic can reach the client.
You wrote that 'the data is flowing, but the video isn't rendering.' This suggests an issue with the web client.
I suggest adding another identical UDP output to your UDP server and sending its output to a computer (or AWS Workspace) running a copy of VLC player. Decoding that new stream will give you a confidence monitor on the output of the entire workflow up to that point. This will help isolate the problem.
You could achieve the same result with a packet capture or TS stream analyzer if you want to go that route instead. If you go this route, I recommend trying to play back one of the packet captures locally with the web client.
My setup is as follows:
OBS Studio to create the video feed
Ant Media Server to distribute the stream
Now I'm building an app that will display this stream and I'm currently using ExoPlayer, however I'm having a hard time getting it to work for both RTMP and HLS, I read some where that I could embed a webplayer in my app would that be easier? Here is my code for ExoPlayer:
//RTMP Url
String url = "rtmp://192.168.1.244/WebRTCApp/379358104902020985845622";
BandwidthMeter bandwidthMeter = new DefaultBandwidthMeter();
TrackSelection.Factory videoTrackSelectionFactory =
new AdaptiveTrackSelection.Factory();
TrackSelector trackSelector =
new DefaultTrackSelector(videoTrackSelectionFactory);
SimpleExoPlayer player = ExoPlayerFactory.newSimpleInstance(this, trackSelector);
PlayerView playerView = findViewById(R.id.simple_player);
playerView.setPlayer(player);
/*
Create RTMP Data Source
*/
RtmpDataSourceFactory rtmpDataSourceFactory = new RtmpDataSourceFactory();
MediaSource videoSource = new ExtractorMediaSource.Factory(rtmpDataSourceFactory)
.createMediaSource(Uri.parse(url));
player.prepare(videoSource);
player.setPlayWhenReady(true);
Any help on this would be much appreciated.
Most online video streaming use Adaptive Bit Rate streaming (ABR) protocols to deliver the video, mainly HSL and DASH this days.
Most Media players, like ExoPlayer, support these protocols well, although they are complex and evolving protocols so there are always edge cases.
Many video conferencing applications use WebRTC which is a real time optimised protocol - the usual approach is to use a WebRTC client for this type of stream.
The difference between the two approaches from a streaming latency point of view, at a very high level, is:
ABR protocols prioritise quality and avoiding interruptions and buffer enough of the video to try to gaurantee uninterrupted playback. They are usually aimed at movie and live video streaming services. Even for low latency implementation the latency is measured in multiple seconds and more.
WebRTC prioritises latency and sacrifices quality if necessary. It is aimed typically at real time sensitive applications like video conferencing where it is important not to fall behind the discussion even if it means a temporary video glitch or even brief interruption in video. Latency is usually measured in sub seconds.
Any Media Server comes from the WebRTC side although recent versions support HLS /CMAF and Low Latency DASH (these are still higher latency than WebRTC generally as noted above).
For your service, if you are able to use a DASH or HLS stream you may find that it is an easier path with ExoPlayer. If you look at the demo app for example you will see DASH and HLS streams but no RTMP ones. You can easily extend or modify the demo app to play your own HLS or DASH stream and this is often an easy way to start - look at the sample material in the assets/media.exolist.json and add your own URL:
https://github.com/google/ExoPlayer/blob/aeb306a164911aa1491b46c2db4da0d329c83c65/docs/demo-application.md
However, ExoPlayer should also support RTMP via an extension if this is your preferred route - there is a specific extension that allows this:
https://github.com/google/ExoPlayer/blob/0ba317b1337eaa789f05dd6c5241246478a3d1e5/extensions/rtmp/README.md
In theory you simply need to add this dependency to your application:
if your application is using DefaultDataSource or DefaultDataSourceFactory, adding support for RTMP streams is as simple as adding a dependency to the RTMP extension
It would be worth checking the issues list in this repository for any recent issues and/or workarounds.
I'm trying to build a system for real-time analysis on server for video streamed from the client using WebRTC.
Here is what I currently have in mind. I would capture the webcam video stream from the client and send it (compressed using H.264?) to my server.
On my server, I would receive the stream and every raw frame to my C++ library for analysis.
The output of the analysis (box coordinates to draw) would then be sent back to the client via WebRTC or a separate WebSocket connection.
I've been looking online and found open-source media server like Kurento and Mediasoup but, since I only need to read the stream (no dispatch to other clients), do I really need to use an existing server? Or could I build it myself and if so, where to start?
I'm fairly new to the WebRTC and video streaming world in general so I was wondering, does this whole thing sound right to you?
That depends on how real-time your requirements are. If you want 30-60fps and near-realtime, getting the images to the server via RTP is the best solution. And then you'll need things like a jitter buffer, depacketization etc, video decoders, etc.
If you require only one image per second, grabbing it from the canvas and sending it via Websockets or HTTP POST is easier. https://webrtchacks.com/webrtc-cv-tensorflow/ shows how to do that in Python.
I am trying to create an application which requires a user to send his local video stream to multiple peers using webRTC. As far as I've seen I am responsible for managing several PeerConnection objects because a PeerConnection can only connect to a single peer at a time. What I want to know is if it is possible to create a connection and send my local stream to a peer without him sendig his local stream to me using webRTC.
Simply don't call peer.addStream for broadcast-viewers to make it oneway streaming!
You can disable audio/video media lines in the session description by setting OfferToReceiveAudio and OfferToReceiveVideo to false.
3-Way handshake isn't drafted by RTCWebb IETF WG, yet.
Because browser needs to take care of a lot stuff simultaneously like multi-tracks and multi-media lines; where each m-line should point out a unique peer.
I would like to know if it is possible for upnp/dlna protocol to stream audio from a single media server to multiple media renderers at the same time. Does the protocol allow this? Thank you.
Basically, it depends on what you mean with "at the same time". It is possible to access the same media by two different DMPs. However, each of them is creating a new stream, means twice the traffic. UPnP and thus DLNA as well is using HTTP as primary streaming protocol. RTP unicast and multicast streams are also considered but mostly not implemented by the manufacturers because it is not mandatory. So, multicast streams aren't well supported.
Additionally, some servers might implement some limitations when accessing a file or media, f.e. when you have a TV tuner or something else which provides exclusive access to a specific media (tv channel). Then, you can access this media only once at a time.