Is it possible with the Programmable Video API from Twilio to build something that resembles the Google Hangouts functionality in terms of how it focuses on the person talking automatically?
I don't see any examples or notes about this in their documentation and the github for this doesn't seem to be frequented that much.
Would appreciate any help, thank you!
Twilio developer evangelist here.
You can build that sort of thing, however it is currently out of scope for the Video SDK itself.
I haven't done this before, but I'd start by looking into analysing the audio coming from each participant in the chat. You can actually create audio sources from an existing <video> or <audio> element. In the case of Twilio Video, each track is created as a separate element, so you want to look for <audio> elements and use them:
var audioElements = document.querySelectorAll('audio');
audioElements.forEach(audio => {
var audioCtx = new AudioContext();
var source = audioCtx.createMediaElementSource(audio);
// create audio analyser, analyse volume in audio
})
You want to use the Web Audio API to then analyse all the remote tracks and work out which is currently making the most noise for a sustained period of time and switch to that one. This blog post may help with the analysis. I've not seen anything that will help with the selection, but hopefully you can work it out from there.
Let me know if this helps at all.
Related
Is it possible to create an audio visualizer with Agora WebSDK-NG? I'm looking for something similar to :
https://webrtc.github.io/samples/src/content/getusermedia/volume/
or
https://www.cssscript.com/audio-visualizer-with-html5-audio-element/
Thanks for any suggestion.
Yes this is technically feasible using Agora's NG WebSDK because the SDK is built using WebRTC.
If you are looking to add this to the local user's interface, look into the documentation for local-audio-tracks specifically you will want to create an audio track locally to be able to pass it to the visualizer.
Or if you want to visualize the audio from a remote stream you can use the user (AgoraRTCRemoteUser) and call user.audioTrack to get the audio track.
#Hermes's answer is correct. If you're looking for some template code to experiment with, I'd recommend starting with the basic live demo or looking at any of the other demos. Either way, what's important is that you need to create a local or remote audio track, then once you have the track you can create an MediaStream object and add the track to it as follows:
const audioStream = new MediaStream(); // Web Audio Api
audioStream.addTrack(remoteAudioTrack._mediaStreamTrack); // remote or local
var mediaSource = audioContext.createMediaStreamSource(audioStream); // don't forget to setup an audio context
const analyser = audioContext.createAnalyser();
mediaSource.connect(analyser);
There's more to do here but this should help you get started. If you're unfamiliar with the Web Audio API, I'd recommend you start with this Video. The MDN web docs also provides all the information you need and some demos.
Good luck
I am using AgoraWebSDK-NG to do a Web implementation.
My application uses interactive audio streaming. Video is not required.
I want to know how to retrieve all users who are part of the "audience" of a channel (and not "hosts"). I want to display such "audience" users in the UI.
Unfortunately I cannot find any method to be able to do so here https://agoraio-community.github.io/AgoraWebSDK-NG/api/en/interfaces/iagorartcclient.html#getlisteners
Please let me know how this would be possible. An example would be great. If it is not possible, then just let me know that as well. Thank you.
this is not possible at the moment with the NG SDK. To achieve this right now, you can use the Agora RTM SDK.
You can create an RTM channel with the same name as the RTC channel. Then you can use the RTM's channel events to know when an audience member joins or leaves.
I need to find out if it is possible to check if there are any sound cloud players playing (or if there are none playing) on a page at any given point.
The sound cloud api seems to allow you to check if a player is paused but i'm not sure if this apply's to all players on a page or not?
Any thoughts or pointers would be much appreciated.
Not an expert on this but looking at the embedded widget page, they are using an iFrame, meaning it's running in an isolated environment. Unless they go out of their way to provide this feature (using cross-document messaging for example), it won't be possible.
(Well, if you want to go for a really weird hack, you could request the user's microphone permissions and detect the audio playing :)
You could just make a separate SC.widget object for each player on the page. You could create a script to find them if all the iframes on the page are soundcloud players like this:
var widgets=new Array();
$('iframe').each(function (index, iframe) {
widgets.push(SC.Widget(iframe));
});
Then you could check if all of them are paused.
I am currently creating a youtube downloader and I have the code to retrieve video thumbnails, however I want to find out a way to retrieve the video title and possibly the video size etc. If anyone can help please answer :D
I would concider you use version 3 of the Youtube API. I don't suggest parsing the html of a youtube page, you never know when they will change the format and you'll be downloading too much information. The API is much more stable.
If you look at the video section .
https://www.googleapis.com/youtube/v3/videos?id=BaW_jenozKc&part=snippet
(I can't test it on this computer)
By changing the part (or adding multiple part) you can get all the information you need about a video. Since the result is returned in json, you can use it with pretty much any programming language.
Also, read the Gettting Started. Youtube limits the number of query unless you register your application and get a developper key.
You need to read the .NET YouTube API documentation, specifically the YouTube API v2.0 – Retrieving Data for a Single Video section.
You will need to query for a single video by the video's ID and then the pieces of data in the ATOM feed that comes back to you are:
feed/entry/Title for the title of the video
feed/entry/media:group/yt:duration for the length of the video
There does not appear to be a value returned for the size of the video, however.
Try the youtube API:-
https://www.youtube.com/get_video_info?video_id=BaW_jenozKc
It should return something along the lines of:-
"&title=youtube-dl+test+video+%22%27%2F%5C%C3%A4%E2%86%AD%F0%9D%95%90".
Credits to youtube-dl:-
https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/youtube.py
Does anyone know if it's possible to use the Vimeo API to return videos based on where in the world the video was upload from or alternatively where in the world the video has been liked from? Does Vimeo store any location specific information? Any help on this would be awesome!
Don't think so. Vimeo currently doesn't expose any geolocation data via their api. Can't say whether or not they're tracking it internally on their backend.
They did just launch a new help page:
https://vimeo.com/help/contact
Might be worth messaging through that site to confirm.