createMediaElementSource plays but getByteFrequencyData returns all 0's - webrtc

I am attempting to visualize audio coming out of an element on a webpage. The source for that element is a WebRTC stream connecting to an Asterisk call via sip.js. The audio works as intended.
However, when I attempt to get the frequency data using web audio api, it returns an array of all 0's, even though the audio is working. This seems be a problem with createMediaElementSource. If I call getUserMedia and use createMediaStreamSource to connect my microphone to the input, I indeed get the frequency data returned.
This was attempted in both Chrome 40.0 and Firefox 31.4. In my search I found similar errors with Android Chrome but my versions of desktop Chrome and Firefox seem like they should be functioning correctly. So far I have a feeling that the error may be due to the audio player getting it's audio from another AudioContext in sip.js, or something having to do with CORS. All of the demos that I have tried work correctly, but only use createMediaStreamSource to get mic audio, or use createMediaElementSource to play a file (rather than streaming to an element).
My Code:
var context = new (window.AudioContext || window.webkitAudioContext)();
var analyser = context.createAnalyser();
analyser.fftSize = 64;
analyser.minDecibels = -90;
analyser.maxDecibels = -10;
analyser.smoothingTimeConstant = 0.85;
var frequencyData = new Uint8Array(analyser.frequencyBinCount);
var visualisation = $("#visualisation");
var barSpacingPercent = 100 / analyser.frequencyBinCount;
for (var i = 0; i < analyser.frequencyBinCount; i++) {
$("<div/>").css("left", i * barSpacingPercent + "%").appendTo(visualisation);
}
var bars = $("#visualisation > div");
function update() {
window.requestAnimationFrame(update);
analyser.getByteFrequencyData(frequencyData);
bars.each(function (index, bar) {
bar.style.height = frequencyData[index] + 'px';
console.debug(frequencyData[index]);
});
};
$("audio").bind('canplay', function() {
source = context.createMediaElementSource(this);
source.connect(analyser);
update();
});
Any help is greatly appreciated.

Chrome doesn't support WebAudio processing of RTCPeerConnection output streams (remote streams); see this question. Their bug is here.
Edit: they now support this in Chrome 50
See the test code for firefox about to land as part of this bug:
Bug 1081819. This bug will add webaudio input to RTCPeerConnections in Firefox; we've had working WebAudio processing of output MediaStreams for some time. The test code there tests both sides; note it depends a lot on the test framework, so just use it as a guide on hooking to webaudio.

Related

Strange offset with Watir/Capybara and PhantomJS

I want to detect the location of elements on a page using Watir and PhantomJS.
My second approach using Capybara resulted in the same offset.
While the elements on the left side look good, the right side is misaligned:
I made the screenshot before and after I grab the positions for each element with element.wd.location, but the offset is always the same. I used evaluate_script and .getBoundingClientRect() with Capybara.
One thing looks suspicious to me: The search input field is not loaded correctly and not only shows a misalignment, but also a different size and the magnifying glass isn't shown. I don't know if this causes the offset.
I tested it with pure PhantomJS 2.1.1 (phantomjs file.js):
var fs = require('fs');
var page = require('webpage').create();
page.viewportSize = {
width: 1024,
height: 768
};
page.open('http://en.wikipedia.org/', function() {
var positions = page.evaluate(function() {
positions = [];
elements = document.getElementsByTagName('IMG');
for (var i=0, l=elements.length; i<l; i++) {
pos = elements[i].getBoundingClientRect();
positions.push(pos.left + ' ' + pos.top);
};
return positions;
});
fs.write('test.txt', positions.join("\r\n"), 'w');
page.render('test.png');
phantom.exit();
});
Same result: If you open the test.png, you see the an image on the right (left: 952px, top: 259px), but the test.txt shows it shifted to the left (left: 891px).
Do you know what could cause this problem?
Do you know what could cause this offset?
A bug in PhantomJS v2.1.1 or in the embedded Qt WebEngine.
Is there any workaround ?
No.
But I want it to work anyway, how?
Fix it yourself or hire someone to fix it or wait for it to be fixed.
Note that the issue no longer occurs in version 2.5, but it is still in beta :
https://github.com/ariya/phantomjs/milestone/16
https://bitbucket.org/ariya/phantomjs/downloads/
Here's a screenshot taken with phantomjs-2.5.0-beta :
This seems to be an issue in PhantomJS.
On the GitHub thread of the issue, #dantarion seems to have found a solution:
I am running this as well.
My fix is to run the following on the page in an evaluate block to force PhantomJS to render at the right height viewport. It works for my use case, and while I want to see it fixed in 2.2, since its still an issue I thought I'd post here.
document.getElementsByTagName("body")[0].style.overflow = "hidden";
document.getElementsByTagName("body")[0].style.height = "1080px";
document.getElementsByTagName("body")[0].style.maxHeight = "1080px";
document.getElementsByTagName("html")[0].style.overflow = "hidden";
document.getElementsByTagName("html")[0].style.height = "1080px";
document.getElementsByTagName("html")[0].style.maxHeight = "1080px";
It seems to be solving the problem. The only problem being that background-size: cover might still be off (as reported by #Luke-SF).

How to use Google Translate TTS with the new V2 API?

I used to call Google Translate TTS to download an audio file using this url:
http://translate.google.com/translate_tts?tl=en&q=Hello+world!
However Google changed the way that works and therefore I can no longer download the audio files.
I've signed up for a free trial for Google Translate API V2, but can't find how to get the TTS audio files.
Any idea?
You can use that link without captcha..
https://translate.google.com/translate_tts?ie=UTF-8&tl=tr-TR&client=tw-ob&q=Batsın+bu+dünya+bitsin+bu+rüya
I stumbled across this thread and wanted to give my take on it, with reference to #Alexandre Andrade, mainly because he didn't submit any code.
I did this in a react app, but the same procedure should works for a vanilla web project.
I did add the meta tag to my head public/index.html,
<head>
...
<meta name="referrer" content="no-referrer">
...
</head>
Then added the audio tag in my component:
Javascript:
const playTTS = (text, lang) => {
// Get the audio element
const audioEl = document.getElementById('tts-audio');
const url= `https://translate.google.com/translate_tts?ie=UTF-8&tl=${lang}&client=tw-ob&q=${text}`;
// add the sound to the audio element
audioEl.src = url;
//For auto playing the sound
audioEl.play();
};
html
...
<audio controls id="tts-audio"/>
...
Then it's just a matter of hooking the function up to some of your life cycle methods. Since I wrote my react code in react hooks, I added the function call in one of my hooks to get it initialized when the component was loaded. (this would be in the componentDidMount() function otherwise).
Hope this helps anyone out!
try this link for English:
https://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&tl=en&q=Hello+World
For Chinese (Puthonghua)
https://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&tl=zh-CN&q=世界+你好
Text-to-speech was always an 'unofficial' API which is now captcha-protected to prevent abuse. It was never advertised as part of the Translate API, and currently there is no TTS functionality in the Translate V2 API, paid or otherwise.
There is some more background on the following groups thread which had been ongoing for some time.
Here's to those who have desperately been trying to play Google TTS as an audio in HTML: let me save you a couple of hours of time and tell you how to do it.
Let's say we have this link:
https://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&tl=en&q=I+love+coffee
If you try to play this audio given the link and using <audio>, <iframe>, using third-party libraries or playing it with Javascript...
var audio = new Audio('https://translate.google.com/translate_tts...');
audio.play();
...then you'll soon find out that none of the aforementioned ways work as Error 404 is being thrown.
Solution
Apparently, the only possible way to play this TTS generic audio is to utilise <embed> tag wrapped into a custom <iframe> and giving the link a unique version number (it is important, as caching by browsers prevents the audio from playing for some reason).
Here is the solution for our example: (assuming you have an iframe#ttsiframe)
function playTTS(lang,sentence) {
//get the iframe
var iFrame = document.getElementById('ttsiframe');
//remove its sandbox property
iFrame.removeAttribute('sandbox');
//this is your reference variable for the iframe body and head tag
var iFrameBody;
//get the body
if (iFrame.contentDocument) { // FF
iFrameBody = iFrame.contentDocument.getElementsByTagName('body')[0];
iFrameHead = iFrame.contentDocument.getElementsByTagName('head')[0];
}
else if (iFrame.contentWindow) { // IE
iFrameBody = iFrame.contentWindow.document.getElementsByTagName('body')[0];
iFrameHead = iFrame.contentWindow.document.getElementsByTagName('head')[0];
}
else {
iFrameBody = iFrame.contentDocument.body;
iFrameHead = iFrame.contentDocument.head;
}
//generate link to Google Translate TTS using arguments (pay attention to random version number at the end)
var link = 'https://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&tl=' + lang + '&q=' + sentence.replace(/ /g,'+').replace(/[.]/g,'') + '&rd=' + getRandomInt(0,50000000);
//add embed element with our link
iFrameBody.innerHTML = '<embed src="' + link + '" id="TTS">';
//isolate iframe
iFrame.setAttribute('sandbox','');
}
you can simply use the link:
Text to Speech

Playing different Sound files in Appcelerator Titanium makes the audio got noise

I have created an app using appcelerator for Iphone , which buy click on buttons it will play a relative sound , here is the code, but the problem is when i play the audio many times and play different audios using this function the sound starts to lag and have noise inside, can anybody help me with it , Thanks.
var soundplaying = 0;
var sound;
function playaudio(url) {
if (soundplaying == 0) {
sound = Ti.Media.createSound({});
sound.setUrl('../assets/audio/' + url);
sound.addEventListener('complete', function() {
sound.release();
soundplaying = 0;
});
sound.play();
soundplaying = 1;
}
}
(i have tried to release the sound object after each time but still no use, I tried to createSound only once but seems the titanium dose not support changing url for Media.Sound) dynamically.
I could have solve this issue temporary , by changing the audio files format to .m4a (aac).
i was using mp3 earlier.

Web Audio API. Play sound on click of button

I am a complete novice with HTML5 and coding for that matter. I have been trying to get to grips with the web audio API. I want a sound to play at a click of a button. I used a tutorial posted on HTML5Rocks, but cannot get it to work. I have tried to use jfiddle to help me troubleshoot, but to no avail.
here is my code:
http://jsfiddle.net/ue8WP/
Try this:
function playsound()
{
var filepath='sounds/'+....+'.mp3'; //example
var audio = new Audio();
audio.src = filepath;
audio.controls = true;
audio.autoplay = true;
}
Here's a simplified version of your code that should work fine. You only need to load some other sample sound, since there is a problem with fetching the one you provided (not allowed by Access-Control-Allow-Origin).
http://jsfiddle.net/WB6Pw/3/

Can Adobe AIR Desktop application take full screen snapshots (aka Print Screen button)

I would like to know if its possible to get full screen snapshots from an air application.
What i am interested in, is functionality similar to PrintScreen button in windows, which takes snapshots of all screens, including third party application windows, not just window in which air app is running.
If its not specific to air, and flash/flex API can provide such functionality, it also would be great.
Thanx a lot in advance.
Check out this article as it explains obtaining a screenshot by calling a native process:
import flash.filesystem.File;
import flash.events.NativeProcessExitEvent;
var process:NativeProcess;
if(NativeProcess.isSupported) {
var file:File = File.applicationDirectory;
var args:Vector.<String> = new Vector.<String>();
if (Capabilities.os.toLowerCase().indexOf("win") > -1) {
file = file.resolvePath("PATH/TO/WINDOWS/printscr");
//use your prefered screenshot tool here (e.g. https://code.google.com/p/screenshot-cmd/
//also setup the args as needed
} else if (Capabilities.os.toLowerCase().indexOf("mac") > -1) {
file = file.resolvePath("/usr/sbin/screencapture");
args[0] = "-i";
args[1] = "screencapture.png";
}
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
nativeProcessStartupInfo.arguments = args;
nativeProcessStartupInfo.executable = file;
nativeProcessStartupInfo.workingDirectory = File.desktopDirectory;
process = new NativeProcess();
process.start(nativeProcessStartupInfo);
process.addEventListener(NativeProcessExitEvent.EXIT,done);
}else trace("NativeProcess NOT SUPPORTED!");
function done(e:NativeProcessExitEvent):void{
trace("screenshot comprete");
}
One important thing to bear in mind is the AIR device profile.
If you're initially testing in ADL, be sure to use the extendedDesktop profile, otherwise NativeProcess.isSupported will return false.
For more details check out the NativeProcess documentation and the Communicating with native processes in AIR developer guide