I have been trying to alter my code from image thumbnail gallery to a video thumbnail gallery. I have the videos encoded in the various formats for html 5...It works in Firefox and Safari. In chrome it works one time for each thumbnail, then doesnt load the video if you click it again. Is there a better way to do this? The html is like <a href = "#" rel ="videos/video1" class = "image"><a href = "#" rel ="videos/video2" class = "image">
$(function() {
$(".image").live('click',function() {
var image = $(this).attr("rel");
var title = $(this).attr("alt");
$('#largevideo').hide();
$('#largevideo').fadeIn(1500);
$('#largevideo').html('<video controls><source src="'+image+'.mp4" type="video/mp4"/><source src="'+image+'.webm" type="video/webm"/><source src="'+image+'.ogv" type="video/ogg"/><embed src=".'+image+'.mp4" type="application/x-shockwave-flash" autoplay="0"allowfullscreen="true" allowScriptAccess="always"></embed></video>');
return false;
});
(location.attr)? $("a [rel="+location.attr+"]").click():$(".thumbs a:first").click();
});
Related
I am creating a video scraper (for the Rumble website) and I am trying to get the src attribute of the video using HTMLUnit, this is because the element is added dynamically to the page (I am a beginner to these APIs):
val webClient = WebClient()
webClient.options.isThrowExceptionOnFailingStatusCode = false
webClient.options.isThrowExceptionOnScriptError = false
webClient.options.isJavaScriptEnabled = true
val myPage: HtmlPage? = webClient.getPage("https://rumble.com/v1m9oki-our-first-automatic-afk-farms-locals-minecraft-server-smp-ep3-live-stream.html")
Thread.sleep(10000)
val document: Document = Jsoup.parse(myPage!!.asXml())
println(document)
The issue is, the output for the <video> element is the following:
<video muted playsinline="" hidefocus="hidefocus" style="width:100% !important;height:100% !important;display:block" preload="metadata"></video>
Whereas -- if you navigate to the page itself and let the JS load -- it should be:
<video muted="" playsinline="" hidefocus="hidefocus" style="width:100% !important;height:100% !important;display:block" preload="metadata" poster="https://sp.rmbl.ws/s8/1/I/6/v/1/I6v1f.OvCc-small-Our-First-Automatic-AFK-Far.jpg" src="blob:https://rumble.com/91372f42-30cf-46b3-8850-805ee634e2e8"></video>
Some attributes are missing, which are crucial for my scraper to work. I need the src value so that ExoPlayer can play the video.
I am not totally sure, but I was wondering whether it had to do with the fact that the crossOrigin attribute is anonymous in the JavaScript:
<video muted playsinline hidefocus="hidefocus" style="width:100% !important;height:100% !important;display:block" preload="'+t+'"'+(a.vars.opts.cc?' crossorigin="anonymous"':"")+'>
I tried to play around with the different HTMLUnit options, as well as look online but I still haven't been able to extract the right attributes I need so that it can work.
How would I be able to bypass this and get the appropriate element values (src) that I need for the scraper using HTMLUnit? Is this even possible to do with HTMLUnit? I was also suspecting that maybe the site owners added this cross origin anonymous statement because it can bypass scrapers, though I am not sure.
How to reproduce my issue
Navigate to this link with a GUI browser.
Press 'Inspect Element' until you find the <video> HTML tag and observe that it contains an src attribute as you would expect to the mp4 file:
<video muted="" playsinline="" hidefocus="hidefocus" style="width:100% !important;height:100% !important;display:block" preload="metadata" src="https://sp.rmbl.ws/s8/2/I/6/v/1/I6v1f.caa.rec.mp4?u=3&b=0" poster="https://sp.rmbl.ws/s8/1/I/6/v/1/I6v1f.OvCc-small-Our-First-Automatic-AFK-Far.jpg"></video>
Now, let's simulate this with a headless browser, so add the following code to IntelliJ or any IDE (add a dependency to HTMLUnit and JSoup):
To gradle (Kotlin):
implementation(group = "net.sourceforge.htmlunit", name = "htmlunit", version = "2.64.0")
implementation("org.jsoup:jsoup:1.15.3")
To gradle (Groovy):
implementation group = 'net.sourceforge.htmlunit', name = 'htmlunit', version = '2.64.0'
implementation 'org.jsoup:jsoup:1.15.3'
Then in Main function:
val webClient = WebClient()
webClient.options.isThrowExceptionOnFailingStatusCode = false
webClient.options.isThrowExceptionOnScriptError = false
webClient.options.isJavaScriptEnabled = true
val myPage: HtmlPage? = webClient.getPage("https://rumble.com/v1m9oki-our-first-automatic-afk-farms-locals-minecraft-server-smp-ep3-live-stream.html")
Thread.sleep(10000)
val document: Document = Jsoup.parse(myPage!!.asXml())
println(".....................")
println(document.getElementsByTag("video").first())
If it throws an exception add this:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.ActiveXObject").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.html.HTMLDocument").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.html.HtmlScript").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit.javascript.host.WindowProxy").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache").setLevel(Level.OFF);
We are simply fetching the page with the headless browser and then using JSoup to parse the HTML output and finding the first video element.
Observe that the output does not contain any 'src' attribute as you saw in the GUI browser:
<video muted playsinline="" hidefocus="hidefocus" style="width:100% !important;height:100% !important;display:block" preload="metadata"></video>
Screenshot of how your output should look like in the console:
This is the major issue I am having, the src attribute of the <video> element is seemingly disappeared in the headless browser, and I am unsure why although I suspect it's related to some sort of mp4 codec issue.
Correct, the js support for the video element was not sufficient for this case.
Have done a bunch of fixes/improvements and the upcoming version 2.66.0 will be able to support this.
Btw: there is no need to parse the page a second time using jsoup - HtmlUnit has all the methods to deeply look inside the dom tree of the current page.
String url = "https://rumble.com/v1m9oki-our-first-automatic-afk-farms-locals-minecraft-server-smp-ep3-live-stream.html";
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
webClient.getOptions().setThrowExceptionOnScriptError(false);
HtmlPage page = webClient.getPage(url);
webClient.waitForBackgroundJavaScript(10_000);
HtmlVideo video = (HtmlVideo) page.getElementsByTagName("video").get(0);
System.out.println(video.getSrc());
}
This code prints https://sp.rmbl.ws/s8/2/I/6/v/1/I6v1f.caa.rec.mp4?u=3&b=0 - the same as the source attribute in the browser.
But there are still two js errors reported when running this code. This is because some other js (i guess some tracking staff) provokes this errors. You can fix this by ignoring the js code for this two locations, this will make the code a bit faster also.
String url = "https://rumble.com/v1m9oki-our-first-automatic-afk-farms-locals-minecraft-server-smp-ep3-live-stream.html";
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
webClient.getOptions().setThrowExceptionOnScriptError(false);
// ignore some js
new WebConnectionWrapper(webClient) {
public WebResponse getResponse(WebRequest request) throws IOException {
WebResponse response = super.getResponse(request);
if (request.getUrl().toExternalForm().contains("sovrn_standalone_beacon.js")
|| request.getUrl().toExternalForm().contains("r2.js")) {
WebResponseData data = new WebResponseData("".getBytes(response.getContentCharset()),
response.getStatusCode(), response.getStatusMessage(), response.getResponseHeaders());
response = new WebResponse(data, request, response.getLoadTime());
}
return response;
}
};
HtmlPage page = webClient.getPage(url);
webClient.waitForBackgroundJavaScript(10_000);
HtmlVideo video = (HtmlVideo) page.getElementsByTagName("video").get(0);
System.out.println(video.getSrc());
Thanks for this report - will inform on https://twitter.com/htmlunit about the new release.
I have adopted various approaches to embed PDF blob in html in IE in order to display it.
1) creating a object URL and passing it to the embed or iframe tag. This works fine in Chrome but not in IE.
</head>
<body>
<input type="file" onchange="previewFile()">
<iframe id="test_iframe" style="width:100%;height:500px;"></iframe>
<script>
function previewFile() {
var file = document.querySelector('input[type=file]').files[0];
var downloadUrl = URL.createObjectURL(file);
console.log(downloadUrl);
var element = document.getElementById('test_iframe');
element.setAttribute('src',downloadUrl);
}
</script>
</body>
2) I have also tried wrapping the URL Blob inside a encodeURIcomponent()
Any pointers on how I can approach to solve this?
IE doesn't support iframe with data url as src attribute. You could check it in caniuse. It shows that the support is limited to images and linked resources like CSS or JS in IE. Please also check this documentation:
Data URIs are supported only for the following elements and/or
attributes.
object (images only)
img
input type=image
link
CSS declarations that accept a URL, such as background, backgroundImage, and so on.
Besides, IE doesn't have PDF viewer embeded, so you can't display PDFs directly in IE 11. You can only use msSaveOrOpenBlob to handle blobs in IE, then choose to open or save the PDF file:
if(window.navigator.msSaveOrOpenBlob) {
//IE11
window.navigator.msSaveOrOpenBlob(blobData, fileName);
}
else{
//Other browsers
window.URL.createObjectURL(blobData);
...
}
I want to render a webpage from a string. I've looked at the docs of phantomjs and they suggested the following:
var webPage = require('webpage');
var page = webPage.create();
var expectedContent = '<html><body><div>Test div</div></body></html>';
var expectedLocation = 'http://www.phantomjs.org/';
page.setContent(expectedContent, expectedLocation);
It's not quite working. Why? (I use the latest version).
I suggest you render a normal page (about:blank works) and then do webPage.content='<html><body><div>Test div</div></body></html>';
then render your page.
hope that helps.
We have implemented google +1 buttons on our site and they have served reliably for some time. However we recently noticed that the buttons are not serving reliably. We rarely see them appear in their designated spaces.
For example on this page: Sample Page : you'll see a gray box of social buttons to left of the page. In it, there is SUPPOSED to be a Google +1 button.
We've requested the button with this code:
<div id="social-google" class="social">
<script type="text/javascript">
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
</script>
<g:plusone size="medium"></g:plusone>
</div>
We've also tried this code:
<div id="social-google" class="social">
<!-- Place this tag where you want the share button to render. -->
<div class="g-plus" data-action="share" data-size="small" data-annotation="bubble"></div>
<!-- Place this tag after the last share tag. -->
<script type="text/javascript">
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
</script>
</div>
Occasionally we'll see a Google +1 button render but, more often than not, the space reserved for the button is apparently blank. When you examine things with firebug, you see that Google HAS attempted to render a button, but for some reason it has placed the button far above and left of the page boundaries.
Here is the top of the html Google generates for the button:
<div id="___plusone_0" style="position: absolute; width: 450px; left: -10000px;">
<iframe id="I0_1377554650466" width="100%" scrolling="no" frameborder="0" hspace="0 marginheight="0" marginwidth="0" style="position:absolute;top:-10000px;width:450px;margin:0px;border-style:none" tabindex="0" vspace="0" name="I0_1377554650466" src="https://apis.google.com/_/+1/fastbutton?bsv=o&usegapi=1&size=medium&hl=en-US&origin=http%3A%2F%2Fwww.comicbookresources.com&url=http%3A%2F%2Fwww.comicbookresources.com%2F%3Fpage%3Darticle%26id%3D47537&gsrc=3p&ic=1&jsh=m%3B%2F_%2Fscs%2Fapps- ...
As you can see Google gave its generated ___plusone_0 div a left position of -10000px and gave the inner iFrame a top position of -10000px. So the button is there. It's just floating out in space. If I manipulate theses position settings (to 0px) the button becomes visible in its appropriate spot.
Any idea why this would happen? Any idea how we can fix this?
You can try adding the following CSS declaration to your stylesheet:
#___plusone_0, #___plusone_0 iframe {
position:static !important;
}
This is a hackaround, so don't depend on it in long term.
Based on an old thread in Drupal Issues.
During the last few days I'm suffering from this problem too. I have a page building app. One of the widgets is google plus: users can enter a url, and the app generates a button. (So there can be more, than 1 button on the page.) Then user saves the page and can see it on Facebook.
Recommendations and observations...
Double check the protocol of google api script. For example, if your website is on https and you are trying to load http://apis.google.com/js/plusone.js, your buttons will probably fail to render.
When I tested this issue on my server, I occasionally opened the app in 2 browser tabs at the same time. Google buttons didn't appear in the first tab, but they did in the second one!
My app requires user to be authorized on Facebook. When I opened the app without authorization, the buttons were shown as expected. But when I logged in and refreshed the page - buttons disappeared.
When I opened the page on Facebook, buttons didn't appear, regardless of whether I was logged in or not.
I beg your pardon, if you think these notices have no sense, but they may save someone's time in future.
Workaround
Suppose, you're parsing the following code:
<!-- google button will be added into this div -->
<div class="googlePlus" data-href="http://google.com"></div>
jQuery function, which parse all .googlePlus divs.
$('.googlePlus').each(function () {
var $googleDiv = $(this);
// check, if button is already parsed
if (!$googleDiv.children().length) {
// add temporary id to the parent div
var $id = 'googlePlus-' + new Date().getTime();
$div.attr({
'id': $id
});
// create, add and render btn (IE compatible method)
var gPlusOne = document.createElement('g:plusone');
gPlusOne.setAttribute('href', $googleDiv.attr('data-href'));
document.getElementById($id).appendChild(gPlusOne);
gapi.plusone.go($id);
// function, correcting css styles
if (!$.isFunction($.fn.fixGooglePlus)) {
$.fn.fixGooglePlus = function () {
$(this).children('div').children('iframe').addBack().css({
position: 'static',
width: 106,
height: 24
});
}
}
// run function, until css is fixed
var $timer = setInterval(function () {
$googleDiv.fixGooglePlus();
if ($googleDiv.find('iframe').css('position') == 'static') {
clearInterval($timer);
$googleDiv.removeAttr('id');
}
}, 100);
} // button hasn't been parsed
});
Put the button code in a a new HTML file and put that file in an iframe. Compared to #U-D13's answer, it's less susceptible to changes by Google.
For example, I am trying to iframe the youtube subscription box on the homepage, and the problem is, if I make the iframe really long, then it wastes space, but if I make the size I want, then if the user clicks the "load more videos" button, then it gets cut off. So is there any way to make the iframe (or any alternatives) be a percentage of the size, or dynamically change when the page changes?
Create a file and call it iframe.html
<html>
<head>
<script type="text/javascript"></span>
function autoIframe(frameId){
try{
frame = document.getElementById(frameId);
innerDoc = (frame.contentDocument) ? frame.contentDocument : frame.contentWindow.document;
if (innerDoc == null){
// Google Chrome
frame.height = document.all[frameId].clientHeight + document.all[frameId].offsetHeight + document.all[frameId].offsetTop;
}
else{
objToResize = (frame.style) ? frame.style : frame;
objToResize.height = innerDoc.body.scrollHeight + 18;
}
}
catch(err){
alert('Err: ' + err.message);
window.status = err.message;
}
}
</script>
</head>
<body>
<iframe id="tree" name="tree" src="tree.html" onload="if (window.parent && window.parent.autoIframe) {window.parent.autoIframe('tree');}"></iframe>
</body>
</html>
Now create an html page called tree.html and put some dummy content in it.Make sure that the iframe.html and the tree.html are in the same directory. Open the .html files in browser and you will observe the o/p.
Some more useful links :
How to detect iframe resize?
How to detect iframe iframe resize