How to set viewport height auto in CasperJS - phantomjs

I would like to capture screenshot of multiple web page, each having varying heights. How can I set height dynamically based on the page height?
Something like this:
casper.viewport(1200, 'auto');

By default PhantomjS browser (which is used by CasperJS) renders full height of the page automatically. In CasperJS the corresponding method is
casper.capture()
To set the horizontal width of the page, set viewport size:
var casper = require('casper').create({
viewportSize: {width: 1280, height: 800}
});
This setting will only affect the width of the screenshot, the height will be as big as necessary to render the whole page.
Example:
var casper = require('casper').create({
verbose: true,
logLevel: 'debug',
pageSettings: {
userAgent: 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36'
}
});
casper.options.viewportSize = {width: 1280, height: 1024};
casper.start("http://stackoverflow.com/questions/tagged/phantomjs");
casper.then(function() {
casper.capture("stackoverflow.png");
});
casper.run();
The resulting image (439 kb) is 3448 pixels high which is way more than 1024 pixels set in options.

Related

Twitch thumbnail not visible in webview

I am working on a React Native app that displays HTML content using the react-native-render-html library. This HTML data is coming from content formatted in WordPress. The same content is also being displayed in the web app made with ReactJS. I am also using this iframe plugin, as recommended by the render-html library, to display the iframes in my HTML.
I have the following problem with rendering Twitch video embeds.
The Twitch embeds are in this format:
<iframe src="https://clips.twitch.tv/embed?clip=ShortHelpfulSquirrelCmonBruh-hWj49qxBfx-VKseO&autoplay=false&parent=my.parent.domain" width="640px" height="360px" frameborder="0" scrolling="no" allowfullscreen="true"></iframe>
Please note that these are not live streams, but clips. The ReactJS website displays these embeds correctly with the thumbnail:
With the same embed code, however, my React Native app displays the embed without the thumbnail but with just a black background (although it shows the title and controls):
Once I play and pause the video, the background does not remain black, however:
Here is the rendererProp that I am passing to the library for iframes:
iframe: {
scalesPageToFit: true,
webViewProps: {
scrollEnabled: false,
mediaPlaybackRequiresUserAction: true,
javaScriptEnabled: true,
domStorageEnabled: true,
userAgent: Platform.select({
ios: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15",
android: "Chrome/56.0.0.0 Mobile android",
}),
},
}
I am passing the IframeRenderer and iframeModel as imported directly from #native-html/iframe-plugin (linked at the beginning) into the renderers and customHTMLElementModels props of the <RenderHtml/> component. Also I am using the WebView from react-native-webview lib.
My team and I cannot figure out why the thumbnail is not appearing in the react-native-webview, but works fine in the website. If anyone has any experience rendering Twitch video iframe embeds on react-native-webview, please take a look. Any help will be appreciated.

Splash is not loading javascript properly for crawling a website (scrapy-splash)

I am having difficulty debugging why Splash is unable to load the javascript properly for this website: https://www.bigc.co.th/2000003703081.html
I have referred to this answer to disable Splash from running in private mode and also other solutions here in the docs, such as increasing wait time and setting user agent.
However, the website is still loading as though without JS on Splash, as seen in this picture below
I am currently debugging why the website is not loading properly for the purpose of incorporating this with scrapy-splash code.
Here is my current script. Thank you for your time and assistance in advance!
function main(splash, args)
splash:set_user_agent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36')
splash.private_mode_enabled = false
splash:set_viewport_full()
splash:go{url=args.url}
assert(splash:wait(1))
return {
html = splash:html(),
png = splash:png(),
har = splash:har(),
}
end

casperjs screenshot returning blank square boxes. Please tell me possible solution to get the character instead of boxes

I am using CasperJS 1.1.0 with PhantomJs 2.0. I am using Linux server.
I have a script, trying to run and take screenshots.
This screenshot is fine for local server nor for live.
Screenshot of the site will look like this:
https://i.stack.imgur.com/dJe4L.png
So, instead of the actual letters, it renders square boxes.
var casper = require('casper').create({
verbose: true,
});
casper.options.waitTimeout = 40000;
casper.options.viewportSize = {width: 1200, height: 768};
phantom.cookiesEnabled = true;
casper.start('http://domain.com/', function() {
this.wait(5000, function () {
this.echo(this.getTitle());
this.capture("screenshot.png");
});
});
casper.run(function () {
this.exit(0);
});
Please tell me where i am wrong.
I would use waitForSelector on the items that you're wanting screenshots of, rather than the 5 second hard wait. Consider adding
verbose: true
waitTimeout:20000
Which will give you 20 second wait limit, longer than the 5 seconds default. Then, instead of your current wait, use something like:
this.waitForSelector('img', function() {
this.capture('test.png')
}
You can change the img selector to be something more specific, but waiting for the images themselves to load is a pretty certain way of making sure they will be captured. The reason we changed the timeout to 20 seconds is in case their site takes a little longer to load their images :)
Let me know if this doesn't help, happy to follow up. I will need examples of the site and the image you'd like though.
EDIT: Can confirm this works for me, let me know if it works for you.
var casper = require('casper').create({
verbose: true,
logLevel: 'error',
pageSettings: {
loadImages: true,
loadPlugins: true,
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11'
},
clientScripts: ['vendor/jquery.min.js', 'vendor/lodash.js'],
viewportSize: {
width: 1600,
height:1000
},
});
casper.start('http://dev.bunkerbradleycouture.com/')
casper.then(function() {
this.waitForSelector('img', function() {
this.capture('test.png')
})
})
casper.run()

inconsistent SVG elements from Highcharts making tests fail with Geb/Selenium and PhantomJS

Having trouble explaining this:
My Geb/Selenium tests are making assertions on SVG rect element width/height generated by Highcharts.
The tests pass locally on Chrome/Windows, and pass on PhantomJS on a Linux CI environment. They fail when I run them on PhantomJS locally on Windows, though. The height seems to be a few pixels off - consistently, it is about 1 or 2 pixels different when I run locally on PhantomJS in Windows. For example, my assertion that rect.height=168 fails because it's 166.
I have confirmed that it is the same version of PhantomJS in both environments.
What else could I be missing? And what could cause Highcharts to generate different SVG in Chrome and PhantomJS?
I can isolate this from the test framework and execute with PhantomJS directly:
var webpage = require('webpage');
var page = webpage.create();
page.viewportSize = {width: 1280, height: 1024};
page.open("http://www.highcharts.com/demo/column-basic", function(status) {
var height = page.evaluate(function() {
return jQuery('g.highcharts-series rect:nth(0)').attr('height');
});
console.log(height);
phantom.exit();
});
This code will print 52 when run on Linux and 53 when Windows.
It depends on what you're going to test, but this could be a solution for adapting to rounding issues. Detect the underlying OS and setup a configuration object with dpi corrections per OS.
var webpage = require('webpage'),
page = webpage.create(),
system = require('system'),
os = system.os,
correction = {
mac: 1,
windows: 1.019
};
page.viewportSize = {
width: 1280,
height: 1024
};
page.open("http://www.highcharts.com/demo/column-basic", function(status) {
var height = page.evaluate(function() {
return jQuery('g.highcharts-series rect:nth(0)').attr('height');
});
console.log(Math.round(height * correction[os.name]));
phantom.exit();
});

CasperJS/PhantomJS unable to open Facebook

I've seen examples such as this one showing how to login to facebook with casperJS:
How to login into a website with CasperJS?
but am unable to get this code to work for me. I'm not interested in the login portion, I just want to load any facebook page into casperjs or phantomjs but keep getting a fail on load.
Is this working for anyone else? or has facebook detected the browser and not allowing access anymore?
Here is a simplified version of what I am unable to do:
var casper = require('casper').create({
verbose: true,
logLevel: 'debug',
pageSettings:{
userAgent :'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4' ,
loadImages: false, // The WebPage instance used by Casper will
loadPlugins: false // use these settings
}
});
casper.start()
casper.thenOpen('https://www.facebook.com/pfchangs', function() {
this.echo(this.getHTML());
//this just prints out empty page: <html><head></head><body></body></html>
});
casper.run();
I'm running this on Windows 7
Looks like this is known bug with casperjs windows batch file:
https://github.com/n1k0/casperjs/commit/0d659f140f1e2120bed967d8301657b5fe79f19c