Just crawling a web page like github.com I hit the famous TypeError: 'undefined' is not a function.
var page = require('webpage').create();
page.open('https://github.com', function() {
phantom.exit();
});
See https://groups.google.com/forum/#!msg/phantomjs/r0hPOmnCUpc/uxusqsl2LNoJ
So exactly how do you fix this problem for someone totally new to PhantomJS?
Do I git clone curjojs/poly and then wrap all my PhantomJS code under poly/function like https://stackoverflow.com/a/19521355/230884?
I don't see any official documentation on solving this particular issue.
Related
I have this website https://www.storytel.com/sg/en/where I am trying to click on the button (refer to the image) in the Subscription Component which resides in the Shadowroot. I have tried with the following codes but it didn't work. It will be great if someone can help. Thanks in advance.
test('Click inside shadowDOM', async t => {
const shadowBtn = Selector(() => document.querySelector('storytel-subscription').shadowRoot.querySelectorAll('*[data-testid="subscription-card-0-button"]'));
await t
.click(shadowBtn);
});
There seems to be a bug with processing elements in shadow dom. I suggest you create an issue in the TestCafe GitHub repository and describe your scenario there: https://github.com/DevExpress/testcafe/issues/new?template=bug-report.md
I am new to Casperjs, phantomjs .I have been trying hard to create some page automation to login and take some steps in a CMS but i am having issues with accessing the global window variables from casperjs evaluate() function. the below example is just checking jquery on Google. Jquery exists in the page and some other global functions but i can't access them from casperjs.
casper.start('https://www.google.ca/#hl=en', function() {
// search for 'casperjs' from google form
this.fill('form[action="/search"]', { q: 'casperjs' }, false);
});
casper.then(function() {
this.evaluate(function jquery() {
console.log('looking for jquery ---');
console.log($ + 'exists');
});
});
getting error - `ReferenceError: Can't find variable: $
How can i fix this ?
Any help is appreciated :)
For use jQuery in casperjs
inject script in page, something like:
var casper = require('casper').create({
some code here,
clientScripts: ['/path/to/jquery.js'],
});
We are using the superb WebdriverJS (with Selenium) to perform acceptance testing on our web app. Everything works fine, and our tests execute successfully when we use Firefox and Safari.
However, when we use PhantomJS, our tests fail with unhelpful errors. It's almost as if... Javascript isn't even running inside the client page! Something that would cause this would be if PhantomJS' javascript environment ran into errors. Unfortunately, I can't seem to find a way to access Javascript errors when using PhantomJS with WebdriverJS.
If we were using PhantomJS directly, we could simply do (from the PhantomJS site):
page.onError = function(msg, trace) {
console.log(msg);
trace.forEach(function(item) {
console.log(' ', item.file, ':', item.line);
});
}
Unfortunately, I don't know how to access this mysterious page object when using PhantomJS within WebdriverJS. Any thoughts?
You can actually access JS errors in your PhantomJS stdout log at INFO level.
$ phantomjs --webdriver 4444 --webdriver-loglevel=INFO
You can even push things forward by setting the log level to DEBUG and see what actually PhantomJS does to execute the commands you send through Webdriver / Ghostdriver.
I figured out a workable solution! Essentially, it involves using an onerror event handler to intercept (and store) the Javascript errors. Then, once the DOM is ready, we report the errors via hidden DOM elements. This allows Selenium to look for specific elements (e.g. ".javascript-errors"), which is something it's naturally quite good at. Thanks go to myriad other blog posts and SO questions for getting me to this point.
The code:
//For detecting and reporting Javascript errors via Selenium. Note that this should be in its own file to allow this code to reliably detect syntax errors in other files.
var errors = [];
//Handle all errors
window.onerror = function(message, url, line) {
errors.push({"message":message, "url":url, "line":line});
}
//Report errors visually via HTML once the DOM is ready
window.onload = function() {
if(errors.length==0)
return;
var div = document.createElement("div");
div.className = 'javascript-errors';
div.innerHTML = '';
var style = "position:absolute; left:-10000px; top:auto; width:1px; height:1px;"; //CSS to hide the errors; we can't use display:none, or Selenium won't be able to read the error messages. Adapted from http://webaim.org/techniques/css/invisiblecontent/
for(var i=0; i<errors.length; i++)
div.innerHTML += '<div class="javascript-error" style="' + style +'"><span class="message">' + errors[i].message.replace('<', '<').replace('>', '>') + '</span><br/><span class="url">' + errors[i].url + '</span><br/><span class="line">' + errors[i].line + '</span></div>';
document.body.appendChild(div);
}
I'm trying to automate an application that uses form security in order to upload a file and then scrape data from the returned HTML.
I started out using the solution from this question. I can define my steps and get through the entire workflow as long as the last step is rendering the page.
Here are the two steps that are the meat of my script:
function() {
page.open("https://remotesite.com/do/something", function(status) {
if ('success' === status) {
page.uploadFile('input[name=file]', 'x.csv');
page.evaluate(function() {
// assignButton is used to associate modules with an account
document.getElementById("assignButton").click();
});
}
});
},
function() {
page.render('upload-results.png');
page.evaluate(function() {
var results = document.getElementById("moduleProcessingReport");
console.log("results: " + results);
});
},
When I run the script, I see that the output render is correct. However, the evaluate part isn't working. I can confirm that my DOM selection is correct by running it in the Javascript console while on the remote site.
I have seen other questions, but they revolve around using setTimeout. Unfortunately, the step strategy from the original approach already has a timeout.
UPDATE
I tried a slightly different approach, using this post and got similar results. I believe that document uses an older PhantomJS API, so I used the 'onLoadFinished' event to drive between steps.
i recomend you use casperjs or if you use PJS's webPage.injectScript() you could load up jquery and then your own script to do form input/navigation.
Probably I'm doing something wrong but:
In my browser I have a code
window.location = '/some/url'
or element with href="/some url"
in node (express.js) I have following route handler:
app.get('/some/url', function(req, res){
res.redirect('http://www.google.com')
});
Content of the page is loaded with strange flaw Google page. And address bar is not changed (http://localhost:3000/some/url).
I'm making request via intermediate server using
req.pipe(request(host + req.url)).pipe(res) and it seems that piped res does not redirect properly. Any ideas how to solve it?
I needed to use {folllowRedirect: false} option for reqest.js