I'm using Zombie.js for testing with Cucumber-js, and I can't seem to get my client side scripts to run.
Visiting the page:
this.browser.visit("http://localhost/boic",function(e, browser,status,errors){
console.log('status',status);
console.log('error',errors);
console.log('console',browser.text("H1"));
});
Returns a status of 200, no errors, and displays the H1 text correctly. However, if I include a script to change the H1 code in the page:
<script>
$('H1').html('hello world');
</script>
The H1 text remains unchanged, and no global variables are accessible via browser.window...
thanks!
Did you load jQuery in your page before the script is called?
there is also the runScripts browser option but that defaults to true.
But I'm gonna recommend running an external phantom.js process and going through the webdriver interface. Just because I spent 6 months trying to get zombie to do what I wanted and phantomjs made it all easy. http://phantomjs.org/release-1.8.html https://github.com/LearnBoost/soda
I agree w/ Jon Biz. Zombie is difficult to work with. Many sites that use JS libraries that might contain minor errors cause it to crash (I think node fails) when the zombie browser encounters them - if you have the runScripts option set. This makes it very hard to use for any application/site that requires external JS - ie most of them.
I also recommend switching to Phantomjs.
Related
I maintain a complex Angular (1.5.x) application that is being E2E tested using Protractor (2.5.x). I am experiencing a problem with this approach, which presents primarily in the way the tests seem flaky. Tests that worked perfectly well in one pull request fail in another. This concerns simple locators, such as by.linkTest(...). I debugged the failing tests and the app is on the correct page, the links are present and accessible.
Has anyone else experienced these consistency problems? Knows of a cause or workaround?
Just Say No to More End-to-End Tests!
That said, here are the few things you can do to tackle our mutual merciless "flakiness" enemy:
update to the latest Protractor (currently 4.0.0) which also brings latest selenium and chromedriver with it
turn off Angular animations
use dragons browser.wait() with a set of built-in or custom Expected Conditions. This is probably by far the most reliable way to approach the problem. Unfortunately, this is use-case and problem specific, you would need to modify your actual tests in the problematic places. For example, if you need to click an element, wait for it to be clickable:
var EC = protractor.ExpectedConditions;
var elm = $("#myid");
browser.wait(EC.elementToBeClickable(elm), 5000);
elm.click();
maximize the browser window (to avoid random element not visible or not clickable errors). Put this to onPrepare():
browser.driver.manage().window().maximize();
increase the Protractor and Jasmine timeouts
slow Protractor down by tweaking the Control Flow (not sure if it works for 4.0.0, please test)
manually call browser.waitForAngular(); in problematic places. I am not sure why this helps but I've seen reports where it definitely helped to fix a flaky test.
use the jasmine done() callback in your specs. This may help to, for example, not to start the it() block until done is called in beforeEach()
return a promise from the onPrepare() function. This usually helps to make sure things are prepared for the test run
use protractor-flake package that would automatically re-run failed tests. More like a quick workaround to the problem
There are also other problem-specific "tricks" like slow typing into the text box, clicking via JavaScript etc.
Yes, I think all of us experienced such flakiness issue.
Actually, the flakiness is quite common issue with any browser automation tool. However, this is supposed to be less in case of Protractor as Protractor has built-in wait consideration which performs actions only after loading the dom properly. But, in few cases you might have to use some explicit waits if you see intermittent failures.
I prefer to use few intelligent wait methods like:
function waitForElementToClickable(locator) {
var domElement = element(by.css(locator)),
isClickable = protractor.ExpectedConditions.elementToBeClickable(domElement);
return browser.wait(isClickable, 2000)
.then(function () {
return domElement;
});
}
Where 2000 ms is used as timeout, you can make it configurable using a variable.Sometimes I also go with browser.sleep() when none of my intelligent wait works.
It's been my experience that some methods (eg. sendKeys()) do not always fire at the expected time, within the controlFlow() queue, and will cause tests to be flakey. I work around this by specifically adding them to the controlFlow(). Eg:
this.enterText = function(input, text) {
return browser.controlFlow().execute(function() {
input.sendKeys(text);
});
};
A workaround that my team has been using is to re-run only failed tests using the plugin protractor-errors. Using this tool, we can identify real failures versus flakey tests within 2-3 runs. To add the plugin, just add a require statement to the bottom of the Protractor config's onPrepare function:
exports.config = {
...
onPrepare: function() {
require('protractor-errors');
}
}
You will need to pass these additional parameters when to run your tests with the plugin:
protractor config.js --params.errorsPath 'jasmineReports' --params.currentTime (timestamp) --params.errorRun (true or false)
There is also a cli tool that will handle generating the currentTime if you don't have an easy way to pass in a timestamp.
So I am in the process of writing some tests with Protractor for an angular application I am working on. I ran into an issue where a test was failing because I tried to click on an element that while existed, it could not be clicked because another element was above it and it was receiving the click event. The error was just that true not does equal false which gives no insight to the real underlaying issue. I have run into this issue many times with other tests so I knew pretty quickly was the issue was but if I had not experienced this before, I don't know how long it would take me to figure it out.
I am 99% sure that when you send a click event with the JSON Wire Protocol that if the element does receive the click, there will be a message relating to that in it's response. Is there any way with Protractor to get the JSON Wire Protocol responses on to the screen when running the tests or at least get the responses captured in a file or something?
Assuming you are using Jasmine (the default) i suggest you start using explicit waits for elements to be present and visible before interacting with them like in your example.
I'm using this custom mathers.
Then:
var theElementFinder = $('#someElm');
expect(theElementFinder).toBePresentAndDisplayed();
Regarding
a way with Protractor to get the JSON Wire Protocol responses
You already see selenium errors in your Terminal / Console output.
Is there a way to get a realtime view of what PhantomJS (or similar) is rendering?
I would like to develop my automation script while interacting with (or at least seeing a screencap of) the page it's targeted to.
No, there is no such thing. SlimerJS has the same API as PhantomJS, but runs the Gecko engine. You can see directly what is going on and run it headlessly with xvfb-run.
You will not be able to interact with it. You may want to use a screengrabber to record a video of the interaction when the tests are long and you don't want to run the test suite again if you didn't catch the problem in the test case.
The obvious way to debug PhantomJS scripts is to render many screenshots using page.render() and logging some objects to the console with
console.log(JSON.stringify(yourObj, undefined, 4));
with nice formatting.
Solution we use is an automatic screenshoting in case of exceptions, phantomJs will render the current page into a file that you can exam later .
That's for test execution phase.
When you writing the tests, just keep additional window open ("normal browser") with the application you trying to test and design the test according to it.
When the design is done, execute the test with phantomJS.
My Suggestion is to use logging alongside.
http://casperjs.org/
CasperJS is an open source navigation scripting & testing utility written in Javascript for the PhantomJS WebKit headless browser and SlimerJS (Gecko). It eases the process of defining a full navigation scenario and provides useful high-level functions, methods & syntactic sugar for doing common tasks such as:
defining & ordering browsing navigation steps
filling & submitting forms
clicking & following links
capturing screenshots of a page (or part of it)
testing remote DOM
logging events
downloading resources, including binary ones
writing functional test suites, saving results as JUnit XML
scraping Web contents
The solution to this problem is using the remote debugger:
--remote-debugger-port=9000
Using slimerjs for testing scripts with a browser is not advisable since it is based on gecko, which means the script might work on slimerjs and not on phantomjs or viceversa.
take a look at this guide for more info...
https://drupalize.me/blog/201410/using-remote-debugger-casperjs-and-phantomjs
We're using WatiN to test our web portals. During the course of an E2E test, we'll occasionally see client-side script errors on the IE status bar. I'd like to chain a handler onto the script error event and record the error for later analysis and bug filing.
Problem is, I don't know that there's a global script error event or how to chain into it. And if there's not a browser-agnostic way to accomplish this, I can create MyIE and MyFF subclasses but then this becomes two browser-specific questions.
In essence, I'm thinking of something like this entirely made-up call:
browser.ScriptEngine.SetCustomErrorHandler(LogScriptingError);
... where LogScriptErrors is my code that does the obvious.
Many of our client-side scripting errors don't necessarily prevent the test from continuing (a pretty UI element didn't animate, for example, but the underlying form is still submittable), so I'd like to log the error and forge ahead in most cases.
You probably looking for this:
window.onerror=function(message, url, line){logError();};
You can add this code to your pages to handle errors in logError(). but this may not work in all browser(works in IE), check this for browser compatibility:
http://www.quirksmode.org/dom/events/error.html
Or you may try this commercial product:
exceptionhub.com/
You could maybe co-opt the ability to inject eval code (described under "Added Eval functionality") to add a script that caught all errors, not just errors from the eval'ed script. I'm not sure if this would work, but it's an area to explore. Another resource might be this blog post, which discusses how to evaluate Javascript in WatiN.
When selenium tries to open popup window I'm getting JS error permission denied in file
file:///C:/DOCUME~1//LOCALS~1/Temp/customProfileDir8708f7f69e14482ba857f4b2e74775c1/core/RemoteRunner.hta
So this break script execution, could you assist? I saw a related topic at MSDN and openqa but didn't find resolution that could help me.
I've just encountered this error. In the end it was because I was running IE in 'Offline' mode. Open the File menu and make sure that "Work Offline" does not have a tick next to it.
I've just updated a section about that in the Selenium docs. The website build is not working right now, so if you go to the site you will find the old version.
I'll paste the raw text here, I think your case is the second: JS trying to access sections that are still not loaded, so your solution would be a waitForPopUp command:
Why am I getting a permission denied
error?
The most common reason for this error
is that your session is attempting to
violate the same-origin policy by
crossing domain boundaries (e.g.,
accesses a page from http://domain1
and then accesses a page from
http://domain2) or switching protocols
(moving from http://domainX to
https://domainX). For this to be
solved, try using the Heightened
Privileges Browsers if you're working
with the Proxy Injection browsers.
This is covered in some detail in the
tutorial. Make sure you read the
sections about The Same Origin Policy
and Proxy Injection carefully.
If the previous situation was not your
case, it can also occur when
JavaScript attempts to look at
objects which are not yet available
(before the page has completely
loaded), or tries to look at objects
which are no longer available (after
the page has started to be unloaded).
This is most typically encountered
with AJAX pages which are working with
sections of a page or subframes that
load and/or reload independently of
the larger page. For this type of
problem, it is common that the error
is intermittent. Often it is
impossible to reproduce the problem
with a debugger because the trouble
stems from race conditions which are
not reproducible when the debugger's
overhead is added to the system. Try
first adding a static pause to make
sure this is the situation and then
moving on to the waitFor kind of
commands.