I have noticed that some Geb functional tests pass with Chrome but fail with PhantomJS, holding all other variables constant. This happens mostly with pages that have some kind of asynchronous activity - one call to $(selector).click() triggers an event handler that updates the DOM, and the DOM updates need to complete before calling $(anotherSelector).click().
I can make the PhantomJS tests pass again by aggressively using waitFor but I don't understand why this would be required with the PhantomJS GhostDriver and not the Chrome driver.
Unfortunately I haven't been able to construct a minimal test case yet isolated from my application.
The only advice I can have is to always make sure that any actions in your tests around asynchronous activities are guarded with waitFor statements. You will avoid problems where one driver is quick enough to finish the asynchronous activity before your test tries to access the new/modified element in your page but other isn't. Not using waitFor around asynchronous activities will also bite you when you start running your tests on CI where they are usually slower and you'll see more failures related to asynchronicity in tested pages.
I also wouldn't consider using waitFor to guard every asynchronous activity in your test as being aggressive. You have to remember that waitFor periodically polls the condition and continues as soon as the condition is fulfilled - so if your browser is quick and the page gets updated before waitFor polls for the first time then you're not penalized speed-wise at all but if it's not then you have a guarantee that there will be a retry to see if the asynch action has finished and the condition was fulfilled. What I find aggressive is using ridiculously high timeouts like 30s where they definitely aren't needed - it just means that if your test fail it will take very long time for it to happen.
I had a recent experience even with IE and Firefox. This is a list you can try:
Of course, waitFor{}/(timeout: ) is your friend
I am pretty sure Chrome and PhantomJS are not behaving exactly the same way. So, manually try and observe what's the difference, then use something like this:
if(System.properties["geb.env"]=="chrome") {
$(selector).click()
} else {
// Do something else
}
Related
In our web application there are tables where users can enter/delete data.
Checking once something has been added, I'd like UFT to wait until the element is no longer on the page before continuing. Each time a row, or element is deleted from the webpage the page refreshes, often UFT will try to continue thus causing the test to fail.
It seems .WaitProperty "visible", false, 3000 waits for the timeout to complete and then continues.
I'm looking for something similar to .StalenessOf in Selenium.
Looking to have the wait be more dynamic to make these tests run as fast as possible. If we end up using UFT I'll have a very large test suite and would like to reduce the amount of hard-coded waits, and avoid them if I could.
UFT supports Exist to check if an object exists.
You can pass a zero timeout and wait until the object doesn't exist.
While Obj.Exist(0)
Wait 1
WEnd
I am using Java and Selenium to write a test. Many times in my test i have to wait till the page or some part of it loads something. I want to wait till the load is finished. I found these two:
1) pageLoadTimeout
2) setScriptTimeout
But I don't know the exact difference between them or even if they are what I need.
The document says:
setScriptTimeout: Sets the amount of time to wait for an asynchronous script to finish execution before throwing an error. If the timeout is negative, then the script will be allowed to run indefinitely.
and
pageLoadTimeout: Sets the amount of time to wait for a page load to complete before throwing an error. If the timeout is negative, page loads can be indefinite.
but I don't really get it.
what I need is a wait that stops the test till nothing is happening in the web page anymore.
Define "nothing is happening in the web page." Now define it in such a way that will satisfy every web page ever created, or that ever will be created. Did your definition include that no more network traffic is coming over the wire? Did your definition include that the onload event has fired? How about document.readyState returning a value meaning "complete?" Did you take JavaScript execution into account? How about JavaScript scheduled using window.setTimeout()? WebSockets? Service Workers? What about <frame> or <iframe> elements?
Now you know why WebDriver doesn't have a "wait for page complete" type of wait. The items mentioned above don't even begin to scratch the surface of the list of things a user could legitimately use in a definition of "nothing is happening on the page anymore." The correct approach would be to wait for the element you want to interact with on the next page to be available in the DOM. The WebDriverWait class is ideally suited for exactly this.
I maintain a complex Angular (1.5.x) application that is being E2E tested using Protractor (2.5.x). I am experiencing a problem with this approach, which presents primarily in the way the tests seem flaky. Tests that worked perfectly well in one pull request fail in another. This concerns simple locators, such as by.linkTest(...). I debugged the failing tests and the app is on the correct page, the links are present and accessible.
Has anyone else experienced these consistency problems? Knows of a cause or workaround?
Just Say No to More End-to-End Tests!
That said, here are the few things you can do to tackle our mutual merciless "flakiness" enemy:
update to the latest Protractor (currently 4.0.0) which also brings latest selenium and chromedriver with it
turn off Angular animations
use dragons browser.wait() with a set of built-in or custom Expected Conditions. This is probably by far the most reliable way to approach the problem. Unfortunately, this is use-case and problem specific, you would need to modify your actual tests in the problematic places. For example, if you need to click an element, wait for it to be clickable:
var EC = protractor.ExpectedConditions;
var elm = $("#myid");
browser.wait(EC.elementToBeClickable(elm), 5000);
elm.click();
maximize the browser window (to avoid random element not visible or not clickable errors). Put this to onPrepare():
browser.driver.manage().window().maximize();
increase the Protractor and Jasmine timeouts
slow Protractor down by tweaking the Control Flow (not sure if it works for 4.0.0, please test)
manually call browser.waitForAngular(); in problematic places. I am not sure why this helps but I've seen reports where it definitely helped to fix a flaky test.
use the jasmine done() callback in your specs. This may help to, for example, not to start the it() block until done is called in beforeEach()
return a promise from the onPrepare() function. This usually helps to make sure things are prepared for the test run
use protractor-flake package that would automatically re-run failed tests. More like a quick workaround to the problem
There are also other problem-specific "tricks" like slow typing into the text box, clicking via JavaScript etc.
Yes, I think all of us experienced such flakiness issue.
Actually, the flakiness is quite common issue with any browser automation tool. However, this is supposed to be less in case of Protractor as Protractor has built-in wait consideration which performs actions only after loading the dom properly. But, in few cases you might have to use some explicit waits if you see intermittent failures.
I prefer to use few intelligent wait methods like:
function waitForElementToClickable(locator) {
var domElement = element(by.css(locator)),
isClickable = protractor.ExpectedConditions.elementToBeClickable(domElement);
return browser.wait(isClickable, 2000)
.then(function () {
return domElement;
});
}
Where 2000 ms is used as timeout, you can make it configurable using a variable.Sometimes I also go with browser.sleep() when none of my intelligent wait works.
It's been my experience that some methods (eg. sendKeys()) do not always fire at the expected time, within the controlFlow() queue, and will cause tests to be flakey. I work around this by specifically adding them to the controlFlow(). Eg:
this.enterText = function(input, text) {
return browser.controlFlow().execute(function() {
input.sendKeys(text);
});
};
A workaround that my team has been using is to re-run only failed tests using the plugin protractor-errors. Using this tool, we can identify real failures versus flakey tests within 2-3 runs. To add the plugin, just add a require statement to the bottom of the Protractor config's onPrepare function:
exports.config = {
...
onPrepare: function() {
require('protractor-errors');
}
}
You will need to pass these additional parameters when to run your tests with the plugin:
protractor config.js --params.errorsPath 'jasmineReports' --params.currentTime (timestamp) --params.errorRun (true or false)
There is also a cli tool that will handle generating the currentTime if you don't have an easy way to pass in a timestamp.
when SetScriptTimeout should be used and please provide me any example.
I know the defn
Sets the amount of time to wait for an asynchronous script to finish execution before throwing an error. If the timeout is negative, then the script will be allowed to run indefinitely.
but not sure what it does exactly.
You've got two answers already, neither of which I find explain clearly the point of setting a script timeout.
First, it is important the script timeout affects only JavaScript code executed with executeAsyncScript and nothing else. In particular, executeScript is not affected by it.
So why do you want to set a timeout for executeAsyncScript? Chandan Nayak correctly explained that the default timeout is 0s so you have to change this timeout if you want to use executeAsyncScript with asynchronous scripts that actually perform some work. But why not just set it to -1 and be done with it? After all, if you set it to -1 then you turn off the timeout. So you won't get any timeouts anymore. Mission accomplished, right? Nope.
What you want to do is set the timeout to a value that allows the code you use with executeAsyncScript to perform it works while at the same time detect when a script has gone rogue. For instance, if from experience you know that a script you pass to executeAsyncScript is going to be done in 2 seconds or less (except perhaps in extremely unusual circumstances), then you set the timeout to 2 seconds so that if there is a bug somewhere and the code never terminates, you get a timeout after 2 seconds. Otherwise, Selenium will happily wait forever for the script to complete.
From WebDriver documentation:
setScriptTimeout(long time, java.util.concurrent.TimeUnit unit)
Sets the amount of time to wait for an asynchronous script to finish execution before throwing an error. This works only for Assync scripts (executeAsyncScript)
Let's run a simple javascript: (Do not set setScriptTimeout ) - Now this shall execute without throwing any issue.
((JavascriptExecutor) driver).executeScript("alert('hello world');");
Lets run a simple Assync Script: ( Do not set setScriptTimeout) - This shall fail with error - "Timed out waiting for async script result after 0ms"
((JavascriptExecutor) driver).executeAsyncScript("window.setTimeout(arguments[arguments.length - 1], 500);");
To resolve the issue: setScriptTimeout to 1 Second:
driver.manage().timeouts().setScriptTimeout(1, TimeUnit.SECONDS);
And then run the same Assync Script mentioned above and it shall execute without any error.
Reason:
The default timeout for a script to be executed is 0ms. In most cases, including the examples below, one must set the script timeout WebDriver.Timeouts.setScriptTimeout(long, java.util.concurrent.TimeUnit) beforehand to a value sufficiently large enough
More Reference Links:
When should setScriptTimeout be used?
WebDriver executeAsyncScript vs executeScript
WebDriver Doc
Web application automation is dependent on many factors like browser, network speed, embedded scripting etc. To write robust code for running in all environments we need to inserts wait for WebElements before performing any operation on that.
WebDriver wait (synchronization) can be obtained either by using support.ui or driver().manage().timeouts()
If we use driver.manage().timeouts(), a common practice to achieve synchronization is to use JavaScript via JavascriptExecutor which in turn provide two methods for script execution:
executeAsyncScript -> This method doesn't block the execution of next line of code...till execution of this method is completed. This method will execute as well as next line of code will be executed...asynchronously. (without blocking each other)
executeScript -> This method will block the execution till it's execution is completed and then it moves to next line of code. In short your automation code will halt till the Javascript is executed via this method.
Now since executeAsyncScript method doesn't block the execution of next line of code, it might be beneficial to use driver.manage().timeouts().setScriptTimeout(30,SECONDS);
so that our code can wait for specified amount of time for an asynchronous script to finish execution before throwing an error.
I would like my node.js tests to ensure that, once the test is over and test.finish() or similar is called, that there is no more code waiting to be run. No more I/O waiting to finish, no more timers waiting to fire, etc. etc.
Is this possible in node.js?
When using nodeunit each test function keeps running until test.done() has been called. Every test function needs to call this. This way you can make sure your callbacks have been executed. I also like to use async module to clean up my code(callbacks) a bit.
Are you using test.expect() at the beginning of each test and test.done() at the end of each one? Think of them like begin and end braces.
I wrote my own, which essentially spins up a node instance for each test. If your test leaves a callback dangling, node just won't exit, and the whole test suite hangs instead of exiting early with success (which would be bad!)