I wrote a test that passes 95% of the time, failing the other 5%. I still don't know why it fails (my guess is that components are not rendered correctly).
I implemented a page reload call to reload the page and try again, but it's not very reliable.
What's the best way to restart the fixture in case it fails?
Here's a sample test that intentionally fails to emulate my selector that works most of the time, but fails sometimes.
import { Selector } from 'testcafe';
const URL = 'https://www.youtube.com/watch?v=RWQtB6Xv01Q';
fixture `Portal Experience playback`
.page `${URL}`;
test('Testing YouTube', async t => {
await t.click(Selector('.wrong-selector')); // to emulate my unstable test
});
Results in
✖ Testing YouTube
1) The specified selector does not match any element in the DOM tree.
Is it possible to put the test in a for loop and have it break out of the loop in case the test passes?
The quarantine mode serves this purpose. In this mode, TestCafe will restart a failed test again until it passes or fails 3 times and will consider it failed only if it fails 3 times. Get more details about the quarantine mode in this article: Quarantine Mode.
Related
When I try to look for an element in the software I am working on and it is not found the automation stops.
What can I do so that if the element is not found the automation will continue to run?
There are multiple solutions out there for Cypress. As a quick solution, you can set a cookie value and decide to skip the particular test. add this code to your cypress/support/index.js
afterEach(function onAfterEach() {
if (this.currentTest.state === ‘failed’) {
cy.setCookie(‘shouldSkip’, ‘true’);
//set cookie to skip tests for further specs
Cypress.runner.stop();
//this will skip tests only for current spec
}
});
You can read more and see the full blog post here:
Skipping Cypress tests on first failure
I am using WebDriverIO to implement a test automation framework in TypeScript. My framework includes some complex models for the application under test and I need to debug in VS Code to ensure that the models have been implemented correctly.
Unfortunately, although I have configured wdio.conf.js and .vscode/launch.json in such a manner as to properly start the debugging session and the execution stops at my breakpoint (most of the time?), I can't add a watch on any variables or properties of my model - nothing gets added in the watch window. Similarly, if I use the debug console, nothing happens when I type in a property name.
I've tried using browser.debug() as per the documentation (https://webdriver.io/docs/debugging.html), but all that seems to be designed for is stopping your code execution so that you can evaluate the state of the application under test in the browser's DevTool console.
Doing console.log(this) in VS COde's debug console when the execution hits browser.debug() returns what looks like a WebDriverIO instance rather than the context of execution.
Has anyone had luck with this sort of thing?
In your code, include the word 'debugger' where you would like the debug execution to break. See example of 'debugger' placement.
describe('Navigate and debug ', () => {
it('should have the correct title', () => {
browser.url('localhost')
debugger
assert.strictEqual(browser.getTitle(), 'foo')
})
})
I'm using phantomjs and webdriverio to fetch and render a webpage that's loaded by javascript, then save it to be parsed later by cheerio.
Here's the code for that:
import phantomjs from 'phantomjs-prebuilt'
const webdriverio = require('webdriverio')
const wdOpts = {
desiredCapabilities: {
browserName: 'phantomjs'
}
}
async parse (parseUrl) {
return phantomjs.run('--webdriver=4444').then(program => {
return webdriverio.remote(wdOpts)
.init()
.url(parseUrl)
.waitForExist('.main-ios', 100000)
.pause(5000)
.getHTML('html', true)
.then((html) => {
program.kill()
return html
})
})
}
Even though I call program.kill() I notice that the phantomjs in the list of processes, and it does use up quite a bit of RAM and CPU.
I'm wondering why the process doesn't terminate.
.close() just closes the window. There is a known bug, if it is the last window it stays open.
.quit() should do it, but there are issues associated with that as well.
PhantomJS bug report: https://github.com/detro/ghostdriver/issues/162
someone has a decent workaround posted at the bottom of that thread:
https://github.com/SeleniumHQ/selenium/issues/767#issuecomment-140367536
this fix shoots a SIGTERM to end it: (In python, but might be usefull)
# assume browser = webdriver.PhantomJS()
browser.service.process.send_signal(signal.SIGTERM)
browser.quit()
I like to just open a Docker container with my automation, and run it in there. Docker closes it up for me, however that is prolly out of scope for what you want to do.. i would recommend the above SIGTERM+quit method.
PhantomJS is a 2 component product. There is the Javascript which runs on the client (Whether web or other Script) side as part of your code. Then there is the part that runs as a server-side application (The command line call)
It has been my experience with PhantomJS that when an error is encountered, the PHantomJS server side "hangs" but is unresponsive. If you can update your call to this script to provide output logging, you may b able to see what the error is that PhantomJS application is encountering.
phantomjs /path/to/script/ > /path/to/log/file 2>&1
Hope this Helps! If you'd like me to clarify anything, or elaborate I'm happy to update my answer, just let me know in a comment, Thanks!
I'm not quite getting how to use PhantomJS and Mocha together, specifically through mocha-phantomjs.
I've read some tutorials (the one at tutsplus is quite helpful) and am not seeing how I can test external pages using Phantom and Mocha. I'm sure I just need a nudge.
In the tutorial the author creates a tests.js file with some DOM setup/manipulation, as well as some mocha assertions. Then, he creates a separate HTML file that loads the tests.js file and uses mocha-phantomjs to fire up phantom and run the tests.
That's where I'm a little confused, how the mochaPhantomJS.run() method actually does things behind the scenes, whether it knows to search the js file for a describe block and run all tests within, that sort of thing. I don't need chapter and verse, but a high-level summary would be ideal.
Also, if I want to test an outside page, how can I best do that? In the tutorial all the DOM investigation and testing is done on the test page. If I want to test a different page, do I change or setup my assertions differently? Should I call the PhantomJS API from those tests and point to an external site?
Mocha will run tests that have been specified in javascript that has been included in the html page that is launched. If you look at the example html page on mocha-phantomjs it expects the test definitions using describe/it calls to be in the test/mycode.js file. If you put something like
These tests are only testing what is in the main file and associated javascript, there isn't anything special that mocha-phantomjs provides to test external html files. If you want to test your own html files I think you can head in a couple of directions, I came up with these:
first option: Create a way to load the parts of you app that you want to test into the main testing html file. How to do this depends a lot on your application setup. It is probably well-suited for a modular system. Just include the javascript from your application and test it. Not so good for full-page-html tests.
second option: Open new windows with the pages to test from the main testing html file (from within phantom that is). You can open a page using window.open() just like a normal browser. I created a proof of concept like this:
describe('hello web', function () {
it('world', function (done) {
var windowReference = window.open("fileundertest.html");
// windowReference.onload didn't work for me, so I resorted to this solution
var intervalId = window.setInterval(function () {
if (windowReference.document && windowReference.document.readyState === 'complete') {
window.clearInterval(intervalId);
expect(windowReference.document.title).to.equal("File Under Test");
done();
} else {
console.log('not ready yet');
}
}, 10);
});
}
)
This solution is relatively simple, but has the same drawbacks as any page-loading solution: you never know when the page is fully initialized and have to resort to some sort of timeout/wait system to wait for the page to get into the correct state. If you want to test a lot of separate html files these delays start to add up. Additionally waiting for 'onload' on the page that I opened wouldn't work so I created my own function based on setInterval and a (non-perfect) test on the document that was being loaded. I found out there are differences in behavior between loading an html page from the filesystem and loading the same page via a web-server.
third option: Create a mocha test that you run nodejs-style from the command line, and launch phantomjs with a specific page as part of your mocha tests. This is what I'd say you need if your system really depends on html pages that are quite different from each other.
I quickly tested the third option, here is my test based on the example I found on the phantom page (which is an alternative solution to phantomjs that is used by mocha-phantomjs -- I've used neither for more than brief experiments so I cannot recommend which one to use)
'use strict';
var chai = require('chai'),
phantom = require('phantom');
var expect = chai.expect;
describe('hello node', function () {
it('world', function (done) {
phantom.create(function (ph) {
ph.createPage(function (page) {
page.open("http://www.google.com", function (status) {
console.log("opened google? ", status);
page.evaluate(function () { return document.title; }, function (result) {
console.log('Page title is ' + result);
ph.exit();
expect(result).to.equal("Google");
done();
});
});
});
});
});
}
)
While it is possible to test this way I'd say that maybe the overhead of the communication between the code in the phantom-world and the testing code in the nodejs world isn't worth it. You can of course move a lot of general functionality to a couple of helper functions, but you are still stuck with having to call page.evaluate() to perform specific tests on the page. The same issues with timeouts/waits as above apply.
As an aside: do already know CasperJS? Maybe it can be helpful for your setup should you choose to build something on 'plain' phantomjs.
I read about user extensions and extending selenium but am wondering how to call a command from within a custom command I'm creating.
I added a file similar to the following to Selenium core extensions (user-extensions.js) in Selenium IDE Options.
// selenium-action-example.js
Selenium.prototype.doExample = function() {
this.doOpen("/"); // doesn't waitForPageToLoad like the command does
// These two commands are equivalent to the clickAndWait command. NOT!
// For proof, see the filterForRemoteControl function:
// http://code.google.com/p/selenium/source/browse/trunk/ide/src/extension/content/formats/formatCommandOnlyAdapter.js?r=8284#68
this.doClick("css=a#example");
this.doWaitForPageToLoad(); // doesn't wait at all
this.doClick("link=Example");
this.doWaitForElementPresent("example"); // error! undefined function
this.doClick("example");
};
In other words, how can I wait for things between clicks within a custom action?
Your command
this.doWaitForPageToLoad(); // doesn't wait at all
Doesn't wait as you have not specified wait time in brackets. You should write it as
this.doWaitForPageToLoad(30000); // time in milliseconds
Tour another Command
this.doWaitForElementPresent("example"); // error! undefined function
as no function is there in Selenium. whenever it waits for an element it checks that element is present or not so you should wait for time until it is visible/present.
Using For loop and ispresent commands you can do it.
Regards
Waiting for an page load does not to work in current versions of Selenium. As far as I can see, this is because the doWaitForPageToLoad defers the waiting until the end of the current Selenium IDE command, i.e. waiting for a page load stops the test execution until the page has loaded, but not the execution of the actual javascript function that this was executed in.
You will have to split your function in two at this point.