I am using WebDriverIO to implement a test automation framework in TypeScript. My framework includes some complex models for the application under test and I need to debug in VS Code to ensure that the models have been implemented correctly.
Unfortunately, although I have configured wdio.conf.js and .vscode/launch.json in such a manner as to properly start the debugging session and the execution stops at my breakpoint (most of the time?), I can't add a watch on any variables or properties of my model - nothing gets added in the watch window. Similarly, if I use the debug console, nothing happens when I type in a property name.
I've tried using browser.debug() as per the documentation (https://webdriver.io/docs/debugging.html), but all that seems to be designed for is stopping your code execution so that you can evaluate the state of the application under test in the browser's DevTool console.
Doing console.log(this) in VS COde's debug console when the execution hits browser.debug() returns what looks like a WebDriverIO instance rather than the context of execution.
Has anyone had luck with this sort of thing?
In your code, include the word 'debugger' where you would like the debug execution to break. See example of 'debugger' placement.
describe('Navigate and debug ', () => {
it('should have the correct title', () => {
browser.url('localhost')
debugger
assert.strictEqual(browser.getTitle(), 'foo')
})
})
Related
I wrote a test that passes 95% of the time, failing the other 5%. I still don't know why it fails (my guess is that components are not rendered correctly).
I implemented a page reload call to reload the page and try again, but it's not very reliable.
What's the best way to restart the fixture in case it fails?
Here's a sample test that intentionally fails to emulate my selector that works most of the time, but fails sometimes.
import { Selector } from 'testcafe';
const URL = 'https://www.youtube.com/watch?v=RWQtB6Xv01Q';
fixture `Portal Experience playback`
.page `${URL}`;
test('Testing YouTube', async t => {
await t.click(Selector('.wrong-selector')); // to emulate my unstable test
});
Results in
✖ Testing YouTube
1) The specified selector does not match any element in the DOM tree.
Is it possible to put the test in a for loop and have it break out of the loop in case the test passes?
The quarantine mode serves this purpose. In this mode, TestCafe will restart a failed test again until it passes or fails 3 times and will consider it failed only if it fails 3 times. Get more details about the quarantine mode in this article: Quarantine Mode.
I'd like to debug my test when it is run by CIDER:
(let [test-system (atom (new-test-system))]
(some-setup-code)
(deftest my-test
;uses test-system in here)
As you see, my tests are wrapped in a let, which gets debugged as normal: I can hit c to move through the forms until it gets to the deftest, then the debugger quits, and my-test is highlighted in red, ready to be debugged when run.
When I run the test with , t t it doesn't debug, it passes as normal. I expected it to break and let me debug inside the test.
What am I doing wrong? I'm rather new to CIDER.
I'm facing a similar issue.
Even if I instrument a function that is called from within the deftest I can't debug it when I execute the test with , t t (cider-test-run-focused-test).
It seems that this action clears out the instrumentation.
If I run the test manually via (clojure.test/run-tests) (switching to the namespace before e.g. via cider-repl-set-ns) I can debug both tests and auxiliary functions without problems.
My app displays an error dialog whenever a JavaScript error occurs. This is always a bad sign, so I want to set up my tests so that, if the error dialog appears, it causes the test to fail there and then.
So I'd like to do something like (very much pseudocode!);
// start a new 'guard' thread;
start {
found = this.driver.wait(untilVisible(By.css('.myErrorDialog')), VERY_LONG_TIMEOUT);
if (found) {
// the error dialog appeared! That's bad!
throw();
}
}
// now run the test
login();
clickButton();
testBannerContains();
But I'm having trouble and I think it has to do with the way Selenium schedules actions.
What I've found is that for a single driver, I can only schedule one thing at a time, so the guard I set up early in the test blocks the body of the test from starting.
Is there a better way to handle conditions like 'this should never happen', or a way to create two independent threads in the same test?
So the problem with the code you have is that it immediately runs it and waits for a VERY_LONG_TIMEOUT amount of time for that error dialog to appear. Since it never does, it continues to wait. You have already discovered that is not what you want... ;)
I haven't done anything like this but I think you want a JS event handler that watches for the event that is triggered when the error dialog appears. See the link below for some guidance there.
Can my WebDriver script catch a event from the webpage?
One option would be to watch for that event to fire and then store true (or whatever) in some JS variable. Before leaving a page, check to see if the variable is set to true and if so, fail the test. You can set and get JS variables using JavascriptExecutor. Some google searches should get you all you need to use it.
I'm not quite getting how to use PhantomJS and Mocha together, specifically through mocha-phantomjs.
I've read some tutorials (the one at tutsplus is quite helpful) and am not seeing how I can test external pages using Phantom and Mocha. I'm sure I just need a nudge.
In the tutorial the author creates a tests.js file with some DOM setup/manipulation, as well as some mocha assertions. Then, he creates a separate HTML file that loads the tests.js file and uses mocha-phantomjs to fire up phantom and run the tests.
That's where I'm a little confused, how the mochaPhantomJS.run() method actually does things behind the scenes, whether it knows to search the js file for a describe block and run all tests within, that sort of thing. I don't need chapter and verse, but a high-level summary would be ideal.
Also, if I want to test an outside page, how can I best do that? In the tutorial all the DOM investigation and testing is done on the test page. If I want to test a different page, do I change or setup my assertions differently? Should I call the PhantomJS API from those tests and point to an external site?
Mocha will run tests that have been specified in javascript that has been included in the html page that is launched. If you look at the example html page on mocha-phantomjs it expects the test definitions using describe/it calls to be in the test/mycode.js file. If you put something like
These tests are only testing what is in the main file and associated javascript, there isn't anything special that mocha-phantomjs provides to test external html files. If you want to test your own html files I think you can head in a couple of directions, I came up with these:
first option: Create a way to load the parts of you app that you want to test into the main testing html file. How to do this depends a lot on your application setup. It is probably well-suited for a modular system. Just include the javascript from your application and test it. Not so good for full-page-html tests.
second option: Open new windows with the pages to test from the main testing html file (from within phantom that is). You can open a page using window.open() just like a normal browser. I created a proof of concept like this:
describe('hello web', function () {
it('world', function (done) {
var windowReference = window.open("fileundertest.html");
// windowReference.onload didn't work for me, so I resorted to this solution
var intervalId = window.setInterval(function () {
if (windowReference.document && windowReference.document.readyState === 'complete') {
window.clearInterval(intervalId);
expect(windowReference.document.title).to.equal("File Under Test");
done();
} else {
console.log('not ready yet');
}
}, 10);
});
}
)
This solution is relatively simple, but has the same drawbacks as any page-loading solution: you never know when the page is fully initialized and have to resort to some sort of timeout/wait system to wait for the page to get into the correct state. If you want to test a lot of separate html files these delays start to add up. Additionally waiting for 'onload' on the page that I opened wouldn't work so I created my own function based on setInterval and a (non-perfect) test on the document that was being loaded. I found out there are differences in behavior between loading an html page from the filesystem and loading the same page via a web-server.
third option: Create a mocha test that you run nodejs-style from the command line, and launch phantomjs with a specific page as part of your mocha tests. This is what I'd say you need if your system really depends on html pages that are quite different from each other.
I quickly tested the third option, here is my test based on the example I found on the phantom page (which is an alternative solution to phantomjs that is used by mocha-phantomjs -- I've used neither for more than brief experiments so I cannot recommend which one to use)
'use strict';
var chai = require('chai'),
phantom = require('phantom');
var expect = chai.expect;
describe('hello node', function () {
it('world', function (done) {
phantom.create(function (ph) {
ph.createPage(function (page) {
page.open("http://www.google.com", function (status) {
console.log("opened google? ", status);
page.evaluate(function () { return document.title; }, function (result) {
console.log('Page title is ' + result);
ph.exit();
expect(result).to.equal("Google");
done();
});
});
});
});
});
}
)
While it is possible to test this way I'd say that maybe the overhead of the communication between the code in the phantom-world and the testing code in the nodejs world isn't worth it. You can of course move a lot of general functionality to a couple of helper functions, but you are still stuck with having to call page.evaluate() to perform specific tests on the page. The same issues with timeouts/waits as above apply.
As an aside: do already know CasperJS? Maybe it can be helpful for your setup should you choose to build something on 'plain' phantomjs.
I've got the following code in a thread in my application:
while (true) {
if (ready) {
progressIndicatorController.value++;
return;
}
}
The ready variable is changed from a delegate method. This code works great when I open the application by clicking the "Run" button in Xcode's toolbar. However, if I open this application's .app (which I create by clicking Product > Archive and then following the steps) this code somehow doesn't work anymore.
progressIndicatorController.value is never incremented and this if-statement never evaluates to true. What could cause this problem?
This is probably caused by optimization from the compiler.
When you build with Archive, XCode enabled optimization in the compiler that could throw this kind of code away. I think setting the ready variable to volatile could fix your problem, altough if I were you I'd just try to rewrite it so it doesn't trigger this problem.
You can test with optimization turned on by choosing Edit Schemes in the scheme dropdown. Then set Build Configuration to Release in the Run MyApp.app. Don't forget to set it back to Debug when you're done though, as the debugger gets somewhat confused when optimization are on (i.e. you can't see the value of most variables, some breakpoints may behave erratically, etc...)