I am having an issue running Testcafe through cucumber. for whatever reason, When I run testCafe through cucumber, the process will always exit with exit code 0 even in the test fails.
If I run puppeteer through cucumber I don't get this issue. I am thinking that this behavior is due to the way I have things set up in my hooks file where I am not properly interpreting the test cafe exit code.
In my hooks file, I am creating a testCafe runner and in my Before hook and then closing it during my after hook.
I am wondering what command I could use to get the TestCafe exit code, and I haven't been able to find any info on this.
For example, is the exit code returned from the close function or what?
TestCafe API does not call the process.exit method since it should work inside custom node scripts.
TestCafe calls process.exit only in CLI.
I suppose that you want to get information about failed tests in API. The runner.run method returns this information. Please see the following example:
const createTestCafe = require('testcafe');
let runner = null;
let tc = null;
createTestCafe('localhost', 1337, 1338)
.then(testcafe => {
tc = testcafe;
runner = tc.createRunner();
})
.then(() => {
return runner
.src('...')
.browsers('chrome')
.run();
})
.then(failedCount => {
console.log(failedCount)
return tc.close();
});
Here, you can call process.exit if you find that failedCount > 0;
Related
I am new to jest with selenium automation test. I am trying to add a beforeAll() and afterAll() functions to open and close the browsers once and run all the tests across multiple files, instead of calling it individually in all the files and opening multiple browsers and loading the website everytime.
Here is my test:
enter image description here
Output
This is beforeAll() and afterAll() methods sitting in a separate file
Actual Test
beforeAll is different from it or describe. The first argument can only be empty or with done if you using callback:
// works
beforeEach((done) => {
...
done()
});
// works
beforeEach(() => {
return asyncSomething()
});
// Doesn't work.
beforeEach(("Don't add string here") => {..});
I want to run testcafe tests concurrently BUT only executing against 1 file at a time.
In other words, I want to wait for all the tests of a specific fixture to be done executing before the tests from the next fixture start executing.
How do I do it?
You can do this using the TestCafe programming interface.
Please see the following example:
const createTestCafe = require('testcafe');
let testcafe = null;
let runner = null;
createTestCafe('localhost', 1337, 1338)
.then(tc => {
testcafe = tc;
runner = tc.createRunner()
.browsers('chrome')
.concurrency(3);
})
.then(() => {
return runner.src('fixture1.js').run();
})
.then(() => {
return runner.src('fixture2.js').run();
})
.then(() => {
testcafe.close();
});
However, please note that I run tests twice in sequence here. That means that your browsers will be opened twice too. You will also get two different reports.
I wanted to have more verbose output for each test step;
Any ideas on how could I best achieve this without adding console.log after each step ?
I tried to overload the t object as shown below but can't get to have it work more than one time in the output.
in mylib.js
exports.init = function(t) {
t.oTypeText = t.typeText;
t.typeText = function fn(selector, data, opts) {
console.log('typing text in '+selector+': '+data);
return t.oTypeText(selector, data, opts);
};
return;
};
in test.js
import { Selector } from 'testcafe';
const mylib = require('./mylib');
fixture("Getting Started")
.page("https://devexpress.github.io/testcafe/example");
test('My first test', async t => {
mylib.init(t);
await t.typeText('#developer-name', 'John Smith')
.selectText('#developer-name').pressKey('delete')
.typeText('#developer-name', 'new name')
.selectText('#developer-name').pressKey('delete')
.typeText('#developer-name', 'another name');
await t.click('#submit-button');
});
result is:
Using locally installed version of TestCafe.
Running tests in:
- Firefox 68.0.0 / Mac OS X 10.14.0
Getting Started
(node:62978) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
typing text in #developer-name: John Smith
✓ My first test
1 passed (4s)
TestCafe doesn't support this feature out of box. I've created suggestion for you use case - https://github.com/DevExpress/testcafe/issues/4001 in the TestCafe repository. We can use the way with action overriding right now, but theoretically it can broke some functionality.
I find myself writing this at the start of pretty much all of my unit tests in mocha:
it('should do something', (done) => {
Vue.config.errorHandler = done;
// do something aynchronous
});
By default, Vue catches all errors itself and logs them to the console, so mocha can't see them. This code makes sure that thrown errors fail the tests.
Is there a way with mocha to do this without having to start every single async test with this line of code? If I have to write / use a plugin, that's fine.
Try:
Vue.config.errorHandler = function (err, vm, info) {
throw err
}
in your test entry.
Running the following jasmine test under webdriver.io like this: node path/to/test/script.js, the test executes (web browser is pulled up, target page visited), and thanks to the last line, the jasmine 'it' functions (below) do execute (without the last line, they don't, although the 'describe' function still runs).
But jasmine doesn't provide any kind of report result for the 'it' tests and the 'expect' assertions; there's nothing on the console from jasmine. There's no 'pass/fail' result, and so forth.
How to get jasmine to make a report, and esp. one that is readable by Jenkins?
The problem test script:
var webdriverjs = require('foo-bar/node_modules/webdriverio');
var jasmine = require('foo-bar/node_modules/jasmine-node');
var options = {
port: 4445,
desiredCapabilities: {
browserName: process.argv[2] || 'phantomjs'
}
};
describe('my webdriverjs tests', function () {
var client;
jasmine.DEFAULT_TIMEOUT_INTERVAL = 9999999;
beforeEach(function() {
client = webdriverjs.remote(options);
client.init();
});
it('shows the correct title', function (done) {
client
.url('http://localhost:4444').getTitle(function(err, title) {
expect(title).toBe('foo bar');
}).call( done );
});
afterEach(function(done) {
client.end(done);
});
});
jasmine.getEnv().execute();
Note: Cross-posted here: https://groups.google.com/forum/#!topic/webdriverio/-EOrQ003B9I
I ran into some of the same challenges when I was looking into this. The big issue is that this test needs to be executed as a jasmine test, not a webdriver test.
decribe('my webdriverio tests with jasmine', function(){
var client;
beforeEach(function(){
client = require('path/to/webdriverio').remote({
desiredCapabilities: {browserName:'safari'}
}).init.url('https://www.stackoverflow');
}, 5000);
afterEach(function(done){
client.end(done);
}, 5000);
it('runs a very simple test',function(done){
client.getTitle(function(err,result){
expect(result).toBe('Stack Overflow');
}).call(done);
}, 5000);
});
Now to run this test, you would just run a typical jasmine-node command from your terminal.
It comes down to the naming convention you are using. First, you need to remove the last line: jasmine.getEnv().execute(); then run the jasmine-node command with the --matchall flag:
jasmine-node --matchall path/to/test/script.js
If you named your file script_spec.js, then you could run it without the --matchall flag.
This is also assuming you have jasmine-node installed globally. If you want to use the local node_modules dependency, then you need to run this command:
./node_modules/jasmine-node/bin/jasmine-node --matchall path/to/test/script.js
When you are using jasmine-node module you should run your spec with
node_modules/jasmine-node/bin/jasmine-node $TEST_DIRECTORY
And your test should end with *spec.js, *spec.coffee or *spec.litcoffee as docs said.
And jasmine.getEnv().execute(); and var jasmine = require('foo-bar/node_modules/jasmine-node'); should not be in your script.