Running Testcafe tests in sequence with html reports - testing

This is not a question. I just want to share my solution for running Testcafe tests in sequence with HTML reports.
Running tests in parallel on different browsers was not a solution for me. I have to wait for tests on one env to finish, then to run on the next env.
It took me a while to figure this out, but it works for me.
If someone has better solution, please inform me.
Just add this code (with your specific custom settings) into a runner file. i.e. runner.js
and run it with node runner.js command.
The solution:
const createTestCafe = require('testcafe');
const fs = require('fs');
const browsers = [
'chrome',
'firefox'
];
let stream = null;
const runTest = async browser => {
console.log('----------------- starting tests on ' + browser);
await createTestCafe('localhost', 1337, 1338)
.then(tc => {
testcafe = tc;
const runner = testcafe.createRunner();
return runner
.src([
"./smokeTests/someTests.js"
])
.browsers(browser)
.reporter('html', stream)
.run();
})
.then(async failedCount => {
console.log('Tests failed: ' + failedCount);
await testcafe.close();
return;
});
}
const runAllBrowsers = async () => {
for (const browser of browsers) {
stream = fs.createWriteStream('./testResults' +'/report_' + browser + '.html');
await runTest(browser);
await testcafe.close();
}
}
runAllBrowsers();
I used the original idea from https://github.com/DevExpress/testcafe/issues/2495.
I'd like to thank nabrahamson for the original idea!

Thank you for sharing this solution with everyone.
You might want to contribute your sample to the testcafe-examples repo as well.

Related

How to share same webdriver between multiple step definition files in Selenium JS / Cucumber framework?

Unable to figure out why the following error comes up, when there is a common configuration file that serves to share webdriver among different feature files
Scenario: Amazon login # features\loginR.feature:2
Before # features\Step_Defn\cucumber.common.js:4
Given :user goes to amazon site
Undefined. Implement with the following snippet:
Given(':user goes to amazon site', function () {
Write code here that turns the phrase above into concrete actions
return 'pending';
});
Before part is invoked.Chrome pops up and closes since After is executed.But is unable to get the step definition and feature binding correct.
Following is way of invoking in VSCode
./node_modules/.bin/cucumber-js --require ./features/Step_Defn/cucumber.common.js./features/loginR.feature -f json:test/report/cucumber_report.json
the project structure is
> Selen_proj
> |
> |
> Features
> |
> Login.feature
> LoginR.feature
> |
> Step_Defn
> |
> LoginR.steps.js
> Login.steps.js
cucumber.common.js file:
require('chromedriver');
const { Before, After } = require('#cucumber/cucumber');
const webdriver = require('selenium-webdriver');
Before(() => {
global.driver = new webdriver.Builder()
.forBrowser('chrome')
.build();
})
After(() => {
driver.quit();
})
Snippet of step defn file:
require('chromedriver');
const { Given,When, Then, After } = require('#cucumber/cucumber');
const { Builder, By, until } = require('selenium-webdriver');
//let chai = require('chai');
var {setDefaultTimeout} = require('#cucumber/cucumber'); setDefaultTimeout(60 * 1000);
const { assert } = require('chai');
let driver ;
Given(':user navigates to amazon url', async function () {
driver = await new Builder().forBrowser("chrome").build();
await driver.get("https://www.amazon.com");
await driver.manage().window().maximize();
});

How to get browser console logs when using Browser library in Robotframework

I'm using Robotframework and Browser library to automate some tasks on the web. I used to use Selenium, and with selenium there is a way to get the logs, for example in the case of a failure:
driver = webdriver.Remote()
logs = driver.get_log('browser')
I've been struggling to find a way to do the same exact thing using Playwright's Browser library. Is it possible?
Certainly. You can use the page.on('console') event to log what appears in the DevTools console. Here's an example of using debug library to do so.
Make sure to export DEBUG=playwright:console or you won't see anything.
Here's how to do it in JS:
const playwright = require('playwright');
const debugConsole = require('debug')('playwright:console');
(async () => {
const browser = await playwright.chromium.launch({ headless: false });
const context = await browser.newContext();
const page = await context.newPage();
await page.on('console', (msg) => {
if (msg && msg.text) {
if (typeof msg.text === 'function') {
debugConsole('PAGE LOG:', msg.text());
} else {
debugConsole('PAGE LOG:', msg.text);
}
} else {
debugConsole('PAGE LOG:', msg);
}
});
await page.goto('https://example.com', { waitUntil: 'networkidle' });
})();
And in python:
from playwright.sync_api import sync_playwright
def print_args(msg):
for arg in msg.args:
print(arg.json_value())
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
page.on("console", print_args)
page.goto("https://abrahamjuliot.github.io/creepjs/", wait_until="networkidle")
page.wait_for_timeout(5000)
browser.close()
If you are looking for more system-level stuff, there is also a dumpio launch parameter that you can set, which will cause Playwright to provide verbose logs on the actual launch of browser executable.

testcafe - Source files do not contain valid 'fixture' and 'test' declarations

I'm writing a test in testcafe for the first time, and trying to do a forEach loop in order to iterate various files.
When trying to run the test, getting this error Source files do not contain valid 'fixture' and 'test' declarations.
import testcafe from 'testcafe';
import { promises } from 'fs';
import { parse } from 'node-html-parser';
let specFiles: string[] = [];
fixture ('OpsLevel test cafe reporter').before( async t => {
specFiles = await promises.readdir('./gauge-reports/html-report/specs')
});
specFiles.forEach(specFile => {
test(`Generate testcafe report from ${specFile}`, async t => {
const gaugeReport = await promises.readFile(specFile, {encoding: 'utf8', flag: 'r'});
const parsedReport = parse(gaugeReport);
const structuredText = parsedReport.structuredText;
await t.expect(structuredText.includes('Success Rate 100%')).eql(true);
});
});
Please advise.
Runtime DataSet initialization for data driven tests is not supported. Please refer to the documentation to learn how to properly initialize a dataset.

master runner in testcafe for several other runners?

I have several runners which are using promise.race to complete the testcase at a particular time
Say I have runner1.js, runner2.js runner3.js, how do I create a master runner so that I can run all these runners together?
const createTestCafe = require('testcafe');
let testcafe = null;
// createTestCafe('localhost', 1337, 1338)
createTestCafe()
.then(tc => {
testcafe = tc;
//create test runner for configuring and launching test tasks
const runner = testcafe.createRunner();
return runner
//run test from specified folders/files
.src(['*the path to all runner.js*'])
//configure the test runner to run tests in the specified browsers
.browsers('chrome')
.reporter('html-testrail')
.run({skipJsErrors:true})
})
.catch(failedCount => {
console.log('Tests failed: ' + failedCount);
testcafe.close();
})
it's not working this way, any suggestions?
TestCafe allows running multiple test runners at the same time. Check the following code:
const createTestCafe = require('testcafe');
(async () => {
const testCafe = await createTestCafe();
const runner1 = testCafe
.createRunner()
.src('test1.js')
.reporter([{ name: 'spec', output: 'report1.txt' }])
.browsers('chrome');
const runner2 = testCafe
.createRunner()
.src('test2.js')
.reporter([{ name: 'spec', output: 'report2.txt' }])
.browsers('firefox');
await Promise.all([runner1, runner2].map(runner => runner.run()));
await testCafe.close();
})();

How to test an Electron app with selenium webdriver

I have read the documentation and I have followed the tutorial step by step and I only have managed to run the app.
Documentation: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/
The connection with chromedriver I cannot make it work, when I launch the test and try click a simple button I get this:
Error: ChromeDriver did not start within 5000ms at Error (native)
at node_modules/spectron/lib/chrome-driver.js:58:25 at
Request._callback (node_modules/spectron/lib/chrome-driver.js:116:45)
at Request.self.callback
(node_modules/spectron/node_modules/request/request.js:200:22) at
Request.
(node_modules/spectron/node_modules/request/request.js:1067:10) at
IncomingMessage.
(node_modules/spectron/node_modules/request/request.js:988:12) at
endReadableNT (_stream_readable.js:913:12) at _combinedTickCallback
(internal/process/next_tick.js:74:11) at process._tickCallback
(internal/process/next_tick.js:98:9)
My code:
"use strict";
require("co-mocha");
var Application = require('spectron').Application;
var assert = require('assert');
const webdriver = require('selenium-webdriver');
const driver = new webdriver.Builder()
.usingServer('http://127.0.0.1:9515')
.withCapabilities({
chromeOptions: {
binary: "./appPath/app"
}
})
.forBrowser('electron')
.build();
describe('Application launch', function () {
this.timeout(100000);
var app;
beforeEach(function () {
app = new Application({
path: "./appPath/app"
});
return app.start();
});
afterEach(function () {
if (app && app.isRunning()) {
return app.stop();
}
});
it('click a button', function* () {
yield driver.sleep(5000);
yield driver.findElement(webdriver.By.css(".classSelector")).click();
});
});
Thanks and sorry for my English.
I recommend you to use Spectron. which is a less painful way of testing your electron app. in my opinion perfect combination is using it with Ava test framework, which allows the concurrently test.
async & await is also another big win. which allows you to have so clean test cases.
and also if you have a test which needs to happen serial, you can use test.serial
test.serial('login as new user', async t => {
let app = t.context.app
app = await loginNewUser(app)
await util.screenshotCreateOrCompare(app, t, 'new-user-mission-view-empty')
})
test.serial('Can Navigate to Preference Page', async t => {
let app = t.context.app
await app.client.click('[data-test="preference-button"]')
await util.screenshotCreateOrCompare(app, t, 'new-user-preference-page-empty')
})
and just for reference; my helper test cases.
test.before(async t => {
app = util.createApp()
app = await util.waitForLoad(app, t)
})
test.beforeEach(async t => {
t.context.app = app
})
test.afterEach(async t => {
console.log('test complete')
})
// CleanUp
test.after.always(async t => {
// This runs after each test and other test hooks, even if they
failed
await app.client.localStorage('DELETE', 'user')
console.log('delete all files')
const clean = await exec('rm -rf /tmp/DesktopTest')
await clean.stdout.on('data', data => {
console.log(util.format('clean', data))
})
await app.client.close()
await app.stop()
})
util function,
// Returns a promise that resolves to a Spectron Application once the app has loaded.
// Takes a Ava test. Makes some basic assertions to verify that the app loaded correctly.
function createApp (t) {
return new Application({
path: path.join(__dirname, '..', 'node_modules', '.bin',
'electron' + (process.platform === 'win32' ? '.cmd' : '')),
// args: ['-r', path.join(__dirname, 'mocks.js'), path.join(__dirname, '..')],
env: {NODE_ENV: 'test'},
waitTimeout: 10e3
})
}
First off, Spectron (which is a wrapper for WebdriverIO) and WebdriverJS (which is part of Selenium-Webdriver) are two different frameworks, you only need to use one of them for your tests.
If you are using WebdriverJS, then you need to run ./node_modules/.bin/chromedriver in this step: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/#start-chromedriver
I could get ChromeDriver working by adding a proxy exception in my terminal.
export {no_proxy,NO_PROXY}="127.0.0.1"