Is it possible to run some tests in quarantine mode and run others without the mode?
For example, we have some lightweight tests and we don't need 3 tries for them, but for others it can be necessary
You can run two testcafe runners in series with different filters. One of them may be with the quarantine mode.
For example:
const createTestCafe = require('testcafe');
let testcafe = null;
const runTests = (testFiles, quarantineMode, testPrefix) => {
const runner = testcafe.createRunner();
return runner
.src(testFiles)
.filter((testName) => testName.startsWith(testPrefix))
.browsers(['chrome'])
.run({ quarantineMode });
};
createTestCafe('localhost', 1337, 1338)
.then(tc => {
testcafe = tc;
return runTests(['test.js'], false, 'Simple')
.then(() => runTests(['test.js'], true, 'Complex'));
})
.then(() => testcafe.close());
tests:
test('Simple Test', async t => {
//...
});
test('Complex Test', async t => {
//...
});
Related
Consider the following tests:
import { Selector, t } from 'testcafe'
const validSelector = Selector('html')
const invalidSelector = Selector('non-existent')
//the following test passes and establishes a baseline for the assertions below
test
('baseline', async t => {
await t
.expect(validSelector.exists).ok()
.expect(invalidSelector.exists).notOk()
})
//here the first set of assertions pass, while the second fail
test
('Promise.allSettled is misleading', async t => {
let selectors = [validSelector, validSelector, invalidSelector]
let expected = ['fulfilled', 'fulfilled', 'rejected']
await Promise.allSettled(selectors.map(async selector => await t.expect(selector.exists).ok()))
.then(async o => await t.expect(JSON.stringify(o.map(result => result.status)) === JSON.stringify(expected)).ok())
selectors = [validSelector, invalidSelector, validSelector]
expected = ['fulfilled', 'rejected', 'fulfilled']
await Promise.allSettled(selectors.map(async selector => await t.expect(selector.exists).ok()))
.then(async o => await t.expect(JSON.stringify(o.map(result => result.status)) === JSON.stringify(expected)).ok(JSON.stringify(o.map(result => result.status))))
})
const handleExceptions = async (selector) =>
await t.expect(selector.exists).ok()
.then(f => true)
.catch(r => false)
//here all of the promises are fulfilled due to the exception handling,
//but as one may expect the results are the same as in the previous test -- the assertion at the end fails
test
('Promise.allSettled with exception handling is still misleading', async t => {
const selectors = [validSelector, invalidSelector, validSelector]
const expected = [true, false, true]
await Promise.allSettled(selectors.map(async selector => await handleExceptions(selector)))
.then(async o => await t.expect(JSON.stringify(o) === JSON.stringify(expected)).ok(JSON.stringify(o)))
})
//in this test the assertion actually passes
test
('Snapshots seem like the only viable option', async t => {
const tags = ['html', 'non-existent', 'html']
const expected = [true, false, true]
await Promise.allSettled(tags.map(async tag => await Selector(tag)))
.then(async selectors => await Promise.allSettled(selectors.map(async s => await s.value())))
.then(snapshots => snapshots.map(snapshot => snapshot.value != null))
.then(async o => await t.expect(JSON.stringify(o) === JSON.stringify(expected)).ok(JSON.stringify(o)))
})
The question is -- is the described behaviour of Promise.allSettled in the cases above expected due to some perk of TestCafe or the usage above is in any way wrong?
In case it's the former, then should users take the snapshots route as the last test suggests or there are other viable options (maybe safer, faster, etc.)?
It seems this issue is related to the following open TestCafe issue: https://github.com/DevExpress/testcafe/issues/5623. Please vote for it and subscribe to notifications to learn about our progress.
In my Testcafe project I have multiple runners and I would like to have one reporter for all runners instead of logging the failed tests. Any suggestions?
Code below:
const testCafe = createTestCafe("localhost", 1337, 1338);
import util from "util";
(async () => {
try {
const runner1 = testCafe
.createRunner()
.src("testcafe/fixtures/first-fixture.js")
.reporter([{ name: "spec", output: "report1.txt" }])
.browsers("chrome");
const runner2 = testCafe
.createRunner()
.src("testcafe/fixtures/second-fixture.js")
.reporter([{ name: "spec", output: "report2.txt" }])
.browsers("chrome");
await Promise.all([runner1, runner2].map(runner => runner.run())).then(
result => {
console.log(`Failed tests: ${result[0] + result[1]}`);
}
);
} catch (error) {
await console.error(util.inspect(error));
} finally {
await testCafe.close();
}
})();
This is the code
const createTestCafe = require('testcafe');
let testcafe = null;
let connection = null;
const angulartests = [
'tests/Selector.test.ts'
]
const concurrency = 2;
var processFailed = false;
createTestCafe('localhost', 1337, 1338)
.then(async tc => {
testcafe = tc;
return testcafe.createBrowserConnection();
})
.then(async bc => {
connection = bc;
console.log(connection.url);
const runner = testcafe.createRunner();
const angulartestfailedcount = await runner
.src(angulartests)
// .browsers(['chrome'])
.browsers(connection)
//.{speed : (0.1) }
.screenshots('C:/dbm',true,'${DATE}_${TIME}/${FIXTURE}/${TEST}_${TEST_INDEX}.png')
// .concurrency(concurrency)
.reporter('xunit', 'C:/dbm/Labsession.xml').run();
return angulartestfailedcount;
})
.then(failedCount => {
console.log('Tests failed: ' + failedCount);
testcafe.close();
if (!failedCount)
process.exit(0);
process.exit(-1)
});
Result:
This is the URL generated:
http://localhost:1337/browser/connect/yX3Zt2j
Currently, to run in another machine's browser, we need to manually connect to that machine and run on the browser.
But Remote execution is something automatically runs on other Desktop.
In this case how to run the generated URL on other desktops without manual interruption.
At present, TestCafe doesn't have a such functionality. There is a suggestion for similar feature in the TestCafe repository on Github. Track it to be informed about progress.
I want to use sinon to write test code for the business logic of a service using ExpressJS and Mongoose.
I wrote the following test code, but findOneService takes only id as an argument and returns a document with that id.
//service_test.js
const sinon = require('sinon');
// Service
const { findOneService } = require('../services/service');
// Schema
const Post = require('../models/mongoose/schemas/post');
describe('findOneService', () => {
let find;
beforeEach(() => {
find = sinon.stub(Post, 'findOne');
});
afterEach(() => {
find.restore();
});
it('should findOne', async () => {
const id = ???;
...?
});
})
//service.js
exports.findOneDocument = async (id) => {
const result = await Post.findOne({_id: id});
if (!result) {
throw new Error('404');
}
return result;
};
How can I define the result of this to pass the test code?
To test this kind of behaviour, I strongly suggest an integration test (with an embedded/dockerized MongoDB, for example). This would allow to test-drive more things than just the service, such as schema, migration, db config.
However, if you just want to test-drive the if (!result)... logic, you can stick with sinon. What you're missing is stubbing the return value:
it('returns the document if found', async () => {
find.returns('a post');
expect(await findOneService.findOneDocument('id')).toReturn('a post');
});
it('throw error when document does not exist', async () => {
find.returns(null);
expect(() => await findOneService.findOneDocument('non-existent id')).toThrow(Error);
});
I am writing automated tests using Jest & Puppeteer for a Front-end application written in Vue.js
So far I managed to write a set of tests, but they all reside in the same file:
import puppeteer from 'puppeteer';
import faker from 'faker';
let page;
let browser;
const width = 860;
const height = 1080;
const homepage = 'http://localhost:8001/brt/';
const timeout = 1000 * 16;
beforeAll(async () => {
browser = await puppeteer.launch({
headless: false, // set to false if you want to see tests running live
slowMo: 30, // ms amount Puppeteer operations are slowed down by
args: [`--window-size=${width},${height}`],
});
page = await browser.newPage();
await page.setViewport({ width, height });
});
afterAll(() => {
browser.close();
});
describe('Homepage buttons', () => {
test('Gallery Button', async () => {
// navigate to the login view
await page.goto(homepage);
await page.waitFor(1000 * 0.5); // without this, the test gets stuck :(
await page.waitForSelector('[data-testid="navBarLoginBtn"]');
await page.click('[data-testid="navBarLoginBtn"]'),
await page.waitForSelector('[data-testid="navBarGalleryBtn"]');
await page.click('[data-testid="navBarGalleryBtn"]'),
// test: check if we got to the gallery view (by checking nr of tutorials)
await page.waitForSelector('.card-header');
const srcResultNumber = await page.$$eval('.card-header', (headers) => headers.length);
expect(srcResultNumber).toBeGreaterThan(1);
}, timeout);
});
describe('Register', () => {
const btnLoginToRegister = '#btn-login-to-register';
const btnRegister = '#btn-register';
const btnToLogin = '#btn-goto-login';
test('Register failed attempt: empty fields', async () => {
// navigate to the register form page via the login button
await page.goto(homepage);
await page.waitForSelector(navLoginBtn);
await page.click(navLoginBtn);
await page.waitForSelector(btnLoginToRegister);
await page.click(btnLoginToRegister);
// test; checking for error messages
await page.waitForSelector(btnRegister);
await page.click(btnRegister);
const errNumber = await page.$$eval('#errMessage', (err) => err.length);
expect(errNumber).toEqual(3);
}, timeout);
test('Register failed: invalid char count, email format', async () => {
// fill inputs
await page.waitForSelector('#userInput');
await page.type('#userInput', 'a');
await page.waitForSelector('#emailInput');
await page.type('#emailInput', 'a');
await page.waitForSelector('#emailInput');
await page.type('#passInput', 'a');
await page.waitForSelector(btnRegister);
await page.click(btnRegister);
// test: check if we 3 errors (one for each row), from the front end validations
const err = await page.$$eval('#errMessage', (errors) => errors.length);
expect(err).toEqual(3);
}, timeout);
test('Register: success', async () => {
await page.click('#userInput', { clickCount: 3 });
await page.type('#userInput', name1);
await page.click('#emailInput', { clickCount: 3 });
await page.type('#emailInput', email1);
await page.click('#passInput', { clickCount: 3 });
await page.type('#passInput', password1);
await page.waitForSelector(btnRegister);
await page.click(btnRegister);
// test: check if go to login link appeared
await page.waitForSelector(btnToLogin);
await page.click(btnToLogin);
// await Promise.all([
// page.click(btnToLogin),
// page.waitForNavigation(),
// ]);
}, timeout);
test('Register failed: email already taken', async () => {
// navigate back to the register form
await page.waitForSelector(btnLoginToRegister);
await page.click(btnLoginToRegister);
await page.click('#userInput');
await page.type('#userInput', name2);
await page.click('#emailInput');
await page.type('#emailInput', email1); // <- existing email
await page.click('#passInput');
await page.type('#passInput', password2);
await page.click(btnRegister);
const err = await page.$eval('#errMessage', (e) => e.innerHTML);
expect(err).toEqual('Email already taken');
}, timeout);
});
I would like to be able to have a single test file that does the beforeAll and afterAll stuff, and each test suite: HomepageButtons, Register, etc. to reside in it's own test file. How would I be able to achieve this?
I've tried splitting tets into:
testsUtils.js that would contain the beforeAll and afterAll hooks and code but it doesn't guarantee that it runs when it needs: the beforeAll code to fire before all other test files and the afterAll code to fire after all the test files finished.
Sorry, I'd rather comment on your question, but I don't have reputation for that. Anyway, I think that you are looking for something like a "global beforeAll" and "global afterAll" hooks, right? Jest has it alread. It's called "globalSetup" and "globalTeardown".
Take a look at globalSetup. Excerpt:
This option allows the use of a custom global setup module which
exports an async function that is triggered once before all test
suites.
The Global Teardown one goes the same.
I think you'll have a headache trying to get a reference to the page or browser in globalSetup/globalTeardown and I confess that I never try this. Maybe the answer for that problem (if you have it) is on this page, under "Custom example without jest-puppeteer preset section.
Also there is a repo that tries to facilitate Jest + Puppeteer integration. Maybe you find it util: repo.
Good luck. :)