Could somebody help with finding the most adequate way to run the same fixture with different preconditions using testcafe (fixture beforeEach)?
Given I have following fixture:
fixture('[Test] My Page')
.meta({ env: 'aat', mobile: 'true', author: 'me' })
.beforeEach(async t => {
await t.navigateTo(myPage.url)
await waitForReact(10000)
})
test('Page should load all data successfully', async t => {
const { Query: queries } = await t.ctx.graphQLTools.mockManagementClient.getCalls()
for (const q of Object.keys(queries)) {
for (const { args, context } of Object.keys(queries[q])) {
await t.expect(queries[q][args]).typeOf('undefined')
await t.expect(queries[q][context]).typeOf('undefined')
}
}
})
In the code example above I need to run this test for several times:
I'm an anonymous user
I'm the admin user
I'm a default user
I've tried the following code, but it looks cumbersome and will create complexity when I have more than 1 test in the fixture that needs to be checked(taken from https://testcafe-discuss.devexpress.com/t/multiple-execution-of-one-test-with-different-data/219):
const a = ['anonymous', 'logged In']
for (const user of a) {
test.meta({ wip: 'true' })(`Page should load all data successfully for ${user} user`, async t => {
if (R.equals('logged In', user)) {
await helpers.setCookie({ key: 'access_token', value: '123123123' })
await helpers.reloadPage()
}
const { Query: queries } = await t.ctx.graphQLTools.mockManagementClient.getCalls()
await t.expect(helpers.isEqlToEmptyQuery(queries.basket)).eql(true)
await t.expect(helpers.isEqlToEmptyQuery(queries.categories)).eql(true)
if (R.equals('logged In', user)) {
await t.expect(helpers.isEqlToEmptyQuery(queries.customer)).eql(true)
}
})
}
Is there a way I can run the whole fixture for 2-3 times with different beforeEach fixture hooks?
One simple solution could be to add three npm scripts in package.json:
"test": "testcafe chrome <all testcafe options>"
"test-anonymous": "npm test -- --user=anonymous"
"test-admin": "npm test -- --user=admin"
"test-default": "npm test -- --user=default"
Then read this custom command-line option in the beforeEach as I explained in Testcafe - Test command line argument outside test case
Related
I run detox in version 17.13.2 with jest-circus as the test runner. My main problem is the app is not reset either after or before I run the tests which leads to an inconsistent state of the app.
My test file:
import { by, device, element, expect, waitFor } from "detox"
describe("Login", () => {
beforeEach(async () => {
await device.reloadReactNative()
})
it("should login with correct data", async () => {
await element(by.id("login_email_input")).typeText("ch.tietz#gmail.com")
await element(by.id("login_password_input")).typeText("12345678")
await element(by.id("login_submit_button")).tap()
await waitFor(element(by.id("workout_screen"))).toBeVisible()
// additional test steps
})
})
Now if the login actually works but the test fails at one of the subsequent steps, the state of the app user will still be "logged in".
From what I understand, it's not possible to actually alter the app state, e.g. by clearing the AsyncStorage or interacting directly with the state mgmt tool. Instead, it's recommended to just reinstall the app - but this is exactly where I am struggling.
I have tried numerous approaches and none of them worked. What makes it really hard to understand configuration is that detox completely changed how the configuration works and switched to jest-circus as the main test runner.
My setup is basically the one created by jest init -r jest. From what I understand, this already includes some defaults for detox.init() and detox.cleanup():
{
"detox": {
"behavior": {
"init": {
"reinstallApp": true,
"launchApp": true,
"exposeGlobals": true
},
"cleanup": {
"shutdownDevice": false
}
}
}
}
However, this does not seem to be sufficient to actually wipe the app state after running the tests.
I tried working with an init script as setupFilesAfterEnv, which would call cleanup() after the test suite is run. Actually that works in an older project which still uses jasmine 2 as test runner.
import { cleanup, init } from 'detox';
const adapter = require('detox/runners/jest/adapter');
const specReporter = require('detox/runners/jest/specReporter');
const config = require('../package.json').detox;
// Set the default timeout
jest.setTimeout(120000);
jasmine.getEnv().addReporter(adapter);
// This takes care of generating status logs on a per-spec basis. By default, jest only reports at file-level.
// This is strictly optional.
jasmine.getEnv().addReporter(specReporter);
beforeAll(async () => {
await init(config, { launchApp: false });
}, 300000);
beforeEach(async () => {
await adapter.beforeEach();
});
afterAll(async () => {
await adapter.afterAll();
await cleanup();
});
First off, it complains that jasmine is not defined. I guess that's because actually the adapter in this case should be a DetoxAdapterCircus which it is not, even though in my config file I specify:
{
"preset": "react-native",
"testEnvironment": "./environment.ts",
"testRunner": "jest-circus/runner",
"testTimeout": 120000,
"testRegex": "\\.e2e\\.ts$",
"reporters": ["detox/runners/jest/streamlineReporter"],
"verbose": true
}
"testRunner": "jest-circus/runner"
Another idea would be to alter the CustomDetoxEnvironment but I do not understand how I can get access to the detox lifecycle hooks.
const {
DetoxCircusEnvironment,
SpecReporter,
WorkerAssignReporter,
} = require("detox/runners/jest-circus")
class CustomDetoxEnvironment extends DetoxCircusEnvironment {
constructor(config) {
super(config)
// should I access the hooks here now?
// This takes care of generating status logs on a per-spec basis. By default, Jest only reports at file-level.
// This is strictly optional.
this.registerListeners({
SpecReporter,
WorkerAssignReporter,
})
}
}
module.exports = CustomDetoxEnvironment
Tl;dr: I don't know where to put my reusable lifecycle hooks in the latest version of Detox. Also, I wonder if these custom configurations are even needed to reinstall the app and wipe the app data before each test suite.
You can still use setupFilesAfterEnv, but you're not supposed to call the same hooks/cleanup as before (see https://github.com/wix/Detox/issues/2410#issuecomment-744387707).
here's an example of our init script:
// init.ts
// with "setupFilesAfterEnv": ["./init.ts"] in the conf
beforeAll(async () => {
// to reset the state
await device.clearKeychain();
// we are launching the app manually
await device.launchApp({
permissions: { notifications: 'YES', location: 'inuse' },
});
await device.setURLBlacklist([
// ...
]);
});
I'm having some issues where I am stuck on what to do to be able to tag different tests that I want to run.
The tests that I want is only required at the end where I want to be able to tag which payment I would like to pay which is Mastercard, Visa or Amex. I do have a test that does the details page such as writing users information, choose shipment but then at the end I do have multiply options that I would like to test pending on what I want to test:
paymentPage.js
describe('Payment page', function () {
paymentPage = new PaymentPage();
// The details page is accessible by the specified URL
it(`Credit Card - Has a form that can receive user data`, async function () {
await paymentPage.creditCardPayment();
});
//Visa
it('Enets payment', async function () {
//TODO
});
//Amex
it('Enets payment', async function () {
//TODO
});
});
As you can see there is 3 options that I would like to test so whenever I do etc "protractor e2e run mastercard" so that means it should run the first test case, if I choose visa then do the second test case and skip the rest basically.
However I do have couple of tests that executes before coming to the payment page but they all need to do the same for each payment so meaning that all test cases before payment needs to do exactly the same thing everytime (So I guess we just need to do tags at the payment to let the script know which payment to choose)?
How can I do a sort of tagging or maybe someone has another solution that is better? That I can be able to choose what payment provider I want to run
edit:
exports.config = {
capabilities: {
browserName: 'chrome',
chromeOptions: {
args: [
'incognito', 'disable-extensions', 'start-maximized', 'disable-infobars', '--window-size=1920,1080'
]
},
loggingPrefs: { browser: 'ALL' },
platform: 'ANY',
version: ''
},
specs: [
'pagesDesktop/testPage.js',
'pagesDesktop/paymentPage.js'
],
jasmineNodeOpts: {
reporter: "mochawesome",
defaultTimeoutInterval: 60000
},
SELENIUM_PROMISE_MANAGER: false,
framework: 'jasmine',
params: {
cardType: {
}
}
};
describe('Payment page', function () {
paymentPage = new PaymentPage();
console.log(browser.params.cardType);
if (browser.params.cardType === "mastercard") {
// The details page is accessible by the specified URL
it(`Credit Card - Has a form that can receive user data`, async function () {
await paymentPage.creditCardPayment();
});
}
just add if/else logic
describe('Payment page', function () {
paymentPage = new PaymentPage();
// The details page is accessible by the specified URL
it(`Credit Card - Has a form that can receive user data`, async function () {
await paymentPage.creditCardPayment();
});
if (browser.params.cardType === 'visa') {
it('Enets payment', async function () {
//TODO
});
} else if (browser.params.cardType === 'amex') {
//Amex
it('Enets payment', async function () {
//TODO
});
}
});
You can read how to parameterize teste here How can I use command line arguments in Angularjs Protractor?, or here
https://medium.com/#nicklee1/making-your-protractor-tests-data-driven-e3c9e2a5e4e7
We are using Cypress.io to build our automation suite. We have a requirement to seed our database before every test and to clear the data afterward. This could be done like below.
describe('Our test suite', function() {
before(function () {
//loadDbSeed is a custom command that will load the seed file based on the spec file
seed = cy.loadDbSeed()
cy.task('seed:db', seed)
})
it('should be true', function() {
//Some test with some action followed by an assertion
cy.visit('/some-page')
cy.get('[data-cy="identifier"]')
.click()
expect(true).to.equal(true)
})
after(function () {
// clearDb is a custom command that will clear out the DB.
// We are still debating if we must clear the DB after the tests.
// But we might still need to do some common actions for the entire suite after
cy.clearDb()
})
})
The problem we see is that the same before and after operations will be required for all our test suites. So we would like to override these methods so that our tests are something like this.
describe('Our test suite', function() {
before(function () {
// DB seeding is done automatically
// Write only custom before steps required for this test
})
it('should be true', function() {
//Some test with some action followed by an assertion
cy.visit('/some-page')
cy.get('[data-cy="identifier"]')
.click()
expect(true).to.equal(true)
})
after(function () {
// DB clearing is done automatically
// Write only custom after steps required for this test
})
})
How do we achieve this? I have been digging around in the Cypress code and haven't found anything obvious.
If you look at the Cypress docs, using after isn't recommended - Cypress Docs. I'd caution against setting data globally, will you really need it for every test? If you need to enter data on a per-test basis at some point, will that conflict with this global data? You could do something like this on a per test basis:
describe('Our test suite', function() {
beforeEach(function () {
cy.then(async () => {
await MyDatabaseService.resetdata()
await MyDatabaseService.createSomeData()
});
});
it('should be true', function() {
//Some test with some action followed by an assertion
})
})
I've also had to nest some tests as follows when specific tests needed specific data (sorry if some of the formatting here is a bit out, hopefully it'll make sense!):
describe('Our test suite', function() {
beforeEach(function () {
cy.then(async () => {
await MyDatabaseService.resetdata()
await MyDatabaseService.createSomeData()
});
});
it('should be true', function() {
//Some test with some action followed by an assertion
});
describe('These tests need more data, this is a nested describe', function () {
before(function () {
cy.then(async () => {
await MyDatabaseService.addSomeMoreData()
});
it('this test uses the extra data', function () {
// Do some funky tests here
});
});
});
})
For the second tests above, the test will run all three database actions.
Upshot is, if you clear data before you run your tests then it makes things much clearer.
I hope that helps. I'm new to Cypress myself and it can be hard to drop that (bad?) habits we've used for some time in Selenium!
I have this cypress e2e tests.
/// <reference types="Cypress" />
describe('Toss Full Test', function () {
let polyfill;
const uuid = Cypress._.random(0, 1e6)
before(() => {
const polyfillUrl = 'https://unpkg.com/whatwg-fetch#3.0.0/dist/fetch.umd.js';
cy.request(polyfillUrl).then(response => {
polyfill = response.body;
});
});
Cypress.on('window:before:load', win => {
delete win.fetch;
win.eval(polyfill);
});
const SubscribeEmail = "tosstests" + uuid + "#yopmail.com";
const SubscribePassword = "tossTests123456!!";
const SubscribeLogin = "tosstests" + uuid;
it('Full process', function (win) {
cy.server();
cy.visit("/");
cy.route('POST', '/api/account/register').as('register');
//this could be lagging as ravendb is starting
cy.get("#LinkLogin", { timeout: 20000 }).click();
//register
cy.get("#LinkRegister").click();
cy.get("#NewEmail").type(SubscribeEmail);
cy.get("#NewName").type(SubscribeLogin);
cy.get("#NewPassword").type(SubscribePassword);
cy.get("#NewConfirmPassword").type(SubscribePassword);
cy.window()
.then(win => {
// disables runCaptcha
win.runCaptcha = new win.Function(['action'], 'return Promise.resolve(action)');
})
.then(
() => {
cy.get("#BtnRegister").click();
cy.wait('#register');
cy.get('#register').then(function (xhr) {
expect(xhr.status).to.eq(200);
});
}
);
})
})
The full code can be found here https://github.com/RemiBou/Toss.Blazor/tree/master/Toss.Tests.E2E.Cypress.
When I run my project with the following code
docker-compose up -V ravendb web
./node_modules/.bin/cypress open
The code runs well, the assertion "expect(xhr.status).to.eq(200);" returns true but the test execution never stops. Why is that ?
After a lot of trial and error I just needed to remove the "win" parameter from the method in the "if" call which was a test for getting a window reference. In fact this parameter is a callback you need to call at the end of the tests, I can't find any documentation about it.
I have read the documentation and I have followed the tutorial step by step and I only have managed to run the app.
Documentation: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/
The connection with chromedriver I cannot make it work, when I launch the test and try click a simple button I get this:
Error: ChromeDriver did not start within 5000ms at Error (native)
at node_modules/spectron/lib/chrome-driver.js:58:25 at
Request._callback (node_modules/spectron/lib/chrome-driver.js:116:45)
at Request.self.callback
(node_modules/spectron/node_modules/request/request.js:200:22) at
Request.
(node_modules/spectron/node_modules/request/request.js:1067:10) at
IncomingMessage.
(node_modules/spectron/node_modules/request/request.js:988:12) at
endReadableNT (_stream_readable.js:913:12) at _combinedTickCallback
(internal/process/next_tick.js:74:11) at process._tickCallback
(internal/process/next_tick.js:98:9)
My code:
"use strict";
require("co-mocha");
var Application = require('spectron').Application;
var assert = require('assert');
const webdriver = require('selenium-webdriver');
const driver = new webdriver.Builder()
.usingServer('http://127.0.0.1:9515')
.withCapabilities({
chromeOptions: {
binary: "./appPath/app"
}
})
.forBrowser('electron')
.build();
describe('Application launch', function () {
this.timeout(100000);
var app;
beforeEach(function () {
app = new Application({
path: "./appPath/app"
});
return app.start();
});
afterEach(function () {
if (app && app.isRunning()) {
return app.stop();
}
});
it('click a button', function* () {
yield driver.sleep(5000);
yield driver.findElement(webdriver.By.css(".classSelector")).click();
});
});
Thanks and sorry for my English.
I recommend you to use Spectron. which is a less painful way of testing your electron app. in my opinion perfect combination is using it with Ava test framework, which allows the concurrently test.
async & await is also another big win. which allows you to have so clean test cases.
and also if you have a test which needs to happen serial, you can use test.serial
test.serial('login as new user', async t => {
let app = t.context.app
app = await loginNewUser(app)
await util.screenshotCreateOrCompare(app, t, 'new-user-mission-view-empty')
})
test.serial('Can Navigate to Preference Page', async t => {
let app = t.context.app
await app.client.click('[data-test="preference-button"]')
await util.screenshotCreateOrCompare(app, t, 'new-user-preference-page-empty')
})
and just for reference; my helper test cases.
test.before(async t => {
app = util.createApp()
app = await util.waitForLoad(app, t)
})
test.beforeEach(async t => {
t.context.app = app
})
test.afterEach(async t => {
console.log('test complete')
})
// CleanUp
test.after.always(async t => {
// This runs after each test and other test hooks, even if they
failed
await app.client.localStorage('DELETE', 'user')
console.log('delete all files')
const clean = await exec('rm -rf /tmp/DesktopTest')
await clean.stdout.on('data', data => {
console.log(util.format('clean', data))
})
await app.client.close()
await app.stop()
})
util function,
// Returns a promise that resolves to a Spectron Application once the app has loaded.
// Takes a Ava test. Makes some basic assertions to verify that the app loaded correctly.
function createApp (t) {
return new Application({
path: path.join(__dirname, '..', 'node_modules', '.bin',
'electron' + (process.platform === 'win32' ? '.cmd' : '')),
// args: ['-r', path.join(__dirname, 'mocks.js'), path.join(__dirname, '..')],
env: {NODE_ENV: 'test'},
waitTimeout: 10e3
})
}
First off, Spectron (which is a wrapper for WebdriverIO) and WebdriverJS (which is part of Selenium-Webdriver) are two different frameworks, you only need to use one of them for your tests.
If you are using WebdriverJS, then you need to run ./node_modules/.bin/chromedriver in this step: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/#start-chromedriver
I could get ChromeDriver working by adding a proxy exception in my terminal.
export {no_proxy,NO_PROXY}="127.0.0.1"