I run detox in version 17.13.2 with jest-circus as the test runner. My main problem is the app is not reset either after or before I run the tests which leads to an inconsistent state of the app.
My test file:
import { by, device, element, expect, waitFor } from "detox"
describe("Login", () => {
beforeEach(async () => {
await device.reloadReactNative()
})
it("should login with correct data", async () => {
await element(by.id("login_email_input")).typeText("ch.tietz#gmail.com")
await element(by.id("login_password_input")).typeText("12345678")
await element(by.id("login_submit_button")).tap()
await waitFor(element(by.id("workout_screen"))).toBeVisible()
// additional test steps
})
})
Now if the login actually works but the test fails at one of the subsequent steps, the state of the app user will still be "logged in".
From what I understand, it's not possible to actually alter the app state, e.g. by clearing the AsyncStorage or interacting directly with the state mgmt tool. Instead, it's recommended to just reinstall the app - but this is exactly where I am struggling.
I have tried numerous approaches and none of them worked. What makes it really hard to understand configuration is that detox completely changed how the configuration works and switched to jest-circus as the main test runner.
My setup is basically the one created by jest init -r jest. From what I understand, this already includes some defaults for detox.init() and detox.cleanup():
{
"detox": {
"behavior": {
"init": {
"reinstallApp": true,
"launchApp": true,
"exposeGlobals": true
},
"cleanup": {
"shutdownDevice": false
}
}
}
}
However, this does not seem to be sufficient to actually wipe the app state after running the tests.
I tried working with an init script as setupFilesAfterEnv, which would call cleanup() after the test suite is run. Actually that works in an older project which still uses jasmine 2 as test runner.
import { cleanup, init } from 'detox';
const adapter = require('detox/runners/jest/adapter');
const specReporter = require('detox/runners/jest/specReporter');
const config = require('../package.json').detox;
// Set the default timeout
jest.setTimeout(120000);
jasmine.getEnv().addReporter(adapter);
// This takes care of generating status logs on a per-spec basis. By default, jest only reports at file-level.
// This is strictly optional.
jasmine.getEnv().addReporter(specReporter);
beforeAll(async () => {
await init(config, { launchApp: false });
}, 300000);
beforeEach(async () => {
await adapter.beforeEach();
});
afterAll(async () => {
await adapter.afterAll();
await cleanup();
});
First off, it complains that jasmine is not defined. I guess that's because actually the adapter in this case should be a DetoxAdapterCircus which it is not, even though in my config file I specify:
{
"preset": "react-native",
"testEnvironment": "./environment.ts",
"testRunner": "jest-circus/runner",
"testTimeout": 120000,
"testRegex": "\\.e2e\\.ts$",
"reporters": ["detox/runners/jest/streamlineReporter"],
"verbose": true
}
"testRunner": "jest-circus/runner"
Another idea would be to alter the CustomDetoxEnvironment but I do not understand how I can get access to the detox lifecycle hooks.
const {
DetoxCircusEnvironment,
SpecReporter,
WorkerAssignReporter,
} = require("detox/runners/jest-circus")
class CustomDetoxEnvironment extends DetoxCircusEnvironment {
constructor(config) {
super(config)
// should I access the hooks here now?
// This takes care of generating status logs on a per-spec basis. By default, Jest only reports at file-level.
// This is strictly optional.
this.registerListeners({
SpecReporter,
WorkerAssignReporter,
})
}
}
module.exports = CustomDetoxEnvironment
Tl;dr: I don't know where to put my reusable lifecycle hooks in the latest version of Detox. Also, I wonder if these custom configurations are even needed to reinstall the app and wipe the app data before each test suite.
You can still use setupFilesAfterEnv, but you're not supposed to call the same hooks/cleanup as before (see https://github.com/wix/Detox/issues/2410#issuecomment-744387707).
here's an example of our init script:
// init.ts
// with "setupFilesAfterEnv": ["./init.ts"] in the conf
beforeAll(async () => {
// to reset the state
await device.clearKeychain();
// we are launching the app manually
await device.launchApp({
permissions: { notifications: 'YES', location: 'inuse' },
});
await device.setURLBlacklist([
// ...
]);
});
Related
Hello im searching for a way to use component testing as well as end to end testing with nuxt.
we want to be able to test components (which already works) and also check if pages parse their url parameters correctly or sitemaps are correctly created and other page level features and router functions
i tried ava but we already implemented the component testing with jest which works fine now and in the nuxt docs the server rendering for testing was described with ava and i adapted that to jest now but i get timeout errors so i increased the time out to 40 seconds but still get a timeout.
did anybody get the testing to work with the nuxt builder like in the example (https://nuxtjs.org/guide/development-tools)?
this is my end to end test example file
// test.spec.js:
const { resolve } = require('path')
const { Nuxt, Builder } = require('nuxt')
// We keep the nuxt and server instance
// So we can close them at the end of the test
let nuxt = null
// Init Nuxt.js and create a server listening on localhost:4000
beforeAll(async (done) => {
jest.setTimeout(40000)
const config = {
dev: false,
rootDir: resolve(__dirname, '../..'),
telemetry: false,
}
nuxt = new Nuxt(config)
try {
await new Builder(nuxt).build()
nuxt.server.listen(4000, 'localhost')
} catch (e) {
console.log(e)
}
done()
}, 30000)
describe('testing nuxt', () => {
// Example of testing only generated html
test('Route / exits and render HTML', async (t, done) => {
const context = {}
const { html } = await nuxt.server.renderRoute('/', context)
t.true(html.includes('<h1 class="red">Hello world!</h1>'))
jest.setTimeout(30000)
done()
})
})
// Close server and ask nuxt to stop listening to file changes
afterAll((t) => {
nuxt.close()
})
my current error is :
● Test suite failed to run
Timeout - Async callback was not invoked within the 40000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 40000ms timeout specified by jest.setTimeout.
any info is very appreciated as i could not resolve this issue myself
I am trying to write a tests with Spectron for our Electron App, but I am running into problems with the setup. I use the classical setup with chai. I have one file which contains setup code:
const path = require("path");
const { Application } = require("spectron");
module.exports = {
initialiseSpectron: () => {
let electronPath = path.join(__dirname, "../../node_modules", ".bin", "electron");
if (process.platform == "win32") {
electronPath += ".cmd";
}
return new Application({
path: electronPath,
args: [path.join(__dirname, "../index.ts"), path.join(__dirname, "../../package.json")],
env: {
ELECTRON_ENABLE_LOGGING: true,
ELECTRON_ENABLE_STACK_DUMPING: true,
NODE_ENV: "development"
},
startTimeout: 10000,
chromeDriverLogPath: "../chromedriverlog.txt"
});
},
sleep: time => new Promise(resolve => setTimeout(resolve, time))
};
And then the test itself:
const chaiAsPromised = require("chai-as-promised");
const chai = require("chai");
chai.should();
chai.use(chaiAsPromised);
const testHelper = require("./initialise");
const app = testHelper.initialiseSpectron();
// Setup Promises and start Application
before(() => app.start());
// Tear down App after Tests are finished
after(() => {
if (app && app.isRunning()) {
return app.stop();
}
});
describe("Login", () => {
it("opens a window", function() {
return app.client
.waitUntilWindowLoaded()
.getWindowCount()
.should.eventually.equal(1);
});
it("tests the title", () =>
app.client
.waitUntilWindowLoaded()
.getTitle()
.should.eventually.equal("VIPFY"));
});
My problem is that I always get this error:
1) "before all" hook in "{root}"
0 passing (2s)
1 failing
1) "before all" hook in "{root}":
Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
So it looks like the App does not start. But that is not true. The App Window opens, but it seems like the test does not recognize that. I have already tried changing the path using all kinds of syntax. But nothing worked. What am I missing?
Have you tried to increase the timeout for mocha?
Sometimes I had it fail first time, then worked on the second try.
See a working sample here with Electron 6:
https://github.com/florin05/electron-spectron-example
Given the example official Nuxt end-to-end test example using Ava:
import test from 'ava'
import { Nuxt, Builder } from 'nuxt'
import { resolve } from 'path'
// We keep a reference to Nuxt so we can close
// the server at the end of the test
let nuxt = null
// Init Nuxt.js and start listening on localhost:4000
test.before('Init Nuxt.js', async t => {
const rootDir = resolve(__dirname, '..')
let config = {}
try { config = require(resolve(rootDir, 'nuxt.config.js')) } catch (e) {}
config.rootDir = rootDir // project folder
config.dev = false // production build
config.mode = 'universal' // Isomorphic application
nuxt = new Nuxt(config)
await new Builder(nuxt).build()
nuxt.listen(4000, 'localhost')
})
// Example of testing only generated html
test('Route / exits and render HTML', async t => {
let context = {}
const { html } = await nuxt.renderRoute('/', context)
t.true(html.includes('<h1 class="red">Hello world!</h1>'))
})
// Close the Nuxt server
test.after('Closing server', t => {
nuxt.close()
})
How can you use Nuxt or Builder to configure/access the applications Vuex store? The example Vuex store would look like:
import Vuex from "vuex";
const createStore = () => {
return new Vuex.Store({
state: () => ({
todo: null
}),
mutations: {
receiveTodo(state, todo) {
state.todo = todo;
}
},
actions: {
async nuxtServerInit({ commit }, { app }) {
console.log(app);
const todo = await app.$axios.$get(
"https://jsonplaceholder.typicode.com/todos/1"
);
commit("receiveTodo", todo);
}
}
});
};
export default createStore;
Currently trying to run the provided Ava test, leads to an error attempting to access #nuxtjs/axios method $get:
TypeError {
message: 'Cannot read property \'$get\' of undefined',
}
I'd be able to mock $get and even $axios available on app in Vuex store method nuxtServerInit, I just need to understand how to access app in the test configuration.
Thank you for any help you can provide.
Just encountered this and after digging so many tutorial, I pieced together a solution.
You have essentially import your vuex store into Nuxt when using it programmatically. This is done by:
Importing Nuxt's config file
Adding to the config to turn off everything else but enable store
Load the Nuxt instance and continue your tests
Here's a working code (assuming your ava and dependencies are set up)
// For more info on why this works, check this aweomse guide by this post in getting this working
// https://medium.com/#brandonaaskov/how-to-test-nuxt-stores-with-jest-9a5d55d54b28
import test from 'ava'
import jsdom from 'jsdom'
import { Nuxt, Builder } from 'nuxt'
import nuxtConfig from '../nuxt.config' // your nuxt.config
// these boolean switches turn off the build for all but the store
const resetConfig = {
loading: false,
loadingIndicator: false,
fetch: {
client: false,
server: false
},
features: {
store: true,
layouts: false,
meta: false,
middleware: false,
transitions: false,
deprecations: false,
validate: false,
asyncData: false,
fetch: false,
clientOnline: false,
clientPrefetch: false,
clientUseUrl: false,
componentAliases: false,
componentClientOnly: false
},
build: {
indicator: false,
terser: false
}
}
// We keep a reference to Nuxt so we can close
// the server at the end of the test
let nuxt = null
// Init Nuxt.js and start listening on localhost:5000 BEFORE running your tests. We are combining our config file with our resetConfig using Object.assign into an empty object {}
test.before('Init Nuxt.js', async (t) => {
t.timeout(600000)
const config = Object.assign({}, nuxtConfig, resetConfig, {
srcDir: nuxtConfig.srcDir, // don't worry if its not in your nuxt.config file. it has a default
ignore: ['**/components/**/*', '**/layouts/**/*', '**/pages/**/*']
})
nuxt = new Nuxt(config)
await new Builder(nuxt).build()
nuxt.listen(5000, 'localhost')
})
// Then run our tests using the nuxt we defined initially
test.serial('Route / exists and renders correct HTML', async (t) => {
t.timeout(600000) // Sometimes nuxt's response is slow. We increase the timeont to give it time to render
const context = {}
const { html } = await nuxt.renderRoute('/', context)
t.true(html.includes('preload'))
// t.true(true)
})
test.serial('Route / exits and renders title', async (t) => {
t.timeout(600000)
const { html } = await nuxt.renderRoute('/', {})
const { JSDOM } = jsdom // this was the only way i could get JSDOM to work. normal import threw a functione error
const { document } = (new JSDOM(html)).window
t.true(document.title !== null && document.title !== undefined) // simple test to check if site has a title
})
Doing this should work. HOWEVER, You may still get some errors
✖ Timed out while running tests. If you get this you're mostly out of luck. I thought the problem was with Ava given that it didn't give a descriptive error (and removing any Nuxt method seemed to fix it), but so far even with the above snippet sometimes it works and sometimes it doesn't.
My best guess at this time is that there is a delay on Nuxt's side using either renderRouter or renderAndGetWindow that ava doesn't wait for, but on trying any of these methods ava almost immediately "times out" despite the t.timeout being explicitly set for each test. So far my research has lead me to checking the timeout for renderAndGetWindow (if it exists, but the docs doesn't indicate such).
That's all i've got.
I have read the documentation and I have followed the tutorial step by step and I only have managed to run the app.
Documentation: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/
The connection with chromedriver I cannot make it work, when I launch the test and try click a simple button I get this:
Error: ChromeDriver did not start within 5000ms at Error (native)
at node_modules/spectron/lib/chrome-driver.js:58:25 at
Request._callback (node_modules/spectron/lib/chrome-driver.js:116:45)
at Request.self.callback
(node_modules/spectron/node_modules/request/request.js:200:22) at
Request.
(node_modules/spectron/node_modules/request/request.js:1067:10) at
IncomingMessage.
(node_modules/spectron/node_modules/request/request.js:988:12) at
endReadableNT (_stream_readable.js:913:12) at _combinedTickCallback
(internal/process/next_tick.js:74:11) at process._tickCallback
(internal/process/next_tick.js:98:9)
My code:
"use strict";
require("co-mocha");
var Application = require('spectron').Application;
var assert = require('assert');
const webdriver = require('selenium-webdriver');
const driver = new webdriver.Builder()
.usingServer('http://127.0.0.1:9515')
.withCapabilities({
chromeOptions: {
binary: "./appPath/app"
}
})
.forBrowser('electron')
.build();
describe('Application launch', function () {
this.timeout(100000);
var app;
beforeEach(function () {
app = new Application({
path: "./appPath/app"
});
return app.start();
});
afterEach(function () {
if (app && app.isRunning()) {
return app.stop();
}
});
it('click a button', function* () {
yield driver.sleep(5000);
yield driver.findElement(webdriver.By.css(".classSelector")).click();
});
});
Thanks and sorry for my English.
I recommend you to use Spectron. which is a less painful way of testing your electron app. in my opinion perfect combination is using it with Ava test framework, which allows the concurrently test.
async & await is also another big win. which allows you to have so clean test cases.
and also if you have a test which needs to happen serial, you can use test.serial
test.serial('login as new user', async t => {
let app = t.context.app
app = await loginNewUser(app)
await util.screenshotCreateOrCompare(app, t, 'new-user-mission-view-empty')
})
test.serial('Can Navigate to Preference Page', async t => {
let app = t.context.app
await app.client.click('[data-test="preference-button"]')
await util.screenshotCreateOrCompare(app, t, 'new-user-preference-page-empty')
})
and just for reference; my helper test cases.
test.before(async t => {
app = util.createApp()
app = await util.waitForLoad(app, t)
})
test.beforeEach(async t => {
t.context.app = app
})
test.afterEach(async t => {
console.log('test complete')
})
// CleanUp
test.after.always(async t => {
// This runs after each test and other test hooks, even if they
failed
await app.client.localStorage('DELETE', 'user')
console.log('delete all files')
const clean = await exec('rm -rf /tmp/DesktopTest')
await clean.stdout.on('data', data => {
console.log(util.format('clean', data))
})
await app.client.close()
await app.stop()
})
util function,
// Returns a promise that resolves to a Spectron Application once the app has loaded.
// Takes a Ava test. Makes some basic assertions to verify that the app loaded correctly.
function createApp (t) {
return new Application({
path: path.join(__dirname, '..', 'node_modules', '.bin',
'electron' + (process.platform === 'win32' ? '.cmd' : '')),
// args: ['-r', path.join(__dirname, 'mocks.js'), path.join(__dirname, '..')],
env: {NODE_ENV: 'test'},
waitTimeout: 10e3
})
}
First off, Spectron (which is a wrapper for WebdriverIO) and WebdriverJS (which is part of Selenium-Webdriver) are two different frameworks, you only need to use one of them for your tests.
If you are using WebdriverJS, then you need to run ./node_modules/.bin/chromedriver in this step: http://electron.atom.io/docs/tutorial/using-selenium-and-webdriver/#start-chromedriver
I could get ChromeDriver working by adding a proxy exception in my terminal.
export {no_proxy,NO_PROXY}="127.0.0.1"
I'm trying to build a functional testing system to verify our web site is behaving correctly for our users. I have cobbled together a bunch of Node.js modules and helpers in an attempt to get a framework that provides simple, concise tests without a heap of nested function callbacks and I believe promises can provide that, so my package.json file looks like this:
"dependencies": {
"chai-as-promised": "^4.3.0",
"grunt": "^0.4.5",
"grunt-webdriver": "^0.4.8"
}
My Gruntfile.js looks like this:
module.exports = function(grunt) {
grunt.initConfig({
webdriver: { // for use with webdriver.io
options: {
desiredCapabilities: {
browserName: 'phantomjs' // No Xvfb required
}
},
chrome: {
tests: ['chrome/*.js'],
options: {
desiredCapabilities: {
browserName: 'chrome'
}
}
},
},
});
grunt.loadNpmTasks('grunt-webdriver');
grunt.registerTask('default', ['webdriver']);
};
And finally my test case in chrome/login.js looks like this:
'use strict';
var chai = require('chai'),
chaiAsPromised = require('chai-as-promised'),
assert;
chaiAsPromised.transferPromiseness = browser.transferPromiseness;
chai.use(chaiAsPromised);
assert = chai.assert;
describe('login test', function () {
it('verifies user can log in', function(done) {
browser
.url('https://localhost/')
.setValue('#userauth_username','foo')
.setValue('#userauth_password',"password")
.submitForm('#form_users_login')
.then(function(){
browser.getText('#auth-user-id', function(err, value){
console.log(value);
});
assert.becomes(browser.getText('#auth-user-id'), 'foo');
})//.call(done);
});
});
When I run grunt webdriver:chrome on the command line, I see it start up Chrome and log into the website. The 'auth-user-id' span is correctly displaying the user's id after logging in but for some reason browser.getText() is not returning it and the test is therefore failing. I have tried adding a .pause(100) after the .submitForm() to give me time to interact with the page in Chrome so I know it is a problem in the test case.
What am I doing wrong?
This seems to be the best and most succinct way of doing what I want. I'm not sure I need chai-as-promised yet but maybe I'll move the login function to an included file and use chai-as-promised to assert that the promised login has occurred before entering the tests.
'use strict';
var chai = require('chai'),
chaiAsPromised = require('chai-as-promised'),
assert,
expect;
chaiAsPromised.transferPromiseness = browser.transferPromiseness;
chai.use(chaiAsPromised);
assert = chai.assert;
expect = chai.expect;
describe('login test', function () {
it('verifies user can log in', function(done) {
browser
.url('https://localhost/')
.setValue('#userauth_username','foo')
.setValue('#userauth_password',"password")
.submitForm('#form_users_login')
.waitForExist('#auth-user-id')
.getText('#auth-user-id')
.then(function(text){
//console.log('Username: ' + text);
assert.equal(text, 'foo');
})
.saveScreenshot('out.png')
.call(done)
});
it('should not display logincontrols after login', function(done){
browser
.isVisible('#logincontrols')
.then(function(bool){
expect(bool).to.be.false;
})
.call(done)
});
it('should display loggedin section after login', function(done){
browser
.isVisible('#loggedin')
.then(function(bool){
expect(bool).to.be.true;
})
.call(done)
});
});
and for completeness, this is what I see on the output:
# grunt webdriver:chrome
Running "webdriver:chrome" (webdriver) task
login test
✓ verifies user can log in (7691ms)
✓ should not display logincontrols after login (70ms)
✓ should display loggedin section after login (58ms)
3 passing (8s)
Done, without errors.