Detox using jest timeout error: "is assigned to undefined" - react-native

I'm trying to add detox using jest and jest-circus to my ReactNative app but I'm currently struggling making it work. Detox, jest and jest-circus have been added from scratch at the latest version.
When launching my test, after a successful build, it launches the simulator but hangs at the first test and stops with a timeout and a is assigned to undefined error. It seems like it doesn't find the simulator but it's correctly running and the uninstall / install app process correctly worked too.
Here's the code.
environment.js
const {
DetoxCircusEnvironment,
SpecReporter,
WorkerAssignReporter,
} = require('detox/runners/jest-circus')
class CustomDetoxEnvironment extends DetoxCircusEnvironment {
constructor(config) {
super(config)
// Can be safely removed, if you are content with the default value (=300000ms)
this.initTimeout = 30000
// This takes care of generating status logs on a per-spec basis. By default, Jest only reports at file-level.
// This is strictly optional.
this.registerListeners({
SpecReporter,
WorkerAssignReporter,
})
}
}
module.exports = CustomDetoxEnvironment
config.json for jest
{
"testEnvironment": "./environment",
"testRunner": "jest-circus/runner",
"testTimeout": 120000,
"testRegex": "\\.spec\\.js$",
"reporters": ["detox/runners/jest/streamlineReporter"],
"verbose": true
}
.detoxrc.json
{
"testRunner": "jest",
"runnerConfig": "test/tdd/config.json",
"specs": "test/tdd",
"configurations": {
"ios.sim.release": {
"binaryPath": "ios/build/Build/Products/Release-iphonesimulator/BetaSeriesNative.app",
"build": "export RCT_NO_LAUNCH_PACKAGER=true && xcodebuild -workspace ios/BetaSeriesNative.xcworkspace -UseNewBuildSystem=NO -scheme BetaSeriesNative -configuration Release -sdk iphonesimulator -derivedDataPath ios/build -quiet",
"type": "ios.simulator",
"device": {
"type": "iPhone 8"
},
"artifacts": {
"pathBuilder": "./test/tdd/detox.pathbuilder.ios.js"
}
}
}
}
login.spec.js test
describe('When on the incentive page', () => {
beforeEach(async () => {
await device.reloadReactNative()
})
it('it should open the login view', async () => {
await expect(element(by.id('LoginView'))).toBeVisible()
await expect(element(by.id('LoginView_button'))).toBeVisible()
await element(by.id('LoginView_button')).tap()
await expect(element(by.id('LoginView_form'))).toBeVisible()
})
})
Here's the error.
Thanks!

Ok, it seems like I found where the problem was coming from.
I added await device.disableSynchronization() and removed await device.reloadReactNative() in the beforeEach hook.
I also commented a lot of code in my app and I ended returning null in my first homepage view render.
I tried of course to just return null in my render without adding/removing those lines in my test, but without it, it still doesn't work.
Weird thing though. Sometimes it still hangs when launching the test and I still got the same error as before: is assigned to undefined. And when I relaunch it, sometimes it works like the screenshot below. I'd say it's working now maybe 2 out of 3. Maybe there is still some code in my app that is hanging the test and then times out, so I'll keep looking.
Anyway, I think a better error or maybe a warning that it might come from elsewhere, or the app itself, would be better to understand that kind of error. It's pretty unclear for now to know where the source of the error is coming from, even with debug and verbose enabled.
Hopefully it will help some of you guys.
Cheers.
P.S in the screenshot my test still fails because I didn't edit the test, but at least it's running the test alright :)

Related

How to run Playwright in headless mode?

I created a new Vue app using npm init vue#latest and selected Playwright for e2e tests. I removed firefox and webkit from projects in the playwright.config.ts file, so it will only use chromium.
Running npm run test:e2e works fine, the process exists with a success code.
When forcing the tests to fail by modifying the ./e2e/vue.spec.ts file the output is
but the process does not exit with an error code, it still opened browser windows and so CI environments would freeze.
I searched the docs for a specific flag e.g. "headless" and tried --max-failures -x but that didn't help.
How can I tell Playwright to run in headless mode and exit with an error code when something failed?
Since playwright.config.ts already makes use of process.env.CI I thought about replacing reporter: "html", with reporter: [["html", { open: !process.env.CI ? "on-failure" : "never" }]],
but which arguments should I add to the script "test:e2e:ci": "playwright test", to ensure process.env.CI is set?
Update
I tried to run the script inside my CI environment and it seems to work out of the box ( I don't know how it sets the CI environment flag but the pipeline did not freeze )
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Check if e2e tests are passing
run: npm run test:e2e
If any test fails it exists with an error code
It's serving the html report and asking to press 'Ctr+C' to quite.You can disable it using below configuration.
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
};
export default config;
Refer - Report Doc
Issue - https://github.com/microsoft/playwright/issues/9702
To add to the answer above, you can set headless: true in the 'use' block of the config which is above the projects block. Anything set at that level will apply to all projects unless you specifically override the setting inside a project specific area:
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
use: {
headless: true,
},
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
},
},
},
};
export default config;

How can I prevent flaky tests when Cypress testing elements with "v-b-toggle" in a Bitbucket pipeline?

We have a Vue2-based frontend application which uses v-b-toggle to expand/collapse elements on when clicked. When running Cypress tests (either component or e2e tests) locally (CLI and UI), we have not seen the elements fail to expand or collapse. However when running on our Bitbucket pipelines, they will occasionally fail. Has anybody had this issue, and come across a solution?
We can't reproduce this locally, and it only happens intermittently in our pipeline. We've resorted to skipping most of these tests.
I believe this Github issue shows the same behaviour: https://github.com/cypress-io/cypress/issues/7810
More details:
Versions in package.json
"bootstrap-vue": "^2.21.2"
"cypress": "^10.4.0",
"vue": "^2.6.12",
Scripts used for UI and CLI (pipeline) testing:
"test:e2e:ui": "TZ=Etc/UTC cypress open --e2e --browser=electron",
"test:e2e": "TZ=Etc/UTC cypress run --e2e --browser=electron",
"test:component:ui": "TZ=Etc/UTC cypress open --component --browser=electron",
"test:component": "TZ=Etc/UTC cypress run --component --browser=electron"
cypress.config.js:
const { defineConfig } = require("cypress");
module.exports = defineConfig({
env: {
local_url: "http://localhost:8080/",
},
video: false,
screenshotOnRunFailure: true,
defaultCommandTimeout: 8000,
chromeWebSecurity: false, // Disabled to prevent errors with iframes. Would need to reenable if checking for CORS errors in the future.
e2e: {
// We've imported your old cypress plugins here.
// You may want to clean this up later by importing these.
setupNodeEvents(on, config) {
return require("./cypress/plugins/index.js")(on, config);
},
experimentalSessionAndOrigin: true,
baseUrl: "http://localhost:8080",
},
component: {
setupNodeEvents(on, config) {},
specPattern: "src/**/*spec.{js,jsx,ts,tsx}",
devServer: {
framework: "vue-cli",
bundler: "webpack",
},
},
});
Happy to provide more information if required.

How to debug custom mocha test reporter?

I'm writing a mocha test reporter that I want to use for customized Cypress tests documentation.
Which is the right way to debug reporter code (maybe with intellij Idea)?
EDIT
I tried to debug using intellij Idea tools, running cypress (both open and run) in debug mode.
I also tried the IntelliJ Cypress plugin pro version that allow test debugging.
I can't stop in breakpoints.
So I'm trying at least to print some debug log but I can't see my logs anywere.
I couldn't make it work with Cypress, but I could do with Mocha in VSCode.
Working example here
Steps to debug:
Install ts-node and typescript for your project: npm i ts-node typescript --save-dev
Create custom-reporter.ts in your src folder with this content: (taken from https://mochajs.org/api/tutorial-custom-reporter.html and modified slightly )
import { reporters, Runner, Suite, Test } from 'mocha';
const { EVENT_RUN_BEGIN, EVENT_RUN_END, EVENT_TEST_FAIL, EVENT_TEST_PASS, EVENT_SUITE_BEGIN, EVENT_SUITE_END } = Runner.constants;
// This reporter outputs test results, indenting two spaces per suite
export class CustomReporter extends reporters.Base {
private indents = 0;
constructor(runner: Runner) {
super(runner);
const stats = runner.stats;
runner
.once(EVENT_RUN_BEGIN, () => {
console.info('start');
})
.on(EVENT_SUITE_BEGIN, (suite: Suite) => {
this.increaseIndent();
})
.on(EVENT_SUITE_END, (suite: Suite) => {
this.decreaseIndent();
})
.on(EVENT_TEST_PASS, (test: Test) => {
// Test#fullTitle() returns the suite name(s)
// prepended to the test title
console.log(`${this.indent()}pass: ${test.fullTitle()}`);
})
.on(EVENT_TEST_FAIL, (test: Test, err: any) => {
console.log(`${this.indent()}fail: ${test.fullTitle()} - error: ${err.message}`);
})
.once(EVENT_RUN_END, () => {
console.log(`end: ${stats.passes}/${stats.passes + stats.failures} ok`);
});
}
private indent() {
return Array(this.indents).join(' ');
}
private increaseIndent() {
this.indents++;
}
private decreaseIndent() {
this.indents--;
}
}
We will compile custom-reporter.ts by ts-node at runtime, but we have to pass a .js to Mocha as a reporter.
Therefore we create index.js as follows and we export our reporter as follows:
module.exports = require("./src/custom-reporter").CustomReporter;
Add test script to your package.json if you want to run without debugging:
"scripts": {
"test": "mocha -r ts-node/register specs/*.spec.ts --reporter index"
},
Create .vscode/launch.json in your project root, and add the following code:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Debug Mocha tests",
"cwd": "${workspaceRoot}",
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--reporter",
"index",
"${workspaceFolder}/specs/**/*.spec.ts"
],
"protocol": "inspector",
"sourceMaps": true,
"console": "integratedTerminal"
},
]
}
Place some breakpoints in VSCode into src/custom-reporter.ts
In VSCode open the Run and Debug panel (Ctrl+Shift+D), select Debug Mocha tests and press the play button
This way you should be able to start the test running and hit your breakpoints in VSCode.
Cheers!

Unable to do Parallel testing with Detox and Jest

I have seen similar questions on Stack Overflow but I don't feel like we're having the same issue and its been one year for the last question with no answers.
I have followed the documentation and all my tests are working fine, but when I open 4 simulators to try parallel testing only one of them reacts.
package.json
{
...
"detox": {
"configurations": {
"ios.sim.debug": {
"binaryPath": "ios/build/AppName/Build/Products/Debug-iphonesimulator/AppName.app",
"build": "xcodebuild -project ios/AppName.xcodeproj -scheme AppName -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build",
"type": "ios.simulator",
"device": {
"type": "iPhone 11"
}
}
},
"test-runner": "jest --detectOpenHandles --verbose",
"runner-config": "tests/detox/jest.config.js"
}
}
tests/detox/jest.config.js
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
setupFilesAfterEnv: ['./init.ts']
};
init.ts
import { cleanup, init } from 'detox';
const adapter = require('detox/runners/jest/adapter');
const config = require('../../package.json').detox;
jest.setTimeout(90000);
jasmine.getEnv().addReporter(adapter);
beforeAll(async () => {
await init(config, { initGlobals: false });
}, 90000);
afterAll(async () => {
await adapter.afterAll();
await cleanup();
});
And here is the command I use to start tests, after having 4 IOS simulators running and ready
detox test -l warn -w 4 ./path-to-all-tests
Dependencies
MacOS catalina
xed version 11.4
detox: ^16.0.2
jest: ^24.9.0
ts-jest: ^24.1.0
--detectOpenHandles makes the test run synchronously.
from jest docs:
--detectOpenHandles
Attempt to collect and print open handles preventing Jest from exiting
cleanly. Use this in cases where you need to use --forceExit in order
for Jest to exit to potentially track down the reason. This implies
--runInBand, making tests run serially. Implemented using async_hooks. This option has a significant performance penalty and should only be
used for debugging.
https://jestjs.io/docs/cli
You must remove it to run tests in parallel

How do you remove console.log from a build using the JS Quasar Framework?

I am trying the Quasar Framework (for those not familiar, it's based on Vue) and it's going well. However I've tried running a build (npm run build) and get repeated:
error Unexpected console statement no-console
... so the build fails because it sees console.log(...) and is not happy. My options:
don't use console.log in development. But it's handy.
comment out the eslint rule that presumably enforces that, so letting console.log into production. But that's not ideal for performance/security.
have the build automatically remove any console.log. That's what I'm after.
But how?
I took a look at the build https://quasar.dev/quasar-cli/cli-documentation/build-commands and it mentions using webpack internally and UglifyJS too. Given that, I found this answer for removing console.log in a general Vue/webpack project: https://github.com/vuejs-templates/webpack-simple/issues/21
... but if that's how, where does that go within Quasar since there is no webpack config file? I imagine in the quasar.conf.js file (since I see an 'extendWebpack' line in there - sounds promising). Or is there a better way to do it? How do other people remove console.log in production when using Quasar? Or handle logging without it?
Thanks!
https://quasar.dev/quasar-cli/quasar-conf-js#Property%3A-build
quasar.conf.js:
module.exports = function (ctx) {
return {
...
build: {
...
uglifyOptions: {
compress: { drop_console: true }
}
},
}
}
The above will result in configuring terser plugin with the following:
terserOptions: {
compress: {
...
drop_console: true
},
(https://github.com/terser/terser#compress-options)
(you can see the generated config with quasar inspect -c build -p optimization.minimizer)
You still also need to remove the eslint rule to avoid build errors, see https://github.com/quasarframework/quasar/issues/5529
Note:
If you want instead to configure webpack directly use:
quasar.conf.js:
module.exports = function (ctx) {
return {
...
build: {
...
chainWebpack (chain) {
chain.optimization.minimizer('js').tap(args => {
args[0].terserOptions.compress.drop_console = true
return args
})
}
},
}
}
It will do the same as above.
See https://quasar.dev/quasar-cli/cli-documentation/handling-webpack
and https://github.com/neutrinojs/webpack-chain#config-optimization-minimizers-modify-arguments
https://github.com/quasarframework/quasar/blob/dev/app/lib/webpack/create-chain.js#L315
1 Edit package.json in Vue's project what had created it before.
2 Then find "rules": {}.
3 Change to this "rules":{"no-console":0}.
4 if you Vue server in on, off it and run it again. Then the issue will be done.
As an alternative I can suggest using something like loglevel instead of console.log. It's quite handy and allows you to control the output.