I use cypress for component and for e2e tests. Both tests are in the same folder, .test.e2e.js for e2e tests and .test.ct.ts for component tests. When I run component tests I only want to test the .test.ct.ts files and to test the e2e tests only the .test.e2e.ts files.
It works for component tests, cypress tests only the .test.ct.ts files but when I run the e2e test cypress tests the .test.ct.ts, too and this leads to failed tests.
{
"baseUrl": "http://localhost:8080/",
"fixturesFolder": "test/cypress/fixtures",
"integrationFolder": "test/cypress/tests/",
"pluginsFile": "test/cypress/plugins/index.ts",
"screenshotsFolder": "test/cypress/screenshots",
"supportFile": "test/cypress/support/index.ts",
"videosFolder": "test/cypress/videos",
"video": false,
"component": {
"componentFolder": "test/cypress/tests/",
"testFiles": ["**/*.test.ct.ts","**/*.test.ct.js"],
"supportFile": "test/cypress/support/component.ts"
}
}
This is my cypress config file. For component testing I can configure folder and files, I tried this with integration but it didn't work. I'm using cypress 9.7.0
And this is the config I tried
{
"baseUrl": "http://localhost:8080/",
"fixturesFolder": "test/cypress/fixtures",
"integration": {
"integrationFolder": "test/cypress/tests/",
"testFiles": ["**/*.test.e2e.ts","**/*.test.e2e.js"]
},
"pluginsFile": "test/cypress/plugins/index.ts",
"screenshotsFolder": "test/cypress/screenshots",
"supportFile": "test/cypress/support/index.ts",
"videosFolder": "test/cypress/videos",
"video": false,
"component": {
"componentFolder": "test/cypress/tests/",
"testFiles": ["**/*.test.ct.ts","**/*.test.ct.js"],
"supportFile": "test/cypress/support/component.ts"
}
}
Any ideas?
According to the docs, Cypress configuration defines e2e object, not integration one.
You can try with e2e:
{
"baseUrl": "http://localhost:8080/",
"fixturesFolder": "test/cypress/fixtures",
"e2e": {
"integrationFolder": "test/cypress/tests/",
"testFiles": ["**/*.test.e2e.ts","**/*.test.e2e.js"]
},
"pluginsFile": "test/cypress/plugins/index.ts",
"screenshotsFolder": "test/cypress/screenshots",
"supportFile": "test/cypress/support/index.ts",
"videosFolder": "test/cypress/videos",
"video": false,
"component": {
"componentFolder": "test/cypress/tests/",
"testFiles": ["**/*.test.ct.ts","**/*.test.ct.js"],
"supportFile": "test/cypress/support/component.ts"
}
}
There are a few options you can use to make a script that will run only e2e or component tests with cypress run. The easiest way would be to use the --spec option and pass in the folder path to your e2e tests.
package.json file
[
"scripts": {
"e2e": "cypress run --spec 'test/cypress/tests/*.test.e2e.js'"
}
]
Related
I'm having a storybook with vue3 and vite. I want to measure my code coverage via istanbul when I run playwright tests.
Therefore I configured my storybook vite under .storybook/main.ts as follows:
const config: StorybookViteConfig = {
....
typescript: {
check: false,
checkOptions: {},
},
framework: '#storybook/vue3',
core: {
builder: '#storybook/builder-vite',
},
...
async viteFinal(config, { configType }) {
return mergeConfig(config, {
plugins: [
istanbul({
include: 'src/*',
exclude: ['node_modules', 'test/'],
extension: ['.js', '.ts', '.vue'],
}),
],
....
})
},
}
export default config
When I run storybook in dev mode with start-storybook -p 6006 and execute my playwright tests afterwards, the code is instrumented (coverage is not null) and a code coverage is measured.
However, when I build storybook and start the static build afterwards with these commands: build-storybook && http-server storybook-static --port 6006, the website works fine, but the coverage variable doesn't exist and no code coverage is measured, when I run playwright tests there.
I want to measure my code coverage in the ci used the built storybook for that (see https://storybook.js.org/docs/react/writing-tests/test-runner#run-against-non-deployed-storybooks). Or is there any other way to run playwright tests and measure code coverage in the ci?
We have a Vue2-based frontend application which uses v-b-toggle to expand/collapse elements on when clicked. When running Cypress tests (either component or e2e tests) locally (CLI and UI), we have not seen the elements fail to expand or collapse. However when running on our Bitbucket pipelines, they will occasionally fail. Has anybody had this issue, and come across a solution?
We can't reproduce this locally, and it only happens intermittently in our pipeline. We've resorted to skipping most of these tests.
I believe this Github issue shows the same behaviour: https://github.com/cypress-io/cypress/issues/7810
More details:
Versions in package.json
"bootstrap-vue": "^2.21.2"
"cypress": "^10.4.0",
"vue": "^2.6.12",
Scripts used for UI and CLI (pipeline) testing:
"test:e2e:ui": "TZ=Etc/UTC cypress open --e2e --browser=electron",
"test:e2e": "TZ=Etc/UTC cypress run --e2e --browser=electron",
"test:component:ui": "TZ=Etc/UTC cypress open --component --browser=electron",
"test:component": "TZ=Etc/UTC cypress run --component --browser=electron"
cypress.config.js:
const { defineConfig } = require("cypress");
module.exports = defineConfig({
env: {
local_url: "http://localhost:8080/",
},
video: false,
screenshotOnRunFailure: true,
defaultCommandTimeout: 8000,
chromeWebSecurity: false, // Disabled to prevent errors with iframes. Would need to reenable if checking for CORS errors in the future.
e2e: {
// We've imported your old cypress plugins here.
// You may want to clean this up later by importing these.
setupNodeEvents(on, config) {
return require("./cypress/plugins/index.js")(on, config);
},
experimentalSessionAndOrigin: true,
baseUrl: "http://localhost:8080",
},
component: {
setupNodeEvents(on, config) {},
specPattern: "src/**/*spec.{js,jsx,ts,tsx}",
devServer: {
framework: "vue-cli",
bundler: "webpack",
},
},
});
Happy to provide more information if required.
I'm writing a mocha test reporter that I want to use for customized Cypress tests documentation.
Which is the right way to debug reporter code (maybe with intellij Idea)?
EDIT
I tried to debug using intellij Idea tools, running cypress (both open and run) in debug mode.
I also tried the IntelliJ Cypress plugin pro version that allow test debugging.
I can't stop in breakpoints.
So I'm trying at least to print some debug log but I can't see my logs anywere.
I couldn't make it work with Cypress, but I could do with Mocha in VSCode.
Working example here
Steps to debug:
Install ts-node and typescript for your project: npm i ts-node typescript --save-dev
Create custom-reporter.ts in your src folder with this content: (taken from https://mochajs.org/api/tutorial-custom-reporter.html and modified slightly )
import { reporters, Runner, Suite, Test } from 'mocha';
const { EVENT_RUN_BEGIN, EVENT_RUN_END, EVENT_TEST_FAIL, EVENT_TEST_PASS, EVENT_SUITE_BEGIN, EVENT_SUITE_END } = Runner.constants;
// This reporter outputs test results, indenting two spaces per suite
export class CustomReporter extends reporters.Base {
private indents = 0;
constructor(runner: Runner) {
super(runner);
const stats = runner.stats;
runner
.once(EVENT_RUN_BEGIN, () => {
console.info('start');
})
.on(EVENT_SUITE_BEGIN, (suite: Suite) => {
this.increaseIndent();
})
.on(EVENT_SUITE_END, (suite: Suite) => {
this.decreaseIndent();
})
.on(EVENT_TEST_PASS, (test: Test) => {
// Test#fullTitle() returns the suite name(s)
// prepended to the test title
console.log(`${this.indent()}pass: ${test.fullTitle()}`);
})
.on(EVENT_TEST_FAIL, (test: Test, err: any) => {
console.log(`${this.indent()}fail: ${test.fullTitle()} - error: ${err.message}`);
})
.once(EVENT_RUN_END, () => {
console.log(`end: ${stats.passes}/${stats.passes + stats.failures} ok`);
});
}
private indent() {
return Array(this.indents).join(' ');
}
private increaseIndent() {
this.indents++;
}
private decreaseIndent() {
this.indents--;
}
}
We will compile custom-reporter.ts by ts-node at runtime, but we have to pass a .js to Mocha as a reporter.
Therefore we create index.js as follows and we export our reporter as follows:
module.exports = require("./src/custom-reporter").CustomReporter;
Add test script to your package.json if you want to run without debugging:
"scripts": {
"test": "mocha -r ts-node/register specs/*.spec.ts --reporter index"
},
Create .vscode/launch.json in your project root, and add the following code:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Debug Mocha tests",
"cwd": "${workspaceRoot}",
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--reporter",
"index",
"${workspaceFolder}/specs/**/*.spec.ts"
],
"protocol": "inspector",
"sourceMaps": true,
"console": "integratedTerminal"
},
]
}
Place some breakpoints in VSCode into src/custom-reporter.ts
In VSCode open the Run and Debug panel (Ctrl+Shift+D), select Debug Mocha tests and press the play button
This way you should be able to start the test running and hit your breakpoints in VSCode.
Cheers!
I want to integrate my E2E suite in Travis, so I followed this article. As mentioned in the article I've created a custom protractor.ci.conf.js file of the Travis build. I've placed this file inside my e2e folder (path: e2e/protractor.ci.conf.js).
The only difference in my custom e2e/protractor.ci.conf.js and angular generated protractor.conf.js files is the value in args property displayed below.
e2e/protractor.ci.conf.js
chromeOptions: {
args: [
'--headless',
'window-size=1920,1080'
]
}
protractor.conf.js
const SpecReporter = require('jasmine-spec-reporter').SpecReporter;
exports.config = {
allScriptsTimeout: 11000,
specs: [
'./e2e/**/*.e2e-spec.ts'
],
capabilities: {
shardTestFiles: true,
maxInstances: 2,
'browserName': 'chrome',
chromeOptions: {
args: ['--start-maximized']
}
},
directConnect: true,
baseUrl: 'localhost:4000/',
framework: 'jasmine',
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 300000,
print: function () {
}
},
useAllAngular2AppRoots: true,
onPrepare: function () {
jasmine.getEnv().addReporter(new SpecReporter());
require('ts-node').register({
project: 'e2e/tsconfig.json'
});
}
};
In my package.json file there are 2 scripts one for running tests locally and one on Travis.
Package.json (at the same level where protractor.conf.js is located)
"scripts": {
...
"test": "ng test --watch=false",
"pree2e": "webdriver-manager update",
"e2e": "concurrently --kill-others \"ng e2e --port=4000\" \"npm run _server:run\"",
"e2e:ci": "concurrently --kill-others \"ng e2e --port=4000 --protractor-config=e2e/protractor.ci.conf.js\" \"npm run _server:run\"",
"_server:run": "tsc -p ./server && concurrently \"tsc -w -p ./server\" \"nodemon dist/server/index.js\" ",
...
},
.travis.yml
branches:
only:
- staging
- prod
- functional-testing
script:
...
- if [[ $TRAVIS_COMMIT_MESSAGE == *"[skip e2e]"* ]]; then echo "skipping E2E test"; else npm run e2e:ci; fi
...
before_deploy:
- sed -i '/dist/d' .gitignore
- git add . && git commit -m "latest build"
- cd $TRAVIS_BUILD_DIR/dist
PROBLEM
When simply running npm run e2e, every test is working fine. But when I'm using npm run e2e:ci command scripts hangs and no instance of WebDriver runs.
I/launcher — Running 0 instances of WebDriver
is coming instead of 1 or 2 instances.
That's because since you made a new config file and apparently placed in the folder
/e2e instead of the default root folder.
The path to the test files in your case should also be updated.
So './e2e/**/*.e2e-spec.ts' will get changed to './**/*.e2e-spec.ts'
Since, currently the test is not able to find any files specified, it doesn't run any instances.
I have VueJS application written with NuxtJS. This setup causes that I have many files index.vue in different directories.
When I run the testing suite with commend jest --no-cache --watch --coverage only 1 file index.vue has being picked up by coverage results.
my configuration of jest in package.json is:
"jest": {
"transform": {
"^.+.vue$": "vue-jest",
"^.+.js$": "babel-jest"
},
"collectCoverage": true,
"collectCoverageFrom": [
"**/*.{js,vue}",
"!**/node_modules/**"
],
"coverageReporters": [
"text"
],
"setupTestFrameworkScriptFile": "jest-extended"
}
and results show only coverage for 1 index.vue file (even I have multiple of them as well other .vue files).
What configuration option I need to add to run coverage for all .vue files?
At a first sight, I would expect to see Jest's moduleFileExtensions config option in place like:
"jest": {
"moduleFileExtensions": ["js", "json", "jsx", "node", "vue"],
// The rest of your config...
}
The option tells Jest which file extensions are used by the modules of your application.