I'm having a storybook with vue3 and vite. I want to measure my code coverage via istanbul when I run playwright tests.
Therefore I configured my storybook vite under .storybook/main.ts as follows:
const config: StorybookViteConfig = {
....
typescript: {
check: false,
checkOptions: {},
},
framework: '#storybook/vue3',
core: {
builder: '#storybook/builder-vite',
},
...
async viteFinal(config, { configType }) {
return mergeConfig(config, {
plugins: [
istanbul({
include: 'src/*',
exclude: ['node_modules', 'test/'],
extension: ['.js', '.ts', '.vue'],
}),
],
....
})
},
}
export default config
When I run storybook in dev mode with start-storybook -p 6006 and execute my playwright tests afterwards, the code is instrumented (coverage is not null) and a code coverage is measured.
However, when I build storybook and start the static build afterwards with these commands: build-storybook && http-server storybook-static --port 6006, the website works fine, but the coverage variable doesn't exist and no code coverage is measured, when I run playwright tests there.
I want to measure my code coverage in the ci used the built storybook for that (see https://storybook.js.org/docs/react/writing-tests/test-runner#run-against-non-deployed-storybooks). Or is there any other way to run playwright tests and measure code coverage in the ci?
Related
I created a new Vue app using npm init vue#latest and selected Playwright for e2e tests. I removed firefox and webkit from projects in the playwright.config.ts file, so it will only use chromium.
Running npm run test:e2e works fine, the process exists with a success code.
When forcing the tests to fail by modifying the ./e2e/vue.spec.ts file the output is
but the process does not exit with an error code, it still opened browser windows and so CI environments would freeze.
I searched the docs for a specific flag e.g. "headless" and tried --max-failures -x but that didn't help.
How can I tell Playwright to run in headless mode and exit with an error code when something failed?
Since playwright.config.ts already makes use of process.env.CI I thought about replacing reporter: "html", with reporter: [["html", { open: !process.env.CI ? "on-failure" : "never" }]],
but which arguments should I add to the script "test:e2e:ci": "playwright test", to ensure process.env.CI is set?
Update
I tried to run the script inside my CI environment and it seems to work out of the box ( I don't know how it sets the CI environment flag but the pipeline did not freeze )
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Check if e2e tests are passing
run: npm run test:e2e
If any test fails it exists with an error code
It's serving the html report and asking to press 'Ctr+C' to quite.You can disable it using below configuration.
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
};
export default config;
Refer - Report Doc
Issue - https://github.com/microsoft/playwright/issues/9702
To add to the answer above, you can set headless: true in the 'use' block of the config which is above the projects block. Anything set at that level will apply to all projects unless you specifically override the setting inside a project specific area:
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
use: {
headless: true,
},
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
},
},
},
};
export default config;
I have a nuxt 3 project that is working fine on localhost while developing. I use FormKit and vue-toastification and everything is fine on "npm run dev".
But once I run npm run build to build it for deployment. lots of stylings are missing. Mainly those two plugins. the error messages for the forms by FormKit aren't red and not styled. vue-toastifications display with full height and width of the screen as it has no styling.
would I do any extra steps before running npm run build? like building these plugins or something?
this is my nuxt.config.ts file if it might help!
// #ts-nocheck
import Icons from "unplugin-icons/vite"
// https://v3.nuxtjs.org/api/configuration/nuxt.config
export default defineNuxtConfig({
css: ["~/assets/fonts/droidkufi/droidarabickufi.css", "#formkit/themes/genesis"],
modules: [
"#nuxtjs/tailwindcss",
[
"#pinia/nuxt",
{
autoImports: ["defineStore"],
},
],
"#formkit/nuxt",
],
plugins: ["~/plugins/flowbite.client.ts", "~/plugins/i18n.ts"],
vite: {
plugins: [
Icons({
autoInstall: true,
}),
],
},
runtimeConfig: {
// secret serverside variables
public: {
// baseURL: "http://127.0.0.1:8000/api/",
// apiBase: "http://127.0.0.1:8000/api/",
// homeBase: "http://127.0.0.1:8000",
baseURL: "https://sju.davidlouis.co/api/",
apiBase: "https://sju.davidlouis.co/api/",
homeBase: "https://sju.davidlouis.co",
},
},
vue: {
compilerOptions: {
isCustomElement: (tag) => ["datepicker-hijri"].includes(tag),
},
},
})
I tried to run npm run dev back to test if styles are working. and yes they are working fine. the problem starts when I run npm run build for production and deployment.
We have a Vue2-based frontend application which uses v-b-toggle to expand/collapse elements on when clicked. When running Cypress tests (either component or e2e tests) locally (CLI and UI), we have not seen the elements fail to expand or collapse. However when running on our Bitbucket pipelines, they will occasionally fail. Has anybody had this issue, and come across a solution?
We can't reproduce this locally, and it only happens intermittently in our pipeline. We've resorted to skipping most of these tests.
I believe this Github issue shows the same behaviour: https://github.com/cypress-io/cypress/issues/7810
More details:
Versions in package.json
"bootstrap-vue": "^2.21.2"
"cypress": "^10.4.0",
"vue": "^2.6.12",
Scripts used for UI and CLI (pipeline) testing:
"test:e2e:ui": "TZ=Etc/UTC cypress open --e2e --browser=electron",
"test:e2e": "TZ=Etc/UTC cypress run --e2e --browser=electron",
"test:component:ui": "TZ=Etc/UTC cypress open --component --browser=electron",
"test:component": "TZ=Etc/UTC cypress run --component --browser=electron"
cypress.config.js:
const { defineConfig } = require("cypress");
module.exports = defineConfig({
env: {
local_url: "http://localhost:8080/",
},
video: false,
screenshotOnRunFailure: true,
defaultCommandTimeout: 8000,
chromeWebSecurity: false, // Disabled to prevent errors with iframes. Would need to reenable if checking for CORS errors in the future.
e2e: {
// We've imported your old cypress plugins here.
// You may want to clean this up later by importing these.
setupNodeEvents(on, config) {
return require("./cypress/plugins/index.js")(on, config);
},
experimentalSessionAndOrigin: true,
baseUrl: "http://localhost:8080",
},
component: {
setupNodeEvents(on, config) {},
specPattern: "src/**/*spec.{js,jsx,ts,tsx}",
devServer: {
framework: "vue-cli",
bundler: "webpack",
},
},
});
Happy to provide more information if required.
By default, Cypress compiles e2e tests with a built-in webpack config, which used to be fine because Vue-CLI also used Webpack; however, now that I've upgraded to Vue 3 and Vite, no webpack.
I have two options:
Revive the old webpack config for my Vue 2 project and update it for Vue 3 just to run Cypress' e2e tests.
Figure out how to tell Cypress to compile the app with Vite and not Webpack
I can't figure out #2, and I don't want to do #1 because having two different compilation methods sounds like a really bad future headache.
So far, I have this for my Cypress config:
import { devServer } from '#cypress/vite-dev-server'
import { defineConfig } from 'cypress'
import * as path from 'path'
export default defineConfig({
chromeWebSecurity: false,
projectId: '5kusbh',
requestTimeout: 10000,
responseTimeout: 60000,
viewportHeight: 1080,
viewportWidth: 1920,
e2e: {
baseUrl: 'http://localhost:8080',
setupNodeEvents (on, config) {
on('dev-server:start', (options) => {
return devServer({
...options,
viteConfig: {
configFile: path.resolve(__dirname, 'vite.config.ts'),
},
})
})
return config
},
specPattern: 'cypress/e2e/**/**.spec.js',
},
})
However, when I run Cypress, I get a webpack compilation error, which is telling me Vite is not compiling the application for Cypress.
Note Otherwise, my application is working great - I just can't run Cypress, and we have hundreds of unit, integration, and e2e tests written in Cypress.
TL;DR; I need help configuring Cypress to use my app's Vite config to compile its e2e tests and run it's dev server.
EDIT:
I removed my config to see how it'd run just hitting localhost, but Cypress must be trying to compile my code, because it's struggling with the Vite env variable syntax, import.meta.env.[insert key name here] in non-Cypress JavaScript files because it's not process.env...
For e2e tests, the execution of tests is separate from running the app and you access the app using cy.visit(). #cypress/vite-dev-server was for running component tests prior to cypress 10. But with cypress-vite you can also compile e2e tests using vite and use the same configuration as your app, so you don't need to config the webpack anymore.
According to this article I'd like to get jest-junit code coverage report (Option 2 in article)
So, in my package.json I invoke jest like this: "test": "jest --config=jest.config.js",
jest.config.js includes these settings:
module.exports = {
preset: 'react-native',
moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
setupFiles: ['./jest.setup.js'],
collectCoverage: true,
coverageDirectory: 'src/testCoverage',
coverageReporters: [ "text"],
reporters: ["default",
["jest-junit", {usePathForSuiteName: true, outputDirectory: 'src/testCoverage'}]
],
testResultsProcessor: "jest-junit"
};
I did in this way because the written jest code in package.json doesn't work for me.
When I execute npm run test I get coverage data in src/testCoverage folder and junit.xml
Then I execute test stage in Jenkins pipeline:
stage('test') {
steps{
sh script:'''
#!/bin/bash
npm install -g yarn
yarn install
yarn add --dev jest-junit
npm run test
'''
}
post {
always {
junit 'src/testCoverage/junit.xml'
}
}
}
But I don't see junit coverage report in Jenkins, while the article says that
The line calling junit will publish the report that npm run test created
The only thing that I have - this is test result report of passed and failed tests.
Why I don't get junit coverage report in Jenkins? What should I do or change?
This happens because the junit.xml contains only the list of test cases executed and their duration.
In order to export the line coverage, you need some other coverage reported, for example, cobertura:
coverageReporters: ['text', 'cobertura'],
It will produce coverage/cobertura-coverage.xml file containing the coverage information. Now you can use junit.xml to report the list of tests and cobertura-coverage.xml to report the line coverage to your CICD system.