How to get code coverage report from jest-junit in React project - react-native

According to this article I'd like to get jest-junit code coverage report (Option 2 in article)
So, in my package.json I invoke jest like this: "test": "jest --config=jest.config.js",
jest.config.js includes these settings:
module.exports = {
preset: 'react-native',
moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
setupFiles: ['./jest.setup.js'],
collectCoverage: true,
coverageDirectory: 'src/testCoverage',
coverageReporters: [ "text"],
reporters: ["default",
["jest-junit", {usePathForSuiteName: true, outputDirectory: 'src/testCoverage'}]
],
testResultsProcessor: "jest-junit"
};
I did in this way because the written jest code in package.json doesn't work for me.
When I execute npm run test I get coverage data in src/testCoverage folder and junit.xml
Then I execute test stage in Jenkins pipeline:
stage('test') {
steps{
sh script:'''
#!/bin/bash
npm install -g yarn
yarn install
yarn add --dev jest-junit
npm run test
'''
}
post {
always {
junit 'src/testCoverage/junit.xml'
}
}
}
But I don't see junit coverage report in Jenkins, while the article says that
The line calling junit will publish the report that npm run test created
The only thing that I have - this is test result report of passed and failed tests.
Why I don't get junit coverage report in Jenkins? What should I do or change?

This happens because the junit.xml contains only the list of test cases executed and their duration.
In order to export the line coverage, you need some other coverage reported, for example, cobertura:
coverageReporters: ['text', 'cobertura'],
It will produce coverage/cobertura-coverage.xml file containing the coverage information. Now you can use junit.xml to report the list of tests and cobertura-coverage.xml to report the line coverage to your CICD system.

Related

How to run Playwright in headless mode?

I created a new Vue app using npm init vue#latest and selected Playwright for e2e tests. I removed firefox and webkit from projects in the playwright.config.ts file, so it will only use chromium.
Running npm run test:e2e works fine, the process exists with a success code.
When forcing the tests to fail by modifying the ./e2e/vue.spec.ts file the output is
but the process does not exit with an error code, it still opened browser windows and so CI environments would freeze.
I searched the docs for a specific flag e.g. "headless" and tried --max-failures -x but that didn't help.
How can I tell Playwright to run in headless mode and exit with an error code when something failed?
Since playwright.config.ts already makes use of process.env.CI I thought about replacing reporter: "html", with reporter: [["html", { open: !process.env.CI ? "on-failure" : "never" }]],
but which arguments should I add to the script "test:e2e:ci": "playwright test", to ensure process.env.CI is set?
Update
I tried to run the script inside my CI environment and it seems to work out of the box ( I don't know how it sets the CI environment flag but the pipeline did not freeze )
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Check if e2e tests are passing
run: npm run test:e2e
If any test fails it exists with an error code
It's serving the html report and asking to press 'Ctr+C' to quite.You can disable it using below configuration.
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
};
export default config;
Refer - Report Doc
Issue - https://github.com/microsoft/playwright/issues/9702
To add to the answer above, you can set headless: true in the 'use' block of the config which is above the projects block. Anything set at that level will apply to all projects unless you specifically override the setting inside a project specific area:
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
use: {
headless: true,
},
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
},
},
},
};
export default config;

Can't instrument code when serving storybook production build

I'm having a storybook with vue3 and vite. I want to measure my code coverage via istanbul when I run playwright tests.
Therefore I configured my storybook vite under .storybook/main.ts as follows:
const config: StorybookViteConfig = {
....
typescript: {
check: false,
checkOptions: {},
},
framework: '#storybook/vue3',
core: {
builder: '#storybook/builder-vite',
},
...
async viteFinal(config, { configType }) {
return mergeConfig(config, {
plugins: [
istanbul({
include: 'src/*',
exclude: ['node_modules', 'test/'],
extension: ['.js', '.ts', '.vue'],
}),
],
....
})
},
}
export default config
When I run storybook in dev mode with start-storybook -p 6006 and execute my playwright tests afterwards, the code is instrumented (coverage is not null) and a code coverage is measured.
However, when I build storybook and start the static build afterwards with these commands: build-storybook && http-server storybook-static --port 6006, the website works fine, but the coverage variable doesn't exist and no code coverage is measured, when I run playwright tests there.
I want to measure my code coverage in the ci used the built storybook for that (see https://storybook.js.org/docs/react/writing-tests/test-runner#run-against-non-deployed-storybooks). Or is there any other way to run playwright tests and measure code coverage in the ci?

botium project in eclipse with multiple botium.json not working

I have setup botium project according to direction given in https://chatbotsmagazine.com/5-steps-automated-testing-of-chatbots-in-eclipse-ef4c3dcaf233 and its working fine for single botium.json file.
but when i try to setup multiple connector together ex-
1)botium_dialog.json
{
"botium": {
"Capabilities": {
"PROJECTNAME": "jokes",
"CONTAINERMODE": "dialogflow",
"DIALOGFLOW_PROJECT_ID": "###",
"DIALOGFLOW_CLIENT_EMAIL": "###",
"DIALOGFLOW_PRIVATE_KEY": "###",
"DIALOGFLOW_USE_INTENT": false
}
}
}
2) botium_watson.json
{
"botium": {
"Capabilities": {
"PROJECTNAME": "IBM Watson Conversation Sample",
"SCRIPTING_UTTEXPANSION_MODE": "all",
"SCRIPTING_FORMAT": "xlsx",
"SCRIPTING_XLSX_STARTROW": 2,
"SCRIPTING_XLSX_STARTCOL": 1,
"CONTAINERMODE": "watson",
"WATSON_USER": "#",
"WATSON_PASSWORD": "#",
"WATSON_WORKSPACE_ID": "#"
}
}
}
in the same project but running 1 at a time using
mocha --reporter mochawesome --reporter-options
\"reportDir=reportsDialog,reportFilename=index.html,code=false\"
--convos ./spec/convo/dialog --config botium_dialog.json --exit spec "
its giving error
Error: Capability 'CONTAINERMODE' missing
at BotDriver._getContainer (node_modules\botium-core\src\BotDriver.js:316:13)
at async.series (node_modules\botium-core\src\BotDriver.js:154:30)
The "--convos" and the "--config" command line parameters are actually for the Botium CLI, not for mocha. You either switch your test scripts to Botium CLI, or you configure Botium in a way to use several configuration files and several convo directories. My recommendation would be to pack each section in an own subdirectory - so you have a "botium_dialog" and a "botium_watson" directory, each with it's own package.json, botium.json, spec/convo folders etc.
With some configuration changes, it is also possible to use your current folder structure.
Add multiple botium.spec.js in spec folder:
botium_dialog.spec.js:
const BotiumBindings = require('botium-bindings')
const bb = new BotiumBindings({ convodirs: [ './spec/convo/dialog' ] })
BotiumBindings.helper.mocha().setupMochaTestSuite({ bb })
botium_watson.spec.js:
const BotiumBindings = require('botium-bindings')
const bb = new BotiumBindings({ convodirs: [ './spec/convo/watson' ] })
BotiumBindings.helper.mocha().setupMochaTestSuite({ bb })
Add multiple test scripts to your package.json:
package.json:
...
"scripts": {
"test_dialog": "BOTIUM_CONFIG=botium_dialog.json mocha --reporter spec --exit spec/botium_dialog.spec.js",
"test_watson": "BOTIUM_CONFIG=botium_watson.json mocha --reporter spec --exit spec/botium_watson.spec.js"
}
...
Run both of the test scripts
For example:
npm run test_dialog
npm run test_watson

Watch and rerun Jest JS tests

The Jest documentation suggests using npm test to execute tests.
Is there a way of watching your source and tests to rerun Jest tests automatically when relevant files have been changed?
Thanks to Erin Stanfill for pointing out, Jest already has support for automatically re-running. The better configuration for package.json would be
{
"scripts": {
"test": "jest"
}
}
To turn on the watch mode, just use
$ npm run test -- --watch
Or
$ yarn run test --watch
If you have npm test configured, you can just run npm test -- --watch.
As a complement suggestion you can add "--watchAll"
into your package.json file like this:
"scripts": {
"test": "jest --watchAll"
},
Each time you run npm test, the watch mode will be enable by default.
For more info npm CLI docs
Start you tests in watch mode.
jest --watch fileName.test.js
As per documentation
Run tests that match this spec name (match against the name in describe or test, basically).
jest -t name-of-spec
// or in watch mode
jest --watch -t="TestName"
This example shows how to use gulp to run your Jest tests using jest-cli, as well as a tdd gulp task to watch files and rerun Jest tests when a file changes:
var gulp = require('gulp');
var jest = require('jest-cli');
var jestConfig = {
rootDir: 'source'
};
gulp.task('test', function(done) {
jest.runCLI({ config : jestConfig }, ".", function() {
done();
});
});
gulp.task('tdd', function(done) {
gulp.watch([ jestConfig.rootDir + "/**/*.js" ], [ 'test' ]);
});
gulp.task('default', function() {
// place code for your default task here
});
install a couple of Grunt packages:
npm install grunt-contrib-watch grunt-exec --save-dev
make a Gruntfile.js with the following:
module.exports = function(grunt) {
grunt.initConfig({
exec: {
jest: 'node node_modules/jest-cli/bin/jest'
},
watch: {
files: ['**/*.js'],
tasks: ['exec:jest']
}
});
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-exec');
}
then simply run:
grunt watch
If you want to run a single file in watch mode:
yarn run test --watch FileName.test.jsx
I personally use the npm package jest-watch-typeahead.
You need to do 3 steps:
Install npm packege:
npm install --save-dev jest jest-watch-typeahead
Add to jest.config.js next code:
module.exports = {
watchPlugins: [
'jest-watch-typeahead/filename',
'jest-watch-typeahead/testname',
],
};
Run Jest in watch mode
yarn jest --watch

protractor could not find protractor/selenium/chromedriver.exe at codeship

i'm trying to configure the integration to run portractor tests.
I'm using grunt-protractor-runner task
with following configuration:
protractor: {
options: {
configFile: "protractor.conf.js", //your protractor config file
keepAlive: true, // If false, the grunt process stops when the test fails.
noColor: false, // If true, protractor will not use colors in its output.
args: {
// Arguments passed to the command
}
},
run: {},
chrome: {
options: {
args: {
browser: "chrome"
}
}
}
}
and here is grunt task which i use for running the protractor after the server is running:
grunt.registerTask('prot', [
'connect:test',
'replace:includemocks',//for uncommenting angular-mocks reference
'protractor:run',
'replace:removemocks',//for commenting out angular-mocks reference
]);
It is running well on my local machine, but at codeship i'm getting following error:
Error: Could not find chromedriver at /home/rof/src/bitbucket.org/myrepo/myFirstRepo/node_modules/grunt-protractor-runner/node_modules/protractor/selenium/chromedriver.exe
Which i guess, a result of not having this "chromedriver.exe" at this path.
How can i solve it in codeship environment?
Thanks forwards
Add postinstall to your package.json file and that way npm install will take care of placing the binaries for you ahead of time:
"scripts": {
"postinstall": "echo -n $NODE_ENV | \
grep -v 'production' && \
./node_modules/protractor/bin/webdriver-manager update || \
echo 'will skip the webdriver install/update in production'",
...
},
And don't forget to set NODE_ENV ... not setting it at all will result in echo 'will skip the webdriver install/update in production' piece running. Setting it to dev or staging will get desired results.
Short answer (pulkitsinghal gave the original solution):
./node_modules/grunt-protractor-runner/node_modules/protractor/bin/webdriver-manager update
I'm one of the founders at Codeship.
The error seems to be because you are trying to use the exe file, but we're on Linux on our system. Did you hardcode that executable?
Could you send us an in-app support request so we have a link to look at and can help you fix this?