WebdriverIO : Junit report- How do I generate the junit report under specify directory - webdriver-io

I'm trying to integerate Junit reporter in my webdriverio project
junit config in wdio.conf.js
When I execute the report generates in my terminal output but the xml file is not generated under the junitreports directory.
Terminal output
Can you please help me overcome this issue?

I'm able to generate the XML report using the below configuration in the wdio file:
reporters: [['junit',{
outputDir: './Reports/junit-report/',
outputFileFormat: function(options) { // optional
return 'junit-report.xml'
}
}]
],

Related

How to get code coverage report from jest-junit in React project

According to this article I'd like to get jest-junit code coverage report (Option 2 in article)
So, in my package.json I invoke jest like this: "test": "jest --config=jest.config.js",
jest.config.js includes these settings:
module.exports = {
preset: 'react-native',
moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx', 'json', 'node'],
setupFiles: ['./jest.setup.js'],
collectCoverage: true,
coverageDirectory: 'src/testCoverage',
coverageReporters: [ "text"],
reporters: ["default",
["jest-junit", {usePathForSuiteName: true, outputDirectory: 'src/testCoverage'}]
],
testResultsProcessor: "jest-junit"
};
I did in this way because the written jest code in package.json doesn't work for me.
When I execute npm run test I get coverage data in src/testCoverage folder and junit.xml
Then I execute test stage in Jenkins pipeline:
stage('test') {
steps{
sh script:'''
#!/bin/bash
npm install -g yarn
yarn install
yarn add --dev jest-junit
npm run test
'''
}
post {
always {
junit 'src/testCoverage/junit.xml'
}
}
}
But I don't see junit coverage report in Jenkins, while the article says that
The line calling junit will publish the report that npm run test created
The only thing that I have - this is test result report of passed and failed tests.
Why I don't get junit coverage report in Jenkins? What should I do or change?
This happens because the junit.xml contains only the list of test cases executed and their duration.
In order to export the line coverage, you need some other coverage reported, for example, cobertura:
coverageReporters: ['text', 'cobertura'],
It will produce coverage/cobertura-coverage.xml file containing the coverage information. Now you can use junit.xml to report the list of tests and cobertura-coverage.xml to report the line coverage to your CICD system.

Karma coverage istanbul does not create folder

I am trying to integrate a code coverage reporter to my karma tests.
I set up the following changes in my config:
coverageIstanbulReporter: {
reports: ['html', 'lcovonly', 'text-summary'],
// base output directory. If you include %browser% in the path it will be replaced with the karma browser name
dir: path.join(__dirname, 'coverage'),
// if using webpack and pre-loaders, work around webpack breaking the source path
fixWebpackSourcePaths: true,
},
reporters: config.coverage ? ['kjhtml', 'dots', 'coverage-istanbul'] : ['kjhtml', 'dots'],
But unfortnetly, it does not create a coverage folder.
Here is my full karma.config.js
What am I doing wrong here? Note, that I am calling karma start ./karma.conf.js --coverage and even checked without the config coverage parameter.
Given you have text-summary outputs on Chrome console and you already have the base directory for the reporter specified, you just need to explicitly tell the reporter which sub directory should be used for different report type:
Try to add this entry under coverageIstanbulReporter (from repo's README):
// Most reporters accept additional config options. You can pass these through the `report-config` option
'report-config': {
// all options available at: https://github.com/istanbuljs/istanbul-reports/blob/590e6b0089f67b723a1fdf57bc7ccc080ff189d7/lib/html/index.js#L135-L137
html: {
// outputs the report in ./coverage/html
subdir: 'html'
}
}

Automatically run webdriver for e2e test in Gulp

There is the following protractor.conf.js:
exports.config = {
framework: 'jasmine',
specs: ['e2e-tests/**/*.js'],
seleniumAddress: 'http://127.0.0.1:4444/wd/hub',
seleniumServerJar: './node_modules/protractor/selenium/selenium-server-standalone-2.47.1.jar'
}
Gulp task:
gulp.task('e2e-testing', ['webdriver_standalone'], function() {
gulp.src([]).pipe(protractor({ configFile: "protractor.conf.js" }))
.on('error', function(e) { throw e })
});
This code works good, but I must execute webdriver-manager start --standalone before starting e2e task. How can I omit it? How can I do it automatically? Thanks in advance
Just remove seleniumAddress config option in order to make Protractor instantiate a new server by itself when you start a test suite. Otherwise, if server address is specified, Protractor tries to connect to it rather then create a new one. Take a look at the section Starting the Server from a Test Script in the docs:
Please note that if you set seleniumAddress, the settings for
seleniumServerJar, seleniumPort, seleniumArgs, sauceUser and sauceKey
will be ignored.
Yes, we need to add the below in config.js
seleniumServerJar: 'path of jar file',
seleniumPort: '4444',
and remove seleniumAddress from the config file.
It works absolutely fine

Yeoman webapp generator - How to run mocha tests in the Browser

I've got some JS tests written in mocha/chai and I would like to run them in a project scaffolded using the webapp generator.
I've put my tests inside the "test" folder from Yeoman's structure and the tests are running fine. But the issue is that the grunt test command is only showing the test results in the console, not in the browser.
I'm looking for a way to run the command and have the tests shown in the browser. How can I do that?
Thanks for any help!
Please consider this part of connect's configuration (more precisely, this is a sub-task of connect):
test : {
options : {
middleware : function(connect) {
return [
require('connect-livereload')(),
mountFolder(connect, '.tmp'),
mountFolder(connect, 'test')
];
},
port : grunt.option('port') ? Number(grunt.option('port')) + 1 : 9001
}
}
The above will make files from the specified folders available through http.
.tmp is where my transpiled coffeescript and SCSS is landing as regular JS/CSS
test is where my tests reside together with a very simple index.html file which wires all JS/CSS together, including mocha.js and mocha.css
Later in my Gruntfile I register the test task:
grunt.registerTask('test', function(target) {
var tasks = [
'clean:server',
'coffee',
'jst',
'connect:test'
];
if (target === 'browser') {
tasks.push('open:test');
tasks.push('watch');
} else {
tasks.push('mocha');
}
grunt.task.run(tasks);
});
The part which is relevant to your problem is 'connect:test' which makes it possible to access the tests through the browser, and this one:
if (target === 'browser') {
tasks.push('open:test');
tasks.push('watch');
} else {
tasks.push('mocha');
}
As long as you don't specify browser as your test target, the tests will run headlessly in the console. But if you go like grunt test:browser, Grunt will open a browser thanks to open:test. For your reference, I also include my open:test config:
test : {
path : 'http://localhost:<%= connect.test.options.port %>'
}

Setting up Continuous Integration of Protractor using Jenkins

I am writing automation test scripts using Protractor and now I need to set up the CI for this using Jenkins.
Tasks it needs to perform are:
Starting the selenium standalon server.
Starting the test using conf.js file.
Stopping the selenium standalone server.
Can anyone help in this regard?
I created a small bash script to do this.
# start selenium
./node_modules/protractor/bin/webdriver-manager start > /dev/null 2>&1 &
# wait until selenium is up
while ! curl http://localhost:4444/wd/hub/status &>/dev/null; do :; done
# run the build
grunt cibuild --force
# stop selenium
curl -s -L http://localhost:4444/selenium-server/driver?cmd=shutDownSeleniumServer > /dev/null 2>&1
This script is invoked from a free-style project in jenkins (Build > Execute shell)
Then the test result report is generated by reading the protractor test results. Hence, you have to produce junit reports from protractor, (look here) :
onPrepare: function() {
// The require statement must be down here, since jasmine-reporters
// needs jasmine to be in the global and protractor does not guarantee
// this until inside the onPrepare function.
require('jasmine-reporters');
jasmine.getEnv().addReporter(
new jasmine.JUnitXmlReporter('xmloutput', true, true));
},
To make the report visible in jenkins i add a post build action in the job: Publish JUnit test result report:
Alternatively, you could run this as a Grunt Task. First install grunt on Jenkins. Install the NPM packages for protractor_webdriver and protractor. Setup the configuration file to point the the node_module path and config file paths.
http://sideroad.secret.jp/articles/grunt-on-jenkins/
Then install protractor node modules. The Gruntfile would look similar to this. I created a test directory where the conf and spec files would be located.
module.exports = function (grunt) {
grunt.initConfig({
protractor_webdriver: {
your_target: {
options: {
path: 'node_modules/protractor/bin/',
command: 'webdriver-manager start'
}
}
},
protractor: {
options: {
configFile: "node_modules/protractor/referenceConf.js", // Default config file
keepAlive: true, // If false, the grunt process stops when the test fails.
noColor: false, // If true, protractor will not use colors in its output.
args: {
// Arguments passed to the command
}
},
your_target: {
options: {
configFile: "test/conf.js", // Target-specific config file
args: {} // Target-specific arguments
}
}
}
});
grunt.registerTask('p:test', [
'protractor_webdriver',
'protractor'
]);
});
The newest protractor allows you to run the selenium standalone server directly from the conf.js (or whatever protractor entry point you have).
comment out (or delete) the seleniumAddress: 'http://localhost:4444/wd/hub', line, and replace it with seleniumServerJar: './node_modules/protractor/selenium/latest.jar'.
latest.jar isn't installed by default, I created it as a symlink to the latest version installed via npm install protractor --save. This gives longer life to my conf.js files in the same directory.
Within the ./node_modules/protractor/selenium/ folder I ran ln -s selenium-server-standalone-2.48.2.jar latest.jar
You can use Gulp which is far simpler.
After installing gulp in Jenkins System , you may install the npm dependencies(npm install) & run gulp tasks directly as windows batch command in Jenkins as below:
In the background to make selenium server up and running and providing various other parameters , you may use packages like 'gulp-angular-protractor' in the gulpfile.js as below:
gulpfile.js
'use strict';
var gulp = require('gulp'),
gulpProtractorAngular = require('gulp-angular-protractor'),
gulpStart = gulp.Gulp.prototype.start,
currentStartTaskName;
gulp.Gulp.prototype.start = function (task) {
currentStartTaskName = task;
gulpStart.apply(this, arguments);
};
function executeWebTests(suiteName, appName) {
return gulp.src([])
.pipe(gulpProtractorAngular({
'configFile': './conf.js',
'debug': false,
'autoStartStopServer': true,
args: [
'--suite', suiteName,
'--capabilities.browserName', 'chrome',
'--params.APPNAME', appName,
'--params.SUITENAME', currentStartTaskName,
'--capabilities.platformName', 'Windows'],
keepAlive: false
}))
.on('error', function (e) {
console.log('Ended with below ERROR::',e);
process.exit(1);
})
.on('end', function () {
console.log('Test complete');
process.exit();
});
}
gulp.task('RegressionSuiteTask', function () {
executeWebTests('regressionTests,','Application_Name');
});
conf.js
suites: {
regressionTests: ['testCases/**/*.js']//will run all specs in subfolders
},
I know this already resolved and want to target for beginner to create Jenkins job and running test. I suggest to use selenium-server-standalone jar in configuration file and call configuration file from Jenkins.
conf.js
..
exports.config = {
//seleniumAddress: 'http://localhost:4444/wd/hub',
seleniumServerJar: 'node_modules/protractor/node_modules/webdriver-manager/selenium/selenium-server-standalone-3.5.3.jar',
....
//html reporter logic
.....
Creating Jenkins Job
Install node js on Jenkins Server
Install Html Publisher Plugin for end to end testing report
Create Freestyle Project or whatever your needs
Go to Build Section -> Add build step and choose Execute Windows
batch command if Jenkins server in Windows otherwise choose Execute
Shell for Linux
Call conf.js (install packages and call your configuration file)
For reporting Got to Post-Build Actions Section -> Add Publish Html
Reports and call your report file (file assuming from root of your
project)
However you can customize execution command using gulp or similar other packages. Thanks