Generate HTML report for WebdriverIO/Cucumber framework - selenium

I am using WebdriverIO/Cucumber (wdio-cucumber-framework) for my test automation. I want to get the test execution result in a HTML file. As of now I am using Spec Reporter (wdio-spec-reporter). Which helps to print the results in console window. But I want all the execution reports in a HTML file.
How can I get WebdriverIO test execution result in a HTML file?
Thanks.

OK, finally got some spare time to tackle your question #Thangakumar D. WebdriverIO reporting is a vast subject (there are multiple ways to generate such a report), so I'll go ahead and start with my favorite reporter: Allure!
Allure Reporter:
[Preface: make sure you're in your project root]
Install your package (if you haven't already): npm install wdio-allure-reporter --save-dev
Install Allure CommandLine (you'll see why later): npm install -g allure-commandline --save-dev
Setup your wdio.config.js file to support Allure as a reporter
wdio.config.js:
reporters: ['allure', 'dot', 'spec', 'json'],
reporterOptions: {
outputDir: './wdio-logs/',
allure: {
outputDir: './allure-reports/allure/'
}
}
Run your tests! Notice that, once your regression ends, your /allure-results/ folder has been populated with multiple .json, .txt, .png (if you have screenshot errors), and .xml files. The cotent of this folder is going to be used by Allure CommandLine to render you HTML report.
Go to your /allure-results/ folder and generate the report via: allure generate <reportsFolderPath> (do it like this allure generate .
If you want your /allure-reports/ folder inside /allure-results/)
Now go into your /allure-reports folder and ope index.html into your browser of choice (use Firefox for starters)
Note: The generated index.html file won't have all the content loaded on Chrome unless you do some tweaks. It's due to default WebKit not being able to load all the AJAX calls required. Read more about it here.
If you're successfully completed all the previous steps, it should look something like this:
Hope this helped. Cheers!
Note: I'll try to UPDATE this post when I get some more time with other awesome ways to generate reports from your WebdriverIO reporter logs, especially if this post gets some love/upvotes along the way.
e.g.: Another combo that I enjoy using being: wdio-json-reporter/wdio-junit-reporter coupled with a easy-to-use templating language, Jinja2.

I have been using Mochawesome reporter and it looks beautiful, check it out
here.
Mochawesome reporter generates the mochoawesome.json which then can be used to create a beautiful report using Mochawesome report generator
Installation:
> npm install --save wdio-mochawesome-reporter
> npm install --save mochawesome-report-generator#2.3.2
It is easier to integrate by adding this line in the wdio.conf.js:
// sample wdio.conf.js
module.exports = {
// ...
reporters: ['dot', 'mochawesome'],
reporterOptions: {
outputDir: './', //mochawesome.json file will be written to this directory
},
// ...
};
Add the script to package.json:
"scripts": {
"generateMochawesome": "marge path/to/mochawesome.json --reportTitle 'My project results'"
},

Related

How can I reduce the webpack bundle size for a Vue.js / Nuxt.js project incorporating AWS SDK?

Summary:
I have created projects, with Vue.js and Nuxt.js, where I have installed aws-amplify (which automatically installs aws-sdk) in order that I can implement authentication with AWS Cognito.
In both cases, this works very nicely, but the problems come when I build production versions.
In both cases, I end up with massive bundle sizes which (thanks to webpack-bundle-analyzer) I can immediately see are caused by the aws-sdk which appears to contain code to implement every AWS service, under the sun, despite the fact that I am only importing AWS Cognito's: "Auth" (import { Auth } from 'aws-amplify')
I have tried creating a custom AWS SDK for JavaScript, which only includes the service: AWS.CognitoIdentity, but despite incorporating that (presumably incorrectly), I still end up with the same bundle size (and aws-sdk files) when I build the projects.
As I say, this is happening in both Nuxt and Vue project, but in order to simplify this, I for now just want to find the solution to a very basic sample project created with Vue.
I think I must be doing something dumb, but I can't work out what that is.
Any help would be greatly appreciated! :-)
Steps to reproduce:
Create a basic Vue.js project with defaults. Run: vue create vue-aws-sdk-inv
[Note: Steps 2 - 4, are not crucial to reproduce issue, but install webpack-bundle-analyzer which provides useful extra info.]
In the new project, install webpack-bundle-analyzer. Run: npm install --save-dev webpack-bundle-analyzer
Create root file: vue.config.js
Add the following code to vue.config.js:
const BundleAnalyzerPlugin = require("webpack-bundle-analyzer")
.BundleAnalyzerPlugin;
module.exports = {
configureWebpack: {
plugins: [new BundleAnalyzerPlugin()]
}
};
As a benchmark, build the project. Run: npm run build
At this stage, the project will build (with no console warnings) and webpack-bundle-analyzer will launch (in the browser) showing the file: chunk-vendors..js, at the top of the tree, containing a bunch of other .js files, all of acceptable size.
Install AWS Amplify (and by default aws-sdk). Run: npm i aws-amplify
Open src/components/HelloWorld.vue and add the following under the tag: import { Auth } from "aws-amplify";
Build the project. Run: npm run build
At this stage, the project will build WITH console warnings regarding the following files being too large:
File Size Gzipped
dist/js/chunk-vendors.013ac3f0.js 3055.78 KiB 550.49 KiB
dist/js/app.fa2a21c4.js 4.67 KiB 1.67 KiB
dist/css/app.53019c4c.css 0.33 KiB 0.23 KiB
If installed, webpack-bundle-analyzer should launch (in the browser) showing an inflated: chunk-vendors..js, due to a hefty: aws-sdk.
aws-sdk will include api: .json files and lib: .js files for every AWS service I can think of!
The attempt to rectify:
Navigate to: https://sdk.amazonaws.com/builder/js/
Clear all services.
Select just: AWS.CognitoIdentity
Download "Minified" aws-sdk-.js
Download "Default" aws-sdk-.min.js
[Note: the following are the steps I am guessing I'm getting wrong?...]
In the project, search the node_modules directory for aws-sdk.js and aws-sdk.min.js.
They were found in /node_modules/aws-sdk/dist
Replace both files with the downloaded files (renaming to aws-sdk.js and aws-sdk.min.js respectively.)
Build the project. Run: npm run build
Project will build with same console warnings and same massive aws-sdk, as before, containing all the same .js and .json files for a bunch of services that are not actually imported in the application.
Final pieces of analysis:
Remove aws-sdk.js and aws-sdk.min.js from project's: /node_modules/aws-sdk/dist
Build the project. Run: npm run build
Project is built without even referencing these files.
Rename /node_modules/aws-sdk to /node_modules/TEMP_aws-sdk and attempt to build the project.
Build fails, and this proves (I think) that I was at least trying to add the custom versions, of aws-sdk.js and aws-sdk.min.js, somewhere in the correct directory!
Source Code:
vue.config.js:
const BundleAnalyzerPlugin = require("webpack-bundle-analyzer")
.BundleAnalyzerPlugin;
module.exports = {
configureWebpack: {
plugins: [new BundleAnalyzerPlugin()]
}
};
src/components/HelloWorld.vue:
import { Auth } from "aws-amplify";
As said before, any help would be greatly appreciated! :-)
It looks like import { Auth } from "aws-amplify"; doesn't currently allow for tree shaking according to this issue.
Reading through several related issues, it appears that:
import Auth from '#aws-amplify/auth';
is the best you can currently do. I suspect that over time, the AWS team will figure out a way to better separate the internals.
For readers looking for a way to reduce bundle sizes for the aws-sdk package, see this section of the docs.
In my case:
import S3 from 'aws-sdk/clients/s3';
import AWS from 'aws-sdk/global';
cut the bundle size down by quite a lot. That gets it down to ~57k gz to use S3.
Also, for anyone using nuxt you can just run nuxt build -a to get the build analyzer.

Allure Report Executes but Fails to Populate Data - Jasmine and Protractor

I have set up a Protractor project that uses Allure Reporting. I have managed to get Allure Reporter outputting the HTML file to the allure-report folder, but upon opening the page in the browser it just says "Loading..." on the overview page. When I click on the other tabs, I get a 404 page not found message.
The package.json file contains a scripts section that specifies the test and post-test scripts to run:
"scripts": {
"test": "protractor conf.js",
"posttest": "allure generate allure-results --clean -o allure-report || true"
},
The problem seems to reside in the line:
allure generate allure-results --clean -o allure-report || true
Can anyone advise what I would need to change in package.json in order to populate my report? Is there something obvious I'm missing?
When I run allure serve allure-results
I get the formatted HTML report as expected, but I need to automate the process.
You can view the report if you open index.html file in FireFox browser. Probably you have opened this in Chrome browser. The error you are getting because of default Webkit security settings. Please refer to this answer.

process.env.NODE_ENV is not working with webpack3 [duplicate]

I've got an existing code base in which Vue.js has performance problems. I also see this notice in the browser console:
so I guess an easy fix could be to put Vue into production mode.
In the suggested link I try to follow the instructions for webpack. We're on Webpack version 2.7 (current stable version is 4.20). In the instructions it says that in Webpack 3 and earlier, you’ll need to use DefinePlugin:
var webpack = require('webpack')
module.exports = {
// ...
plugins: [
// ...
new webpack.DefinePlugin({
'process.env.NODE_ENV': JSON.stringify('production')
})
]
}
So in my package.json I've got a build script defined:
To build for production I run yarn run build and it runs a build.js file (paste here) which in turn calls webpack.base.conf.js (paste here) and webpack.prod.conf.js (paste here).
As you can see in the paste I use the DefinePlugin as suggested by the docs.
I also found a file called vue-loader.conf.js (paste here) and to be sure I also added the DefinePlugin in there as well.
I can run yarn run build which ends without errors, but when serve the site over Apache and open the browser it still shows the notification that we're in development mode.
To be sure it actually uses the files created by webpack I completely removed the folder /public/webpack/ and checked that the webinterface didn't load correctly without the missing files and then built again to see if it loaded correctly after the command finished. So it does actually use the files built by this webpack process. But Vue is actually not created in production mode.
What am I doing wrong here?
The problem may be in your 'webpack.base.conf.js' as i suspected, thank you for sharing it, upon searching i've found an issue resolving your 'production not being detected' problem on github here
The solution requires that you change 'vue$': 'vue/dist/vue' to 'vue$': vue/dist/vue.min in production.
You will find the original answer as:
#ozee31 This alias 'vue$': 'vue/dist/vue' cause the problem, use vue/dist/vue.min in production environment.

How to integrate karma in webpack

I am a webpack newbie and have a question about testing.
I have a project which uses webpack, typescript and karma as test runner and I would like to run my tests on every file change (e.g. in "watch" mode)
I am aware of karma-webpack and it works well when I run karma as own process (karma start ...)
But what I want is to integrate karma in the webpack flow.
So, from my naive point of view, I thought karma has to be defined in preloading of webpack (such as a linter).
But I found nothing....
I can not believe that this common workflow is not possible (run tests on every source change)
Can anybody of you give me a suggestion?
Time flies and it's June 2018 already. As there isn't much documentation about this online I want to give out my 2 cents.
I have currently a setup working where I bundle my tests with webpack a watch for changes in order to rerun automatically the tests.
I'm using karma-webpack using the configuration explained in the Alternative usage section and I think it's the proper way to solve the problem asked in the question.
karma.conf.js
{
...
files: [
// only specify one entry point
// and require all tests in there
'test/index_test.js'
],
preprocessors: {
// add webpack as preprocessor
'test/index_test.js': [ 'webpack' ]
},
...
}
test/index_test.js
// require all modules ending in "_test" from the
// current directory and all subdirectories
const testsContext = require.context(".", true, /_test$/)
testsContext.keys().forEach(testsContext)
Watching for changes of the whole bundle as #Adi Prasetyo suggests is wrong in my opinion. We should only watch for changes in the tests files and the files that are imported inside them, which can be achieved with configuration shown in the last URL.
Very important for hot reloading to work (at least in my case it was what made it work), you need to setup webpack-dev-middleware configuration in order to update the tests bundle everytime a change happens in some test file or the files being imported inside them. This is the config I'm using:
karma.config.js
{
...
webpackMiddleware: {
watchOptions: {
aggregateTimeout: 300,
poll: 1000, // customize depending on PC power
},
},
...
}
More info about it here.
I have same problem, as the TDD workflow that i use. After writing test then change the code the test doesn't re-run. Run test on every file change is possible.
As karma files have 3 options: Included, served, watched.
You can specify the bundle as pattern then tell karma to watch it
karma.config.js
files: [
// watch final file so when source change and it's final file, re run the test
{ pattern: './dist/js/*.wp.js', watched: true},
],
but when we use karma start no webpack active and watching. So use concurrently to run karma and webpack. Note that webpack should watch only the source code and karma should watch the bundle file.
Then we can add script property to package.json like so
package.json
"scripts": {
"test": "karma start karma.config.js",
"build": "webpack",
"dev": "concurrently \"webpack --progress --colors --watch\" \"karma start karma.config.js --colors\"",
},
Then run npm run dev to start coding
Well I've never heard of webpack up until now, but I know karma fairly well. I'm not 100% sure what you are asking here, so let me know if this isn't helpful at all. You can setup karma to do exactly what you want it to do (run tests on every file change).
There are two options that you must set in the karma.conf.js file: autoWatch: true and singleRun: false.
karma.conf.js
module.exports = function(config) {
config.set({
// set other options and stuff...
autoWatch: true,
singleRun: false
});
};
autoWatch set to true enables watching files and executing tests whenever any file changes. Setting singleRun to false means that you only need to execute karma start (or however you integrate it into webpack) once to run your tests, instead of having to enter the command every single time you make a change or want to run your suite; it just keeps karma running in the background.

Passing cli parameters to casperjs through grunt task and npm test

I'm running tests with npm test - that actually runs a grunt task grunt casperjs:
casperjs:{
options:{},
files:
['./test_index.js',
'./test_map_regression.js',
'./test_index_get_gush.js'] /
},
using the grunt-casperjs-plugin in order to automate testing with slimerjs along with phantomjs, both running under casperjs in Travis-ci.
In order to do that, I need to pass the engine as a variable from the command line. something like:
casperjs --engine=slimerjs test_suite.js
Question: I can't find a way to pass the options from grunt cli (and I assume npm command line options would delegate to grunt. correctly?) to the files array.
I tried to add:
var engine = grunt.option('engine') || 'phantomjs';
engine = '--engine='+engine;
and then in the file array do:
files:['./test_index.js '+engine,
'./test_map_regression.js '+enging,
'./test_index_get_gush.js '+engine]
but seems that file array has to get real file names without the added args.
I'll be glad for any ideas on how to solve this through.
I haven't tested this, but looking at the grunt-casperjs source, it looks as though you would want to pass the engine as an option.
So, something like this should work:
casperjs:{
options: {
'engine': 'slimerjs'
},
files: [
'./test_index.js',
'./test_map_regression.js',
'./test_index_get_gush.js'
]
}