Webdriver.io test runner - exclude suite from specific capability - selenium

My webdriver.io test runner is organized into suites, and I want to exclude a suite from a specific capability.
For example, I have a suite called suite1:
//
suites: {
suite1: [
'./test/spec/*.js',
],
},
//
And I want to exclude this suite from the Safari capability:
//
{
'tunnelIdentifier' : sltunnel,
'browserName': 'safari',
'version': '10',
'platform': 'OS X 10.12',
'screenResolution': '1600x1200',
'exclude': [
suite1,
],
},
//
Is there a way to achieve this?

Of course there are multiple ways to do this, but at the end of the day it's all up to you.
You can drag your entire suite logic outside of your wdio.conf.js file into a separate config file (I'm sure you might already have one for your environments: test, prod, w/e) and then import it (via require into your wdio.conf.js file):
So let's say you have suites.conf.js file:
module.exports.suites = {
// 1 => run the suite
// 0 => don't run it
"chrome": {
"0": [
"<yourSuiteName-001>.js"
"<yourSuiteName-003>.js"
"<yourSuiteName-005>.js"
],
"1": [
"<yourSuiteName-002>.js"
"<yourSuiteName-004>.js"
]
},
"safari" : {
"0": [
"<yourSuiteName-002>.js"
"<yourSuiteName-004>.js"
"<yourSuiteName-005>.js"
],
"1": [
"<yourSuiteName-001>.js"
"<yourSuiteName-003>.js"
]
}
// <etc>
}
Then you import it into your wdio.conf.js file (var suites = require(./<pathToSuitesConfig>/suites.conf.js);), cycle with a forEach through all your suites and based on your browserName, create a buffer in which you store the runable suites as they match your condition, then add them into the config file. This logic will go in your wdio.conf.js header section and you can update the exportable config via the results.
I just thought of this in like 5 minutes, so I'm sure there's multiple cleaner ways to achieve this.
Alternatively, you can look at other test-runners that are easy to plug into WebdriverIO and which have tagging:
Cucumber has a great tagging mechanism (due to Gherkin syntax);
Mocha has something similar called, you guessed it tagging.

Related

How to exclude specific test in wdio.conf.js

I'm successfully excluding a whole feature file from my chrome runs, using the code below, but I have one specific test in a different feature file failing. I want to exclude the one failing scenario rather than the whole file. Is this possible?
capabilities: [
{
os: "Windows",
os_version: "10",
browser: "Chrome",
exclude: [
'features/myfeature.feature',
'my failing scenario',
],
},
],
No, it's not possible to exclude scenarios in that way. Instead, you could use tags to skip a particular scenario, similar to how it's done via the cucumber-boilerplate webdriverio project:
cucumberOpts: {
tagExpression: 'not #Pending',
},
You may add tag to this scenario e.g. #KnownIssue and exclude this tag by config
cucumberOpts:
{
....
tagExpression: "not #KnownIssue"
.....
}

How to skip protractor+jasmine tests specific to browsers

Assume that I've automated 25 tests and executing in multiple browsers like chrome, firefox, IE, Edge & Safari. All tests (25) are executing well on Chrome. In Firefox, Only 20 tests are running fine due to few protractor APIs are not supported. Similarly IE can execute only 23 tests.
I would like to skip the test only for browsers, which are not supported for particular test? Is there any way available?
You can create protracotr.conf file for each browser with specific suites where will be specified what tests should run. And execute in one time all protractor.conf files.
//protractor.chrome.conf
export let config: Config = {
...
capabilities: {
browserName: 'chrome',
shardTestFiles: true,
maxInstances: 1
},
SELENIUM_PROMISE_MANAGER: false,
specs: [
'../test/chrome/**/*.js'
]
};
and
//protractor.ie.conf
export let config: Config = {
...
capabilities: {
browserName: 'internet explorer',
shardTestFiles: true,
maxInstances: 1
},
SELENIUM_PROMISE_MANAGER: false,
specs: [
'../test/ie/**/*.js'
]
};
in your package.json:
{
...
"scripts": {
"test:all": "npm run test:chrome && test:ie",
"test:chrome": "protractor ./config/protractor.chrome.conf.js",
"test:ie": "protractor ./config/protractor.ie.conf.js",
...
},
...
}
With jasmine2 you can filter tests using a regular expression. Maybe you can add something like #chrome, #ie to your tests and then only run those ones by passing the grep flag:
it('should do stuff #ie #chrome', function() {
...
});
Then run protractor passing the grep flag:
protractor conf.js --grep='#ie'

Webdriver.io - visual regression Service compare not recognized with multi remote

I got visual regression working in webdriver.io and it is very wonderful. I would, however, like to also use it with multi remote. When I add the necessary configuration to my wdio.conf.js file for multiremote, I get the following error
Error: Please provide a visualRegression configuration with a compare method in your wdio-conf.js!
when running my tests and the regression service no longer works.
My relevent wdio.conf.js settings are configured as such:
//...
capabilities: {
browserA: {
desiredCapabilities: {
browserName: 'chrome'
}
},
browserB: {
desiredCapabilities: {
browserName: 'chrome'
}
}
},
// ...
services: ['visual-regression'],
visualRegression: {
compare: new VisualRegressionCompare.LocalCompare({
referenceName: getScreenshotName(path.join(process.cwd(), 'screenshots/reference')),
screenshotName: getScreenshotName(path.join(process.cwd(), 'screenshots/taken')),
diffName: getScreenshotName(path.join(process.cwd(), 'screenshots/diff')),
}),
viewportChangePause: 500,
widths: [1280, 1600],
orientations: ['landscape', 'portrait'],
},
Any help would be much appreciated!

Karma, empty test suite

First of all I know there are similar questions, but none of it works for me.
I have an Angular app, and I would like to test it with Karma, Jasmin, and RequireJs.
I have installed everything I need, and configured everything as it is written here: http://karma-runner.github.io/0.13/plus/requirejs.html
Below is how my files look like
karma.conf.js
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['jasmine', 'requirejs'],
// list of files / patterns to load in the browser
files: [
'src/test/Tests/*.js',
'src/test/Tests/directives/*.js',
//source code
{pattern: 'src/main/webapp/app/*.js', included: false},
{pattern: 'src/main/webapp/app/**/*.js', included: false},
{pattern: 'src/test/test-main.js', included: true}
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'test/Tests/*.js': 'coverage'
},
coverageReporter:{
type:'html',
dir:'../coverage/'
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress', 'coverage'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['PhantomJS'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false,
// Concurrency level
// how many browser should be started simultanous
concurrency: Infinity
})
}
test-main.js
var TEST_REGEXP = /(spec|test)\.js$/i;
var allTestFiles = [];
// Get a list of all the test files to include
Object.keys(window.__karma__.files).forEach(function(file) {
if (TEST_REGEXP.test(file)) {
// Normalize paths to RequireJS module names.
// If you require sub-dependencies of test files to be loaded as-is (requiring file extension)
// then do not normalize the paths
var normalizedTestModule = file.replace(/^\/base\/|\.js$/g, '');
allTestFiles.push(normalizedTestModule);
}
});
require.config({
// Karma serves files under /base, which is the basePath from your config file
baseUrl: '/base',
//dynamically load all test files
deps: allTestFiles,
// we have to kickoff jasmine, as it is asynchronous
callback: window.__karma__.start
});
The output is in IntelliJ IDEA 14.1.5
Empty test suite.
Process finished with exit code 0
The server starts correctly. I have also tried with command line. $ karma start, and $ karma run, which also does nothing. There is no error message at all. The path is correct, there is no 404 error.
Do you have any guess what could be the problem?
Delete the module names from test files.
In my case, this was the solution.
According to RequireJS documentation (http://requirejs.org/docs/api.html#modulename) you can include a name for the module as you can see below:
//Explicitly defines the "foo/title" module:
define("foo/title",
["my/cart", "my/inventory"],
function(cart, inventory) {
//Define foo/title object in here.
}
);
But for me, Karma only executes the tests if I delete the "foo/title" name.

How to run a specific test case from a test suite using InternIO

Currently, I have a few test suites. To avoid running an entire test suite - how can I run just a single test case from a suite. My current workaround is to comment out test cases I don't want.
My intern.js file:
define({
environments: [
{ browserName: 'chrome'},
],
useSauceConnect: false,
//functionalSuites: ['tests/projects/projectA/main.js'],
tunnel: 'NullTunnel',
excludeInstrumentation: /./
});
The following is my main.js file:
define([
'intern',
'intern!object',
'intern/chai!assert'
],
function (intern, registerSuite, assert) {
registerSuite(function () {
var someLib;
return {
setup: function () {
},
'TC001 - Functionality 1': function () {
},
'TC002 - Functionality 2': function() {
},
'TC003 - Functionality 3': function() {
},
};
});
});
I am running this by using the command-line: To run - I issue the following command:
$ node_modules/.bin/intern-runner config=tests/intern functionalSuites=tests/projects/projectA/main.js
Is there a way to run say only "TC001 - Functionality 1" and other selected test cases without commenting out the other tests cases from the suite?
You can use the grep command line option. The grep option specifies a regular expression that filters on test ID (test IDs that match are run), giving you a lot of flexibility when specifying which tests will run. At it's simplest, you can just provide a short string, and Intern will run any tests with IDs containing that string:
$ node_modules/.bin/intern-runner config=tests/intern functionalSuites=tests/projects/projectA/main grep=TC001