Can I run Nightwatch js in paralell with a different env per worker? - testing

I'm currently using Nightwatch js to run my E2E tests in paralell.
The issue I'm having is that my tests share the same database, which is causing me problems with shared data being rewritten across different tests/workers, resulting in flaky tests.
I thought about running each test worker with its own database, but I'm not sure how to do it in practice. My starting point would be having different settings for the test databases, each on a night-watch env which can be accessed by the individual workers, but so far I've not found if this would be possible.
Any ideas?

Yes you can run different tests on different workers, Nightwatch provides a way to do that. I have done something very similar where I ran the same test on three different browsers parallely.
You have to add the test_Worker configuration, which allows the tests to be run in parallel. When this is enabled the test runner will launch a configurable number of child processes and then distribute the loaded tests over to run in parallel.
The workers option configures how many child processes can run concurrently.
a) “auto” – determined by the number of CPUs e.g. 4 CPUs means 4 workers
b) {number} – specifies an exact number of workers
test_workers: {
enabled: true,
workers: 'auto'
}
This is how my Nightwatch.conf.js file looks like:
{
module.exports = {
src_folders: ["tests"],
test_settings: {
default: {
desiredCapabilities: {
browserName: 'chrome'
},
webdriver: {
start_process: true,
port: 4444,
server_path: require('chromedriver').path,
}
},
test_workers: {
enabled: true,
workers: 'auto'
},
safari: {
desiredCapabilities: {
browserName: 'safari',
alwaysMatch: {
acceptInsecureCerts: false
}
},
webdriver: {
port: 4445,
start_process: true,
server_path: '/usr/bin/safaridriver'
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox'
},
webdriver: {
start_process: true,
port: 4446,
server_path: require('geckodriver').path
}
}
}
}
Now to execute the test I use the command:
npx nightwatch tests/TC001_WikiSearch.js -e default,firefox,safari
Referenced from: https://testersdock.com/execute-parallel-tests-nightwatchjs/

Related

Custom args for each instance of Chrome when running parallel tests with Protractor

Is it possible to pass custom args for each instance of Chrome when running parallel tests in Protractor? I need to know the Remote debugging port for each instance so I can connect with the Dev Tools protocol.
As I see it there's two options. Set the port to a specific unique value for each instance, or let it be set automatically and fetch it somehow when preparing the tests. Would it be possible with any of these options?
exports.config = {
framework: 'jasmine',
chromeDriver: chromeDriverPath,
multiCapabilities: [{
browserName: 'chrome',
chromeOptions: {
args: process.env.HEADLESS && puppeteer ? ['--headless', `--remote-debugging-port=${DEV_TOOLS_PORT}`] : [`--remote-debugging-port=${DEV_TOOLS_PORT}`],
binary: puppeteer.executablePath()
},
shardTestFiles: true,
maxInstances: 1
}]
}
If I got the problem right, what you could do is to pass parameters as env variables to protractor on start. So your config would look like this:
exports.config = {
framework: 'jasmine',
chromeDriver: chromeDriverPath,
multiCapabilities: [{
'browserName': 'chrome',
'chromeOptions': {
args: [`--remote-debugging-port=${process.env.PORT_ONE}`]
}
}, {
'browserName': 'chrome',
'chromeOptions': {
args: [`--remote-debugging-port=${process.env.PORT_TWO}`]
}
}]
}
And then start your protractor process with env variables like so:
PORT_ONE=90 PORT_TWO=80 protractor protractor.conf.js
One of the two options are solved. But I would still like to know if there's a possibility to use custom args for each instance.
// Get Remote debugging port for chrome
let chromeRemoteDebuggingPort;
browser.getCapabilities().then((capabilities) => {
const chromeOptions = capabilities.get('goog:chromeOptions');
if( chromeOptions && chromeOptions.debuggerAddress) {
chromeRemoteDebuggingPort = chromeOptions.debuggerAddress.split(':')[1];
}
});

How to skip protractor+jasmine tests specific to browsers

Assume that I've automated 25 tests and executing in multiple browsers like chrome, firefox, IE, Edge & Safari. All tests (25) are executing well on Chrome. In Firefox, Only 20 tests are running fine due to few protractor APIs are not supported. Similarly IE can execute only 23 tests.
I would like to skip the test only for browsers, which are not supported for particular test? Is there any way available?
You can create protracotr.conf file for each browser with specific suites where will be specified what tests should run. And execute in one time all protractor.conf files.
//protractor.chrome.conf
export let config: Config = {
...
capabilities: {
browserName: 'chrome',
shardTestFiles: true,
maxInstances: 1
},
SELENIUM_PROMISE_MANAGER: false,
specs: [
'../test/chrome/**/*.js'
]
};
and
//protractor.ie.conf
export let config: Config = {
...
capabilities: {
browserName: 'internet explorer',
shardTestFiles: true,
maxInstances: 1
},
SELENIUM_PROMISE_MANAGER: false,
specs: [
'../test/ie/**/*.js'
]
};
in your package.json:
{
...
"scripts": {
"test:all": "npm run test:chrome && test:ie",
"test:chrome": "protractor ./config/protractor.chrome.conf.js",
"test:ie": "protractor ./config/protractor.ie.conf.js",
...
},
...
}
With jasmine2 you can filter tests using a regular expression. Maybe you can add something like #chrome, #ie to your tests and then only run those ones by passing the grep flag:
it('should do stuff #ie #chrome', function() {
...
});
Then run protractor passing the grep flag:
protractor conf.js --grep='#ie'

theintern calling /__intern/client.htm in URL

I want to run intern tests locally with selenium-standalone both of which i installed through npm.
when i go to run the tests -> "./node_modules/.bin/intern-runner" config=./pmictests/test/bit/GAT/internEx/intern
the browser starts but the url goes to http://localhost:8585/__intern/client.html?config=.%2Fpmictests%2Ftest%2Fbit%2FGAT%2FinternEx%2Fintern&basePath
As in the _intern/client.html? is not what i want
why is this happening? i'm trying to get my head around it but been stuck on this problem for a while.
my config file looks like this:
define({
proxyPort: 9515,
proxyUrl: 'http://localhost:8585/',
tunnel: 'NullTunnel',
useSauceConnect: false,
capabilities: {
'fixSessionCapabilities' : false,
'selenium-version': '2.35.0',
'idle-timeout': 36
},
environments: [
{ browserName: 'chrome' }
],
maxConcurrency: 3,
useSauceConnect: false,
webdriver: {
host: 'localhost',
port: 4444
},
suites: [ './tests/test/' ],
excludeInstrumentation: /^(?:tests|node_modules)\//
});
That URL is for running unit tests. When you run intern-runner, it automatically loads client.html to run any unit test suites listed in suites. Once the unit tests are finished, Intern runs any functional tests listed in functionalSuites (which will load their own URLs).

Webdriver.io - visual regression Service compare not recognized with multi remote

I got visual regression working in webdriver.io and it is very wonderful. I would, however, like to also use it with multi remote. When I add the necessary configuration to my wdio.conf.js file for multiremote, I get the following error
Error: Please provide a visualRegression configuration with a compare method in your wdio-conf.js!
when running my tests and the regression service no longer works.
My relevent wdio.conf.js settings are configured as such:
//...
capabilities: {
browserA: {
desiredCapabilities: {
browserName: 'chrome'
}
},
browserB: {
desiredCapabilities: {
browserName: 'chrome'
}
}
},
// ...
services: ['visual-regression'],
visualRegression: {
compare: new VisualRegressionCompare.LocalCompare({
referenceName: getScreenshotName(path.join(process.cwd(), 'screenshots/reference')),
screenshotName: getScreenshotName(path.join(process.cwd(), 'screenshots/taken')),
diffName: getScreenshotName(path.join(process.cwd(), 'screenshots/diff')),
}),
viewportChangePause: 500,
widths: [1280, 1600],
orientations: ['landscape', 'portrait'],
},
Any help would be much appreciated!

Protractor pointing to Sauce Labs Selenium Server

I'm trying to integrate Protractor with Sauce Labs from Travis. I can get the sauce_connect server running correctly but am unable to get Travis to point to that particular remote server.
Travis will get to the point where it initiates sauce_connect but when I run "protractor:analytics" it doesn't point to the correct server and fails.
Travis.yml:
language: python
python:
- 3.2_with_system_site_packages
branches:
only:
- develop
before_install:
- sudo apt-get update -qq
- sudo apt-get install python-numpy
install:
- cd lib && python setup.py install
- cd .. && pip install -r requirements/travis_requirements.txt
- npm install
script:
- grunt karma:single
- grunt protractor:analytics
env:
global:
- secure: <string>
- secure: <string>
sauce_connect: true
Gruntfile:
protractor: {
options: {
configFile: './webapp/static/test/e2e/protractor.conf.js',
keepAlive: true
},
singlerun: {},
analytics: {
options: {
//debug : true,
args:{
specs: ['./webapp/static/test/e2e/analytics_spec.js']
}
}
},
},
Protractor Conf:
exports.config = {
chromeOnly: false,
seleniumArgs: [],
// If sauceUser and sauceKey are specified, seleniumServerJar will be ignored.
// The tests will be run remotely using SauceLabs.
sauceUser: process.env.SAUCE_USER,
sauceKey: process.env.SAUCE_KEY,
baseUrl: 'http://localhost:8000',
specs: [
'./*_spec.js',
],
// Patterns to exclude.
exclude: [],
multiCapabilities: [],
// ----- More information for your tests ----
//
// A base URL for your application under test. Calls to protractor.get()
// with relative paths will be prepended with this.
baseUrl: process.env.SN_BASE_URL,
// Selector for the element housing the angular app - this defaults to
// body, but is necessary if ng-app is on a descendant of <body>
rootElement: 'body',
// A callback function called once protractor is ready and available, and
// before the specs are executed
// You can specify a file containing code to run by setting onPrepare to
// the filename string.
onPrepare: function() {
// At this point, global 'protractor' object will be set up, and jasmine
// will be available. For example, you can add a Jasmine reporter with:
// jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(
// 'outputdir/', true, true));
},
// The params object will be passed directly to the protractor instance,
// and can be accessed from your test. It is an arbitrary object and can
// contain anything you may need in your test.
// This can be changed via the command line as:
// --params.login.user 'Joe'
params: {
login: {
user: process.env.SN_TEST_USERNAME,
password: process.env.SN_TEST_PASSWORD
}
},
framework: 'jasmine',
// ----- Options to be passed to minijasminenode -----
//
// See the full list at https://github.com/juliemr/minijasminenode
jasmineNodeOpts: {
// onComplete will be called just before the driver quits.
onComplete: null,
// If true, display spec names.
isVerbose: false,
// If true, print colors to the terminal.
showColors: true,
// If true, include stack traces in failures.
includeStackTrace: true,
// Default time to wait in ms before a test fails.
defaultTimeoutInterval: 30000
},
onCleanUp: function() {}
};
If I did understand well : Sauce connect tool is not used by protractor/selenium when running a test suite.
Well I had this problem, travis requires sauce credentials and protractor requires those credentials and a tunnel id:
.travis.yml:
addons:
sauce_connect:
username: xxx
access_key: xxx
protractor.conf.js:
exports.config = {
...
sauceUser: process.env.SAUCE_USERNAME,
sauceKey: process.env.SAUCE_ACCESS_KEY,
capabilities: {
...
'tunnel-identifier': process.env.TRAVIS_JOB_NUMBER,
}
}