I got visual regression working in webdriver.io and it is very wonderful. I would, however, like to also use it with multi remote. When I add the necessary configuration to my wdio.conf.js file for multiremote, I get the following error
Error: Please provide a visualRegression configuration with a compare method in your wdio-conf.js!
when running my tests and the regression service no longer works.
My relevent wdio.conf.js settings are configured as such:
//...
capabilities: {
browserA: {
desiredCapabilities: {
browserName: 'chrome'
}
},
browserB: {
desiredCapabilities: {
browserName: 'chrome'
}
}
},
// ...
services: ['visual-regression'],
visualRegression: {
compare: new VisualRegressionCompare.LocalCompare({
referenceName: getScreenshotName(path.join(process.cwd(), 'screenshots/reference')),
screenshotName: getScreenshotName(path.join(process.cwd(), 'screenshots/taken')),
diffName: getScreenshotName(path.join(process.cwd(), 'screenshots/diff')),
}),
viewportChangePause: 500,
widths: [1280, 1600],
orientations: ['landscape', 'portrait'],
},
Any help would be much appreciated!
Related
I'm currently using Nightwatch js to run my E2E tests in paralell.
The issue I'm having is that my tests share the same database, which is causing me problems with shared data being rewritten across different tests/workers, resulting in flaky tests.
I thought about running each test worker with its own database, but I'm not sure how to do it in practice. My starting point would be having different settings for the test databases, each on a night-watch env which can be accessed by the individual workers, but so far I've not found if this would be possible.
Any ideas?
Yes you can run different tests on different workers, Nightwatch provides a way to do that. I have done something very similar where I ran the same test on three different browsers parallely.
You have to add the test_Worker configuration, which allows the tests to be run in parallel. When this is enabled the test runner will launch a configurable number of child processes and then distribute the loaded tests over to run in parallel.
The workers option configures how many child processes can run concurrently.
a) “auto” – determined by the number of CPUs e.g. 4 CPUs means 4 workers
b) {number} – specifies an exact number of workers
test_workers: {
enabled: true,
workers: 'auto'
}
This is how my Nightwatch.conf.js file looks like:
{
module.exports = {
src_folders: ["tests"],
test_settings: {
default: {
desiredCapabilities: {
browserName: 'chrome'
},
webdriver: {
start_process: true,
port: 4444,
server_path: require('chromedriver').path,
}
},
test_workers: {
enabled: true,
workers: 'auto'
},
safari: {
desiredCapabilities: {
browserName: 'safari',
alwaysMatch: {
acceptInsecureCerts: false
}
},
webdriver: {
port: 4445,
start_process: true,
server_path: '/usr/bin/safaridriver'
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox'
},
webdriver: {
start_process: true,
port: 4446,
server_path: require('geckodriver').path
}
}
}
}
Now to execute the test I use the command:
npx nightwatch tests/TC001_WikiSearch.js -e default,firefox,safari
Referenced from: https://testersdock.com/execute-parallel-tests-nightwatchjs/
So I am trying to do something very simple yet so hard to understand for me the reason why its not working.
I am using browserstack combined with protractor and it is as simple as adding this to the config:
let SpecReporter = require('mochawesome').SpecReporter;
exports.config = {
baseUrl: 'http://testing.net/',
"browserstackUser": "test",
"browserstackKey": "1234",
multiCapabilities: [
{
os_version: "9.0",
device: "Samsung Galaxy S10 Plus",
real_mobile: false,
browserName: "Android",
project: "Selenium-Test",
build: "Build 1337",
name: "Mobile - Happy Flow"
}
],
mochaOpts: {
reporter: "mochawesome",
timeout: 60000,
},
suites: {
all: 'pages/*.js',
},
framework: 'mocha',
};
and whenever I run the code it starts by:
describe('FIRST TEST', function () {
var EC = protractor.ExpectedConditions;
browser.waitForAngularEnabled(false);
browser.manage().window().maximize();
brwoser.get('http://testing.net/');
}
and it seems to be stuck through here. I was watching the live demo of what is happening and it seems like it doesn't want to open the URL even thought I do have the link which is odd. and I am not sure if I am doing anything wrong for mobile version but yeah.
The problem is I can't open any URL for the phones but works well on desktop
Can you remove browser.manage().window().maximize(); for the mobile code
You can review a sample provided by BrowserStack in the link: https://github.com/browserstack/protractor-browserstack and execute the tests on similar lines
Is it possible to pass custom args for each instance of Chrome when running parallel tests in Protractor? I need to know the Remote debugging port for each instance so I can connect with the Dev Tools protocol.
As I see it there's two options. Set the port to a specific unique value for each instance, or let it be set automatically and fetch it somehow when preparing the tests. Would it be possible with any of these options?
exports.config = {
framework: 'jasmine',
chromeDriver: chromeDriverPath,
multiCapabilities: [{
browserName: 'chrome',
chromeOptions: {
args: process.env.HEADLESS && puppeteer ? ['--headless', `--remote-debugging-port=${DEV_TOOLS_PORT}`] : [`--remote-debugging-port=${DEV_TOOLS_PORT}`],
binary: puppeteer.executablePath()
},
shardTestFiles: true,
maxInstances: 1
}]
}
If I got the problem right, what you could do is to pass parameters as env variables to protractor on start. So your config would look like this:
exports.config = {
framework: 'jasmine',
chromeDriver: chromeDriverPath,
multiCapabilities: [{
'browserName': 'chrome',
'chromeOptions': {
args: [`--remote-debugging-port=${process.env.PORT_ONE}`]
}
}, {
'browserName': 'chrome',
'chromeOptions': {
args: [`--remote-debugging-port=${process.env.PORT_TWO}`]
}
}]
}
And then start your protractor process with env variables like so:
PORT_ONE=90 PORT_TWO=80 protractor protractor.conf.js
One of the two options are solved. But I would still like to know if there's a possibility to use custom args for each instance.
// Get Remote debugging port for chrome
let chromeRemoteDebuggingPort;
browser.getCapabilities().then((capabilities) => {
const chromeOptions = capabilities.get('goog:chromeOptions');
if( chromeOptions && chromeOptions.debuggerAddress) {
chromeRemoteDebuggingPort = chromeOptions.debuggerAddress.split(':')[1];
}
});
I'm attempting to run a simple test script. But I receive the following error:
I have my nightwatch config file setup as so:
nightwatch.conf.js
module.exports = {
"src_folders": [
"tests"// Where you are storing your Nightwatch e2e/UAT tests
],
"output_folder": "./reports", // reports (test outcome) output by nightwatch
"selenium": {
"start_process": true, // tells nightwatch to start/stop the selenium process
"server_path": "./node_modules/selenium-standalone/.selenium/selenium-server/2.53.1-server.jar",
"host": "127.0.0.1",
"port": 4444, // standard selenium port
"cli_args": { "webdriver.chrome.driver" : "./node_modules/selenium-standalone/.selenium/chromedriver/2.25-x64-chromedriver"
}
},
"test_settings": {
"default": {
"screenshots": {
"enabled": true, // if you want to keep screenshots
"path": './screenshots' // save screenshots here
},
"globals": {
"waitForConditionTimeout": 5000 // sometimes internet is slow so wait.
},
"desiredCapabilities": { // use Chrome as the default browser for tests
"browserName": "chrome"
}
},
"chrome": {
"desiredCapabilities": {
"browserName": "chrome",
"javascriptEnabled": true // set to false to test progressive enhancement
}
}
}
}
guinea-pig.js
module.exports = { // addapted from: https://git.io/vodU0
'Guinea Pig Assert Title': function(browser) {
browser
.url('https://saucelabs.com/test/guinea-pig')
.waitForElementVisible('body')
.assert.title('I am a page title - Sauce Labs')
.saveScreenshot('ginea-pig-test.png')
.end();
}
};
The server path and chromedriver path are accurate and the most recent copy. I also have the latest version of chrome installed. Can someone please help me understand what could be the problem? Thanks!
Edit: I've also restarted the whole computer, same issue.
Try using the latest version of the Selenium standalone server v.3.0.1
If that doesn't work, then you can upgrade your chromedriver to the latest version and test. You can find the different versions here:
https://chromedriver.storage.googleapis.com/index.html
Also make sure you are using the most recent version of Nightwatch v0.9.9 and update it in you package.json file.
It is written very clear , your chrome version is lower than what chromedriver needs, just update your chrome to the latest version
I've followed all the steps described here: https://github.com/theintern/intern/wiki/Using-Intern-with-PhantomJS
My intern config is ~ as follows:
define({
proxyPort: 9000,
proxyUrl: 'http://localhost:9000/',
environments: [
{ browserName: 'phantom' }
],
maxConcurrency: 3,
useSauceConnect: false,
webdriver: {
host: 'localhost',
port: 4444
},
reporters: ['runner'],
useLoader: {
'host-node': 'dojo/dojo',
'host-browser': 'node_modules/dojo/dojo.js'
},
loader: {
packages: [
{ name: 'myApp', location: '...' }
],
baseUrl: '...',
paths: {...}
},
suites: [
'test/hello'
],
functionalSuites: [],
excludeInstrumentation: /(^test(\/|\\)|reporters|node_modules)/
});
I run phantomJS with
.\node_modules\.bin\phantomjs --webdriver 4444 --webdriver-loglevel='debug'
and it listens on 4444.
I even disabled Windows Firewall, but still I get
ReferenceError: window is not defined
at ***.js:348:142
at Function.vm.runInThisContext (***\node_modules\intern\node_modules\istanbul\lib\hook.js:163:16)
at ***\node_modules\intern\node_modules\dojo\dojo.js:757:8
at fs.js:266:14
at Object.oncomplete (fs.js:107:15)
as though Intern is running on node, not in Phatom. Phantom's console is also completely silent.
What am I missing? Or is there a way to debug Intern's actions? Thx
OK, I've finally figured this out.
I've been running intern using
.\node_modules\.bin\intern-client config=test/intern
while it should've been
.\node_modules\.bin\intern-runner config=test/intern
Thing is that intern-runner and intern-client are two different applications, one is for running with browsers via WebDriver, the other one for running with Node. It didn't catch my eye even though I was reading and re-reading the docs much more than once. Probably the distinction should be highlighted there.
Hope this helps someone )