Tests that are run from Azure devops build pipeline fails sporadically with the below error.
Error: Unable to establish one or more of the specified browser connections. This can be caused by network issues or remote device failure.
at BrowserSet._waitConnectionsOpened (D:\a\1\s\src\test\Testcafe\node_modules\testcafe\src\runner\browser-set.ts:91:30)
at D:\a\1\s\src\test\Testcafe\node_modules\testcafe\src\runner\browser-set.ts:114:35
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at Bootstrapper._getBrowserConnections (D:\a\1\s\src\test\Testcafe\node_modules\testcafe\src\runner\bootstrapper.ts:216:16)
at async Promise.all (index 0)
at Bootstrapper._bootstrapParallel (D:\a\1\s\src\test\Testcafe\node_modules\testcafe\src\runner\bootstrapper.ts:392:38)
at Bootstrapper.createRunnableConfiguration (D:\a\1\s\src\test\Testcafe\node_modules\testcafe\src\runner\bootstrapper.ts:425:42)
Versions and settings used for running the test.
Testcafe: 1.8.8
Testrunner:
return runner
.browsers(['chrome:headless'])
.concurrency(1)
.screenshots('./screenshots', { takeOnFails: true })
.reporter('junit', stream)
.src(['./*tests/*.js'])
.run({
skipJsErrors: true,
skipUncaughtErrors: true,
quarantineMode: false,
selectorTimeout: 15000,
assertionTimeout: 15000,
pageLoadTimeout: 5000,
speed: 1,
debugOnFail: false,
stopOnFirstFail: false
})
Azure pipeline yaml
steps:
- script: |
node testRunner.js
Related
I am running my whole system using docker-compose and I am trying to execute my end-to-end tests (written with Nightwatch) using a selenium/chrome service that is in the same network. My frontend is VueJS and the image is built from a node base image. I am running the tests with docker-compose exec frontend npm run test-selenium and getting the following output:
> frontend#0.1.0 test-selenium
> nightwatch -c nightwatch-selenium.conf.js
[Login Test] Test Suite
──────────────────────────────────────────────
ℹ Connected to chrome on port 4444 (1494ms).
Using: chrome (102.0.5005.61) on LINUX.
Running login:
───────────────────────────────────────────────────────────────────────────────────────────────────
ℹ Loaded url http://frontend:8090/login in 2049ms
NoSuchElementError: An error occurred while running .click() command on <Element [name=#loginBt]>: Timed out while waiting for element "#loginBt" with "css selector" to be present for 5000 milliseconds.
at Object.login (/app/tests/loginTest.js:8:8)
NoSuchElementError: An error occurred while running .setValue() command on <Element [name=#emailField]>: Timed out while waiting for element "input[name = "email"]" with "css selector" to be present for 5000 milliseconds.
at Page.fillUpData (/app/tests/page-objects/loginPage.js:13:21)
at Object.login (/app/tests/loginTest.js:9:8)
NoSuchElementError: An error occurred while running .setValue() command on <Element [name=#passField]>: Timed out while waiting for element "input[name = "password"]"
with "css selector" to be present for 5000 milliseconds.
at Page.fillUpData (/app/tests/page-objects/loginPage.js:13:21)
at Object.login (/app/tests/loginTest.js:10:8)
NoSuchElementError: An error occurred while running .click() command on <Element [name=#submitBt]>: Timed out while waiting for element "button[name = "submit"]" with
"css selector" to be present for 5000 milliseconds.
at Object.login (/app/tests/loginTest.js:11:8)
✖ Testing if the URL contains '/cards' in 5000ms - expected "contains '/cards'" but got: "http://frontend:8090/login" (5143ms)
at Object.login (/app/tests/loginTest.js:12:15)
FAILED: 1 assertions failed and 4 errors (28.22s)
When running tests locally, everything is working just fine. For local tests, I am using the chrome driver installed as dev dependency, accessing the frontend using http://localhost:8090/.
It is my first time dealing with E2E tests in docker, so I am not really sure what is going wrong. Any help is appreciated. Please find the source code below. Let me know if I need to add any more information.
package.json
"scripts": {
...
"test": "nightwatch",
"test-selenium": "nightwatch -c nightwatch-selenium.conf.js",
...
},
docker-compose.yml
services:
frontend:
build:
context: card-market-frontend/
dockerfile: Dockerfile
command: npm run serve
working_dir: /app
ports:
- "8090:8090"
chrome:
image: selenium/standalone-chrome:latest
hostname: chrome
privileged: true
shm_size: 2g
When running the tests with the test-selenium command I am using the nightwatch.conf below:
const chromedriver = require("chromedriver");
require("dotenv").config();
module.exports = {
src_folders: ["/tests"],
page_objects_path: ["tests/page-objects"],
test_workers: false,
selenium: {
start_process: false,
cli_args: {
"webdriver.chrome.driver": chromedriver.path,
},
},
webdriver: {
start_process: false,
},
test_settings: {
default: {
selenium_port: 4444,
selenium_host: "chrome",
screenshots: {
enabled: true,
path: "tests_output/",
on_failure: true,
},
desiredCapabilities: {
browserName: "chrome",
chromeOptions: {
w3c: false,
args: ["--no-sandbox"],
},
},
},
},
};
My page object loginPage.js
module.exports = {
url: `http://frontend:8090/login`,
elements: {
logoutBt: "#logoutBt",
loginBt: "#loginBt",
emailField: 'input[name = "email"]',
passField: 'input[name = "password"]',
submitBt: 'button[name = "submit"]',
},
commands: [
{
fillUpData(selector, data) {
return this.setValue(selector, data);
},
},
],
};
The test that is running loginTest.js
module.exports = {
login(browser) {
const page = browser.page.loginPage();
const email = "coolEmail#gmail.com";
const pass = "coolPass";
page
.navigate()
.click("#loginBt")
.fillUpData("#emailField", email)
.fillUpData("#passField", pass)
.click("#submitBt")
.assert.urlContains("/cards");
},
};
Your first error is:
NoSuchElementError: An error occurred while running .click() command on <Element [name=#logoutBt]>: Timed out while waiting for element "#logoutBt" with "css selector" to be present for 5000 milliseconds.
which indicates that NoSuchElementError was raised while searching for the Logout button.
However as per the test steps within loginTest.js, ideally you are not supposed to find the Logout button just after navigating to the page:
.navigate()
.click("#logoutBt")
You can find the Logout button only after logging in.
Solution
Ideally, the sequence of the commands will be:
page
.navigate()
.click("#loginBt")
.fillUpData("#emailField", email)
.fillUpData("#passField", pass)
.click("#submitBt")
.assert.urlContains("/cards");
.click("#logoutBt")
I want to start working with CoreUi Admin template and create project containing unit and e2e testing.
In the begining, I install CoreUi like shown in documentation:
git clone https://github.com/coreui/coreui-free-vue-admin-template.git CoreUI-Vue
cd CoreUI-Vue
npm install
Everything is ok. Now I want to run whole unit and e2e tests:
npm run test:unit
npm run test:e2e
All unit tests run properly. But when I run e2e test I got the following errors:
> #coreui/coreui-free-vue-admin-template#3.1.4 test:e2e
> vue-cli-service test:e2e
INFO Starting development server...
[BABEL] Note: The code generator has deoptimised the styling of /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/#coreui/vue/dist/coreui-vue.common.js as it exceeds the max of 500KB.
WARNING Compiled with 2 warnings 2:47:38 PM
warning
asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
css/app.10fee0d6.css (383 KiB) js/chunk-28d6db65.e7636129.js (488 KiB)
js/chunk-2d0a400c.650c6f3a.js (3.37 MiB) js/chunk-2d0ae5e6.22010046.js (390 KiB)
js/chunk-2d237b34.468fc7b7.js (1.36 MiB)
js/chunk-vendors.03b8fe62.js (687 KiB)
warning
entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
app (1.07 MiB)
js/chunk-vendors.03b8fe62.js
css/app.10fee0d6.css
js/app.94002fbe.js
App running at:
- Local: http://localhost:8080/
- Network: http://172.31.33.121:8080/
App is served in production mode.
Note this is for preview or E2E testing only.
INFO Running end-to-end tests ...
[Test] Test Suite
=================
⚠ Error connecting to localhost on port 9515.
_________________________________________________
TEST FAILURE: 1 error during execution; 0 tests failed, 0 passed (348ms)
✖ test
An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally."
Error: An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally."
at endReadableNT (_stream_readable.js:1201:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Error: An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally." at endReadableNT (_stream_readable.js:1201:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
SKIPPED:
- CoreUI Vue e2e tests
ERROR Error: Command failed: /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/nightwatch/bin/nightwatch --config /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/#vue/cli-plugin-e2e-nightwatch/nightwatch.config.js --env chrome
Error: Command failed: /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/nightwatch/bin/nightwatch --config /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/#vue/cli-plugin-e2e-nightwatch/nightwatch.config.js --env chrome
at makeError (/home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/execa/index.js:174:9)
at /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/execa/index.js:278:16
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
bagsiur#DESKTOP-718MBR1:~/Projects/B2B/test-coreui/CoreUI-Vue$
bagsiur#DESKTOP-718MBR1:~/Projects/B2B/test-coreui/CoreUI-Vue$ npm run test:e2e
> #coreui/coreui-free-vue-admin-template#3.1.4 test:e2e
> vue-cli-service test:e2e
INFO Starting development server...
WARNING Compiled with 2 warnings 3:06:39 PM
warning
asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
css/app.10fee0d6.css (383 KiB)
js/chunk-28d6db65.e7636129.js (488 KiB)
js/chunk-2d0a400c.650c6f3a.js (3.37 MiB)
js/chunk-2d0ae5e6.22010046.js (390 KiB)
js/chunk-2d237b34.468fc7b7.js (1.36 MiB)
js/chunk-vendors.03b8fe62.js (687 KiB)
warning
entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
app (1.07 MiB)
js/chunk-vendors.03b8fe62.js
css/app.10fee0d6.css
js/app.94002fbe.js
App running at:
- Local: http://localhost:8080/
- Network: http://172.31.33.121:8080/
App is served in production mode.
Note this is for preview or E2E testing only.
INFO Running end-to-end tests ...
[Test] Test Suite
=================
⚠ Error connecting to localhost on port 9515.
_________________________________________________
TEST FAILURE: 1 error during execution; 0 tests failed, 0 passed (348ms)
✖ test
An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally."
Error: An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally."
at endReadableNT (_stream_readable.js:1201:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Error: An error occurred while retrieving a new session: "unknown error: Chrome failed to start: exited abnormally." at endReadableNT (_stream_readable.js:1201:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
SKIPPED:
- CoreUI Vue e2e tests
ERROR Error: Command failed: /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/nightwatch/bin/nightwatch --config /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/#vue/cli-plugin-e2e-nightwatch/nightwatch.config.js --env chrome
Error: Command failed: /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/nightwatch/bin/nightwatch --config /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/#vue/cli-plugin-e2e-nightwatch/nightwatch.config.js --env chrome
at makeError (/home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/execa/index.js:174:9)
at /home/bagsiur/Projects/B2B/test-coreui/CoreUI-Vue/node_modules/execa/index.js:278:16
at processTicksAndRejections (internal/process/task_queues.js:97:5)
I work on wsl2 and Debian. My node.js version is v13.14.0 and npm is 7.12.1. I installed chromedriver in version: ChromeDriver 2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706) and last stable google chrome: google-chrome-stable 90.0.4430.212-1
What do I miss? Where could be the cause of the error in running e2e tests?
#UPDATE: I put the configuration of Nightwatch.conf.json:
// http://nightwatchjs.org/gettingstarted#settings-file
const path = require('path')
const deepmerge = require('deepmerge')
const chromedriver = require('chromedriver')
// user may have not installed geckodriver
let geckodriver = {}
try {
geckodriver = require('geckodriver')
} catch (e) {}
const userOptions = JSON.parse(process.env.VUE_NIGHTWATCH_USER_OPTIONS || '{}')
const useSelenium = process.env.VUE_NIGHTWATCH_USE_SELENIUM === '1'
const startHeadless = process.env.VUE_NIGHTWATCH_HEADLESS === '1'
const concurrentMode = process.env.VUE_NIGHTWATCH_CONCURRENT === '1'
const chromeArgs = []
const geckoArgs = []
if (startHeadless) {
chromeArgs.push('headless')
geckoArgs.push('--headless')
}
const defaultSettings = {
src_folders: ['tests/e2e/specs'],
output_folder: 'tests/e2e/reports',
page_objects_path: 'tests/e2e/page-objects',
custom_assertions_path: 'tests/e2e/custom-assertions',
custom_commands_path: 'tests/e2e/custom-commands',
globals_path: path.resolve('tests/e2e/globals.js'),
test_workers: concurrentMode,
test_settings: {
default: {
detailed_output: !concurrentMode,
launch_url: '${VUE_DEV_SERVER_URL}'
},
chrome: {
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
w3c: false,
args: chromeArgs
}
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox',
alwaysMatch: {
acceptInsecureCerts: true,
'moz:firefoxOptions': {
args: geckoArgs
}
}
},
webdriver: useSelenium ? {} : {
server_path: geckodriver.path,
port: 4444
}
}
}
}
const baseSettings = deepmerge(defaultSettings, webdriverServerSettings())
module.exports = deepmerge(baseSettings, adaptUserSettings(userOptions))
function adaptUserSettings (settings) {
// The path to nightwatch external globals file needs to be made absolute
// if it is supplied in an additional config file, due to merging of config files
if (settings.globals_path) {
settings.globals_path = path.resolve(settings.globals_path)
}
return settings
}
function webdriverServerSettings () {
if (useSelenium) {
return {
selenium: {
start_process: true,
host: '127.0.0.1',
port: 4444,
server_path: require('selenium-server').path,
cli_args: {
'webdriver.chrome.driver': chromedriver.path,
'webdriver.gecko.driver': geckodriver.path
}
}
}
}
return {
webdriver: {
start_process: true,
port: 9515,
server_path: chromedriver.path
}
}
}
Version mismatch of chromedriver and chrome is the most common reason of e2e failing like this.
ChromeDriver 2.41 supports Chrome v67-69
your google-chrome-stable 90.0.4430 needs ChromeDriver 90.0.4430
I want to run protractor tests on a webserver integration platform without gui based on redhat linux. The tests will be executed whenever the webserver software will be deployed. I use firefox and geckodriver.
The call chain is
protractor calls
firefox on
Xvfb calls
(local) Webserver
and backwards to the caller.
I get Jasmine timeouts when starting Protractor with Xvfb.
My environment:
node: v12.14.1
npm: 6.14.2
Xvfb: ?, installed, callable
xvfb-run: ?, installed, callable
My start scripts:
Selenium:
java -jar -Dwebdriver.chrome.driver=/pathx/geckodriver-v0.26.0 /pathy/selenium-server-standalone-3.141.59.jar &
This is essentially what 'webdriver-manager start' does, but I cannot need the update ahead.
Xvfb:
xvfb-run --auto-servernum --server-args='-screen 0 1920x1200x24' -e xvfb.err firefox &
Protractor:
DISPLAY=:99 protractor --verbose --troubleshoot --logLevel=DEBUG protractor.conf.js
All I get is
[15:00:59] D/launcher - Running with --troubleshoot
[15:00:59] D/launcher - Protractor version: 5.4.3
[15:00:59] D/launcher - Your base url for tests is undefined
[15:00:59] I/launcher - Running 1 instances of WebDriver
[15:00:59] I/direct - Using FirefoxDriver directly...
[15:01:00] D/runner - WebDriver session successfully started with capabilities C apabilities {
map_: Map {
'acceptInsecureCerts' => false,
'browserName' => 'firefox',
'browserVersion' => '60.9.0',
'moz:accessibilityChecks' => false,
'moz:geckodriverVersion' => '0.26.0',
'moz:headless' => false,
'moz:processID' => 32246,
'moz:profile' => '/tmp/rust_mozprofileAz14ww',
'moz:useNonSpecCompliantPointerOrigin' => false,
'moz:webdriverClick' => true,
'pageLoadStrategy' => 'normal',
'platformName' => 'linux',
'platformVersion' => '3.10.0-957.el7.x86_64',
'rotatable' => false,
'timeouts' => { implicit: 0, pageLoad: 300000, script: 30000 }
}
}
[15:01:00] D/runner - Running with spec files /home/xgadvls/az-uss/frontend/e2e/ src/specs/login/login-spec.ts,/home/xgadvls/az-uss/frontend/e2e/src/specs/logout /logout-spec.ts
Started
undefined
F(node:32231) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
FA Jasmine spec timed out. Resetting the WebDriver Control Flow.
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
FA Jasmine spec timed out. Resetting the WebDriver Control Flow.
Failures:
1) Anmeldung Anmeldedate werden eingegeben
Message:
Failed: WebDriverError
Stack:
Error: Failed: WebDriverError
at /home/xgadvls/az-uss/node_modules/jasminewd2/index.js:64:48
...
How can I get a webdriver error when I haven't got it in my call chain?! What am I doing wrong? Can anybody help?
The problem was caused by an invalid certificate.
My way to the solution may be interesting.
I renamed the geckodriver-v0.26.0 (or so) to geckodriver.bin.
I made a script with the original geckodriver name and called in that script the bin version with -vv argument (very verbose) and redirected the output into a file (>myfile). It is important to pass all parameters ($#).
The next test informed me in the log file about the invalid certificate. This can be fixed by a parameter AcceptInsecureCerts for firefox. (I guess chrome as well)
Here we go.
I hope this helps anybody.
I'm setting up CI for our project that is in Angular. We are using Google Cloud Platform's Cloud Builder for CI. Everything is going smoothly, we can setup the environment, install packages via npm, build the project using angular-cli and deploy. But the problem is that our test cases (e2e) in Protractor keeps getting an error. What are we missing here?
This is for the Continuous Integration using Google Cloud Builders, we are running the protractor in headless mode, and using the puppeteer's chromium as the chrome's binary path
This is my protractor.conf.js
// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts
const { SpecReporter } = require('jasmine-spec-reporter');
const puppeteer = require('puppeteer');
process.env.CHROME_BIN = puppeteer.executablePath()
exports.config = {
allScriptsTimeout: 1000000,
specs: [
'./e2e/*.e2e-spec.ts',
'../e2e/*.e2e-spec.ts'
],
capabilities: {
// browserName: 'chrome',
browserName: 'chrome',
chromeOptions: {
binary: process.env.CHROME_BIN,
args: ['--headless', '--disable-gpu', '--no-sandbox', '--disable-extensions', '--disable-dev-shm-usage', '--disable-setuid-sandbox']
}
},
// seleniumAddress: 'http://127.0.0.1:4444/wd/hub',
// ignoreUncaughtExceptions: true,
directConnect: true,
},
};
This is the error we are getting in cloud builders
[11:24:12] E/launcher - unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /workspace/node_modules/puppeteer/.local-chromium/linux-662092/chrome-linux/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.44.609551 (5d576e9a44fe4c5b6a07e568f1ebc753f1214634),platform=Linux 4.15.0-1033-gcp x86_64)
[11:24:12] E/launcher - WebDriverError: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /workspace/node_modules/puppeteer/.local-chromium/linux-662092/chrome-linux/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
I am trying to use GitLab CI to run some client-side unit test written using QUnit. Now to run the Qunit test I am using the grunt-contrib-qunit plugin. To run these tests in headless mode I am using this plugin which hosts it on a localhost server in a console and runs all unit tests. When running this project locally I am successfully able to run all the unit tests but when I checking in my code which kicks of the CI process, on GitLab, it fails on starting the phantomjs server and gives timeout error. I am also providing the jsbin link of the two text files which are basically the output of the unit test from my console. One file is of my local system and another is from the GitLab CI that runs on GitLab website when I check-in my code.
Local Console Output File Dump
Gitlab CI Output Dump
Adding my gitlab-ci.yaml file
image: node:4.2.2
before_script:
- dir
- cd sapui5_repo
- dir
- cd app-with-tests
build:
stage: build
script:
- npm i
- npm run test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
Also adding my gruntfile if that helps
/* global module */
module.exports = function (grunt) {
grunt.initConfig({
qunit: {
all: {
options: {
timeout: 9000,
urls: [
"http://localhost:9000/webcontent/test/unit/unitTests.qunit.html"
]
}
},
//all: ["webcontent/test/unit/unitTests.qunit.html"],
options: {
timeout: 2000,
}
},
connect: {
options: {
//open: true,
},
first: {
options: {
port: 9000,
//livereload: 3500,
base: "./"
}
},
second: {
options: {
open: {
target: "http://localhost:9000/webcontent"
},
keepalive: true,
port: 9000,
livereload: 3501,
base: "./",
}
}
},
});
grunt.loadNpmTasks("grunt-contrib-connect");
grunt.loadNpmTasks("grunt-contrib-qunit");
grunt.registerTask("test", [
"connect:first", "qunit"
]);
grunt.registerTask("default", [
"connect:second"
]);
};