"ChromeHeadless have not captured in 60000 ms, killing." occuring only in Gitlab hosted CI/CD pipeline - selenium

When running a CI/CD pipeline on Gitlab, my Karma tests are timing out with the error:
ℹ 「wdm」: Compiled successfully.
05 08 2019 22:25:31.483:INFO [karma-server]: Karma v4.2.0 server started at http://0.0.0.0:9222/
05 08 2019 22:25:31.485:INFO [launcher]: Launching browsers ChromeHeadlessNoSandbox with concurrency 1
05 08 2019 22:25:31.488:INFO [launcher]: Starting browser ChromeHeadless
05 08 2019 22:26:31.506:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
05 08 2019 22:26:31.529:INFO [launcher]: Trying to start ChromeHeadless again (1/2).
05 08 2019 22:27:31.580:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
05 08 2019 22:27:31.600:INFO [launcher]: Trying to start ChromeHeadless again (2/2).
05 08 2019 22:28:31.659:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
05 08 2019 22:28:31.689:ERROR [launcher]: ChromeHeadless failed 2 times (timeout). Giving up.
npm ERR! Test failed. See above for more details.
This problem does not occur when running tests locally, and it does not occur when running the tests using the same Docker image with Gitlab Runner locally.
I feel like I have tried every possible configuration with karma.conf.js. I have Googled this issue relentlessly and have tried every suggestion from proxy servers, to environment variables, to flags... but alas, no luck. I have tried multiple Docker images as this was initially failing on local Gitlab Runner but I have found that the Docker image selenium/standalone-chrome:latest works fine in local Gitlab Runner.
Here is my karma.conf.js file:
const process = require('process');
process.env.CHROME_BIN = require('puppeteer').executablePath();
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
frameworks: [ 'jasmine' ],
// list of files / patterns to load in the browser
files: [
'src/**/*.spec.js'
],
// list of files / patterns to exclude
exclude: [],
// preprocess matching files before serving them to the browser
preprocessors: {
'src/**/*.spec.js': [ 'webpack' ]
},
webpack: {
// webpack configuration
mode: 'development',
module: {
rules: [
{
test: /\.js$/,
loader: 'babel-loader',
exclude: /node_modules/,
query: {
presets: ['env']
}
}
]
},
stats: {
colors: true
}
},
// test results reporter to use
reporters: [ 'spec' ],
// web server port
port: 9222,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// plugins for karma
plugins: [
'karma-chrome-launcher',
'karma-webpack',
'karma-jasmine',
'karma-spec-reporter'
],
// start these browsers
browsers: ['ChromeHeadlessNoSandbox'],
customLaunchers: {
ChromeHeadlessNoSandbox: {
base: 'ChromeHeadless',
flags: [
'--headless',
'--no-sandbox',
'--disable-gpu'
]
}
},
captureTimeout: 60000,
browserDisconnectTolerance: 5,
browserDisconnectTimeout : 30000,
browserNoActivityTimeout : 30000,
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true,
// Concurrency level
// how many browser should be started simultaneous
concurrency: 1
})
}
And here is my .gitlab-ci.yml file:
.prereq_scripts: &prereq_scripts |
sudo apt -y update && sudo curl -sL https://deb.nodesource.com/setup_10.x | sudo bash && sudo apt -y install nodejs
image: 'selenium/standalone-chrome:latest'
stages:
- test
test:
stage: test
script:
- *prereq_scripts
- npm install
- npm test
I am expecting the tests to run successfully in all three instances (local npm, local Gitlab Runner and remote Gitlab CI/CD pipeline). Currently it only runs in successfully in the first two.

In your karma.conf.js file you need to declare the CHROME_BIN variable inside the module.exports function:
module.exports = function(config) {
const process = require('process');
process.env.CHROME_BIN = require('puppeteer').executablePath();
config.set({
...

Currently, Puppeteer has an issue with Karma on Linux machines, see GitHub issue
There are plenty of solutions on how to make it works without Puppeteer if you use it just to install Headless Chromium.
I have installed it on my Jenkins Alpine machine using only two bash lines:
apk add chromium
export CHROME_BIN=/usr/bin/chromium-browser
Alternatively, you can use Docker with the same setup. One of the examples is here

Docker image with chromeheadless
for example, use a docker image of angular/ngcontainer with chrome headless for testing UI apps.
image: 'angular/ngcontainer:latest'
Also, I created one docker image with the latest chrome
image: 'anulals/angular'
https://hub.docker.com/r/angular/ngcontainer
https://hub.docker.com/r/anulals/angular

Related

Automated Testing with Project Piper

We are currently in the process of setting up a CI/CD pipeline for our SCP CF environment based on Project Piper. After a lot of trial & error, we have made significant process, however we are still not able to run our karma tests.
We understood that we need to use the karma-webdriver-launcher, however we are not able to start/connect to Chrome in the sidecar container.
Relevant logs:
[Pipeline] sh
+ cd ./nodejs
+ npm test
> nodejs#1.0.0 test /var/jenkins_home/workspace/CICD_Multibranch_master/nodejs
> karma start ./karma.conf.js
10 07 2019 14:42:53.897:DEBUG [config]: Loading config /var/jenkins_home/workspace/CICD_Multibranch_master/nodejs/karma.conf.js
10 07 2019 14:42:53.906:DEBUG [karma-server]: Final config [{"LOG_DISABLE":"1","LOG_ERROR":"2","LOG_WARN":"3","LOG_INFO":"4","LOG_DEBUG":"5","frameworks":"6","protocol":"7","port":9876,"listenAddress":"8","hostname":"9","httpsServerConfig":"10","basePath":"11","files":"12","browserConsoleLogOptions":"13","customContextFile":null,"customDebugFile":null,"customClientContextFile":null,"exclude":"14","logLevel":"5","colors":false,"autoWatch":false,"autoWatchBatchDelay":250,"restartOnFileChange":false,"usePolling":true,"reporters":"15","singleRun":true,"browsers":"16","captureTimeout":60000,"proxies":"17","proxyValidateSSL":true,"preprocessors":"18","urlRoot":"19","reportSlowerThan":0,"loggers":"20","transports":"21","forceJSONP":false,"plugins":"22","client":"23","defaultClient":"23","browserDisconnectTimeout":2000,"browserDisconnectTolerance":0,"browserNoActivityTimeout":30000,"processKillTimeout":2000,"concurrency":null,"failOnEmptyTestSuite":true,"retryLimit":2,"detached":false,"crossOriginAttribute":true,"browserSocketTimeout":20000,"cmd":"24","configFile":"25","customLaunchers":"26","junitReporter":"27"},"OFF","ERROR","WARN","INFO","DEBUG",["28"],"http:","0.0.0.0","localhost",{},"/var/jenkins_home/workspace/CICD_Multibranch_master/nodejs",["29","30"],{"level":"31","format":"32","terminal":true},["25"],["33"],["34"],{},{},"/",["35"],["36","37"],["38","39","40","41","42"],{"args":"43","useIframe":true,"runInParent":false,"captureConsole":true,"clearContext":true},"start","/var/jenkins_home/workspace/CICD_Multibranch_master/nodejs/karma.conf.js",{"chromeSel":"44"},{"outputFile":"45","suite":"46"},"qunit",{"pattern":"47","served":true,"included":true,"watched":true,"nocache":false,"weight":"48"},{"pattern":"49","served":true,"included":true,"watched":true,"nocache":false,"weight":"50"},"debug","%b %T: %m","junit","chromeSel",{"type":"51","layout":"52"},"polling","websocket","karma-qunit","karma-junit-reporter","karma-chrome-launcher","karma-webdriver-launcher",{"launcher:chromeSel":"53"},[],{"base":"54","config":"55","browserName":"56"},"test-results-karma.xml","","/var/jenkins_home/workspace/CICD_Multibranch_master/nodejs/server.js",[1,0,0,0,0,0],"/var/jenkins_home/workspace/CICD_Multibranch_master/nodejs/tests/sampleTest.spec.js",[1,0,0,0,0,0],"console",{"type":"57","pattern":"58"},["59",null],"WebDriver",{"hostname":"60","port":4444},"chrome","pattern","%d{DATE}:%p [%c]: %m","factory","selenium"]
10 07 2019 14:42:53.907:DEBUG [plugin]: Loading plugin karma-qunit.
10 07 2019 14:42:53.908:DEBUG [plugin]: Loading plugin karma-junit-reporter.
10 07 2019 14:42:53.923:DEBUG [plugin]: Loading plugin karma-chrome-launcher.
10 07 2019 14:42:53.931:DEBUG [plugin]: Loading plugin karma-webdriver-launcher.
10 07 2019 14:42:54.179:DEBUG [plugin]: Loading inlined plugin (defining launcher:chromeSel).
10 07 2019 14:42:54.193:DEBUG [web-server]: Instantiating middleware
10 07 2019 14:42:54.194:DEBUG [reporter]: Trying to load reporter: junit
10 07 2019 14:42:54.195:DEBUG [reporter]: Trying to load color-version of reporter: junit (junit_color)
10 07 2019 14:42:54.195:DEBUG [reporter]: Couldn't load color-version.
10 07 2019 14:42:54.224:INFO [karma-server]: Karma v4.1.0 server started at http://0.0.0.0:9876/
10 07 2019 14:42:54.225:INFO [launcher]: Launching browsers chromeSel with concurrency unlimited
10 07 2019 14:42:54.231:INFO [launcher]: Starting browser chrome via Remote WebDriver
10 07 2019 14:42:54.232:DEBUG [launcher]: null -> BEING_CAPTURED
10 07 2019 14:42:54.232:DEBUG [temp-dir]: Creating temp dir at /tmp/karma-89562642
10 07 2019 14:42:54.235:DEBUG [WebDriver]: WebDriver config: {"hostname":"selenium","port":4444}
10 07 2019 14:42:54.235:DEBUG [WebDriver]: Browser capabilities: {"platform":"ANY","testName":"Karma test","tags":[],"version":"","base":"WebDriver","browserName":"chrome"}
10 07 2019 14:43:54.281:WARN [launcher]: chrome via Remote WebDriver have not captured in 60000 ms, killing.
10 07 2019 14:43:54.286:DEBUG [launcher]: BEING_CAPTURED -> BEING_KILLED
10 07 2019 14:43:54.355:INFO [WebDriver]: Killed Karma test.
10 07 2019 14:43:54.355:DEBUG [launcher]: Process chrome via Remote WebDriver exited with code -1 and signal timeout
10 07 2019 14:43:54.356:DEBUG [temp-dir]: Cleaning temp dir /tmp/karma-89562642
10 07 2019 14:43:54.358:INFO [launcher]: Trying to start chrome via Remote WebDriver again (1/2).
karma.conf.js:
// Karma configuration
module.exports = function(config) {
var webdriverConfig = {
hostname: 'selenium',
port: 4444
}
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['qunit'],
plugins: ['karma-qunit','karma-junit-reporter','karma-chrome-launcher', 'karma-webdriver-launcher'],
// list of files / patterns to load in the browser
files: [
'server.js',
'tests/sampleTest.spec.js'
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['junit'],
// web server port
//port: 9876,
//hostname: localhost,
// enable / disable colors in the output (reporters and logs)
colors: false,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_DEBUG,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
customLaunchers: {
'chromeSel': {
base: 'WebDriver',
config: webdriverConfig,
browserName: 'chrome'
}
},
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['chromeSel'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true,
junitReporter: {
outputFile: 'test-results-karma.xml',
suite: ''
}
});
};
Jenkinsfile:
#!groovy
#Library('piper-lib-os') _
node(){
stage('Prepare') {
deleteDir()
checkout scm
setupCommonPipelineEnvironment
}
stage('Build') {
karmaExecuteTests script: this, modules: ['./nodejs'],
installCommand: "npm install karma karma-qunit karma-junit-reporter karma-chrome-launcher qunit karma-webdriver-launcher",
runCommand: 'npm test'
mtaBuild script:this, buildTarget: 'CF', applicationName: 'appLibrary'
}
stage('Deploy to QA') {
testsPublishResults script: this, junit: [updateResults: true, archive: true]
cloudFoundryDeploy(
script: this,
cloudFoundry: [apiEndpoint: 'https://api.cf.eu10.hana.ondemand.com', manifest: 'manifest.yml', org:'xxx', space:'xxx', credentialsId: 'xxx'],
deployTool: 'mtaDeployPlugin'
)
}
}
Thanks a lot,
Nico
To run the karmaExecuteTests step on your Jenkins, you need a Docker deamon installed. I assume this is the case.
In the Docker sidecar pattern the containers can talk to each other using their container names (karma & selenium). So for the browser (config.customLaunchers.chromeSel.config.hostname) you set the hostname correctly to selenium but you also need to set config.hostname to karma.

Karma not running tests. Executed 0 of 0 ERROR

I am trying to test in karma, yet every time I run Karma I get the same error:
$ karma start karma.conf.js --single-run
31 07 2018 12:56:48.134:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
31 07 2018 12:56:48.138:INFO [launcher]: Launching browser Chrome with unlimited concurrency
31 07 2018 12:56:48.144:INFO [launcher]: Starting browser Chrome
31 07 2018 12:56:49.890:INFO [Chrome 67.0.3396 (Windows 10.0.0)]: Connected on socket 2we1ypnXUlTXv5IRAAAA with id 14298953
Chrome 67.0.3396 (Windows 10.0.0): Executed 0 of 0 ERROR (0.006 secs / 0 secs)
my (mostly) auto-generated karma.conf.js
located one level above the tests at: C:\Users\king\Desktop\_REPOS\inventivangular\src
// Karma configuration
// Generated on Tue Jul 31 2018 12:30:22 GMT-0400 (Eastern Daylight Time)
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: 'C:/Users/king/Desktop/_REPOS/inventivangular/src/',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['jasmine'],
// list of files / patterns to load in the browser
files: [
'./app/*.component.ts',
'./app/*spec.ts'
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['Chrome'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false,
// Concurrency level
// how many browser should be started simultaneous
concurrency: Infinity
})
}
File path of files to test: C:\Users\king\Desktop\_REPOS\inventivangular\src\app
Wound up just running ng test from C:\Users\king\Desktop\_REPOS\inventivangular which generated its own karma.conf.js and everything worked.
If you are using auto-generated tests you may need to:
import { HttpClientTestingModule } from '#angular/common/http/testing'
and
imports: [HttpClientTestingModule]

PhantomJS timeout issue when running in headless mode in GitLab CI

I am trying to use GitLab CI to run some client-side unit test written using QUnit. Now to run the Qunit test I am using the grunt-contrib-qunit plugin. To run these tests in headless mode I am using this plugin which hosts it on a localhost server in a console and runs all unit tests. When running this project locally I am successfully able to run all the unit tests but when I checking in my code which kicks of the CI process, on GitLab, it fails on starting the phantomjs server and gives timeout error. I am also providing the jsbin link of the two text files which are basically the output of the unit test from my console. One file is of my local system and another is from the GitLab CI that runs on GitLab website when I check-in my code.
Local Console Output File Dump
Gitlab CI Output Dump
Adding my gitlab-ci.yaml file
image: node:4.2.2
before_script:
- dir
- cd sapui5_repo
- dir
- cd app-with-tests
build:
stage: build
script:
- npm i
- npm run test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
Also adding my gruntfile if that helps
/* global module */
module.exports = function (grunt) {
grunt.initConfig({
qunit: {
all: {
options: {
timeout: 9000,
urls: [
"http://localhost:9000/webcontent/test/unit/unitTests.qunit.html"
]
}
},
//all: ["webcontent/test/unit/unitTests.qunit.html"],
options: {
timeout: 2000,
}
},
connect: {
options: {
//open: true,
},
first: {
options: {
port: 9000,
//livereload: 3500,
base: "./"
}
},
second: {
options: {
open: {
target: "http://localhost:9000/webcontent"
},
keepalive: true,
port: 9000,
livereload: 3501,
base: "./",
}
}
},
});
grunt.loadNpmTasks("grunt-contrib-connect");
grunt.loadNpmTasks("grunt-contrib-qunit");
grunt.registerTask("test", [
"connect:first", "qunit"
]);
grunt.registerTask("default", [
"connect:second"
]);
};

theintern calling /__intern/client.htm in URL

I want to run intern tests locally with selenium-standalone both of which i installed through npm.
when i go to run the tests -> "./node_modules/.bin/intern-runner" config=./pmictests/test/bit/GAT/internEx/intern
the browser starts but the url goes to http://localhost:8585/__intern/client.html?config=.%2Fpmictests%2Ftest%2Fbit%2FGAT%2FinternEx%2Fintern&basePath
As in the _intern/client.html? is not what i want
why is this happening? i'm trying to get my head around it but been stuck on this problem for a while.
my config file looks like this:
define({
proxyPort: 9515,
proxyUrl: 'http://localhost:8585/',
tunnel: 'NullTunnel',
useSauceConnect: false,
capabilities: {
'fixSessionCapabilities' : false,
'selenium-version': '2.35.0',
'idle-timeout': 36
},
environments: [
{ browserName: 'chrome' }
],
maxConcurrency: 3,
useSauceConnect: false,
webdriver: {
host: 'localhost',
port: 4444
},
suites: [ './tests/test/' ],
excludeInstrumentation: /^(?:tests|node_modules)\//
});
That URL is for running unit tests. When you run intern-runner, it automatically loads client.html to run any unit test suites listed in suites. Once the unit tests are finished, Intern runs any functional tests listed in functionalSuites (which will load their own URLs).

Protractor pointing to Sauce Labs Selenium Server

I'm trying to integrate Protractor with Sauce Labs from Travis. I can get the sauce_connect server running correctly but am unable to get Travis to point to that particular remote server.
Travis will get to the point where it initiates sauce_connect but when I run "protractor:analytics" it doesn't point to the correct server and fails.
Travis.yml:
language: python
python:
- 3.2_with_system_site_packages
branches:
only:
- develop
before_install:
- sudo apt-get update -qq
- sudo apt-get install python-numpy
install:
- cd lib && python setup.py install
- cd .. && pip install -r requirements/travis_requirements.txt
- npm install
script:
- grunt karma:single
- grunt protractor:analytics
env:
global:
- secure: <string>
- secure: <string>
sauce_connect: true
Gruntfile:
protractor: {
options: {
configFile: './webapp/static/test/e2e/protractor.conf.js',
keepAlive: true
},
singlerun: {},
analytics: {
options: {
//debug : true,
args:{
specs: ['./webapp/static/test/e2e/analytics_spec.js']
}
}
},
},
Protractor Conf:
exports.config = {
chromeOnly: false,
seleniumArgs: [],
// If sauceUser and sauceKey are specified, seleniumServerJar will be ignored.
// The tests will be run remotely using SauceLabs.
sauceUser: process.env.SAUCE_USER,
sauceKey: process.env.SAUCE_KEY,
baseUrl: 'http://localhost:8000',
specs: [
'./*_spec.js',
],
// Patterns to exclude.
exclude: [],
multiCapabilities: [],
// ----- More information for your tests ----
//
// A base URL for your application under test. Calls to protractor.get()
// with relative paths will be prepended with this.
baseUrl: process.env.SN_BASE_URL,
// Selector for the element housing the angular app - this defaults to
// body, but is necessary if ng-app is on a descendant of <body>
rootElement: 'body',
// A callback function called once protractor is ready and available, and
// before the specs are executed
// You can specify a file containing code to run by setting onPrepare to
// the filename string.
onPrepare: function() {
// At this point, global 'protractor' object will be set up, and jasmine
// will be available. For example, you can add a Jasmine reporter with:
// jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(
// 'outputdir/', true, true));
},
// The params object will be passed directly to the protractor instance,
// and can be accessed from your test. It is an arbitrary object and can
// contain anything you may need in your test.
// This can be changed via the command line as:
// --params.login.user 'Joe'
params: {
login: {
user: process.env.SN_TEST_USERNAME,
password: process.env.SN_TEST_PASSWORD
}
},
framework: 'jasmine',
// ----- Options to be passed to minijasminenode -----
//
// See the full list at https://github.com/juliemr/minijasminenode
jasmineNodeOpts: {
// onComplete will be called just before the driver quits.
onComplete: null,
// If true, display spec names.
isVerbose: false,
// If true, print colors to the terminal.
showColors: true,
// If true, include stack traces in failures.
includeStackTrace: true,
// Default time to wait in ms before a test fails.
defaultTimeoutInterval: 30000
},
onCleanUp: function() {}
};
If I did understand well : Sauce connect tool is not used by protractor/selenium when running a test suite.
Well I had this problem, travis requires sauce credentials and protractor requires those credentials and a tunnel id:
.travis.yml:
addons:
sauce_connect:
username: xxx
access_key: xxx
protractor.conf.js:
exports.config = {
...
sauceUser: process.env.SAUCE_USERNAME,
sauceKey: process.env.SAUCE_ACCESS_KEY,
capabilities: {
...
'tunnel-identifier': process.env.TRAVIS_JOB_NUMBER,
}
}