I'm trying to integrate Protractor with Sauce Labs from Travis. I can get the sauce_connect server running correctly but am unable to get Travis to point to that particular remote server.
Travis will get to the point where it initiates sauce_connect but when I run "protractor:analytics" it doesn't point to the correct server and fails.
Travis.yml:
language: python
python:
- 3.2_with_system_site_packages
branches:
only:
- develop
before_install:
- sudo apt-get update -qq
- sudo apt-get install python-numpy
install:
- cd lib && python setup.py install
- cd .. && pip install -r requirements/travis_requirements.txt
- npm install
script:
- grunt karma:single
- grunt protractor:analytics
env:
global:
- secure: <string>
- secure: <string>
sauce_connect: true
Gruntfile:
protractor: {
options: {
configFile: './webapp/static/test/e2e/protractor.conf.js',
keepAlive: true
},
singlerun: {},
analytics: {
options: {
//debug : true,
args:{
specs: ['./webapp/static/test/e2e/analytics_spec.js']
}
}
},
},
Protractor Conf:
exports.config = {
chromeOnly: false,
seleniumArgs: [],
// If sauceUser and sauceKey are specified, seleniumServerJar will be ignored.
// The tests will be run remotely using SauceLabs.
sauceUser: process.env.SAUCE_USER,
sauceKey: process.env.SAUCE_KEY,
baseUrl: 'http://localhost:8000',
specs: [
'./*_spec.js',
],
// Patterns to exclude.
exclude: [],
multiCapabilities: [],
// ----- More information for your tests ----
//
// A base URL for your application under test. Calls to protractor.get()
// with relative paths will be prepended with this.
baseUrl: process.env.SN_BASE_URL,
// Selector for the element housing the angular app - this defaults to
// body, but is necessary if ng-app is on a descendant of <body>
rootElement: 'body',
// A callback function called once protractor is ready and available, and
// before the specs are executed
// You can specify a file containing code to run by setting onPrepare to
// the filename string.
onPrepare: function() {
// At this point, global 'protractor' object will be set up, and jasmine
// will be available. For example, you can add a Jasmine reporter with:
// jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(
// 'outputdir/', true, true));
},
// The params object will be passed directly to the protractor instance,
// and can be accessed from your test. It is an arbitrary object and can
// contain anything you may need in your test.
// This can be changed via the command line as:
// --params.login.user 'Joe'
params: {
login: {
user: process.env.SN_TEST_USERNAME,
password: process.env.SN_TEST_PASSWORD
}
},
framework: 'jasmine',
// ----- Options to be passed to minijasminenode -----
//
// See the full list at https://github.com/juliemr/minijasminenode
jasmineNodeOpts: {
// onComplete will be called just before the driver quits.
onComplete: null,
// If true, display spec names.
isVerbose: false,
// If true, print colors to the terminal.
showColors: true,
// If true, include stack traces in failures.
includeStackTrace: true,
// Default time to wait in ms before a test fails.
defaultTimeoutInterval: 30000
},
onCleanUp: function() {}
};
If I did understand well : Sauce connect tool is not used by protractor/selenium when running a test suite.
Well I had this problem, travis requires sauce credentials and protractor requires those credentials and a tunnel id:
.travis.yml:
addons:
sauce_connect:
username: xxx
access_key: xxx
protractor.conf.js:
exports.config = {
...
sauceUser: process.env.SAUCE_USERNAME,
sauceKey: process.env.SAUCE_ACCESS_KEY,
capabilities: {
...
'tunnel-identifier': process.env.TRAVIS_JOB_NUMBER,
}
}
Related
I created a new Vue app using npm init vue#latest and selected Playwright for e2e tests. I removed firefox and webkit from projects in the playwright.config.ts file, so it will only use chromium.
Running npm run test:e2e works fine, the process exists with a success code.
When forcing the tests to fail by modifying the ./e2e/vue.spec.ts file the output is
but the process does not exit with an error code, it still opened browser windows and so CI environments would freeze.
I searched the docs for a specific flag e.g. "headless" and tried --max-failures -x but that didn't help.
How can I tell Playwright to run in headless mode and exit with an error code when something failed?
Since playwright.config.ts already makes use of process.env.CI I thought about replacing reporter: "html", with reporter: [["html", { open: !process.env.CI ? "on-failure" : "never" }]],
but which arguments should I add to the script "test:e2e:ci": "playwright test", to ensure process.env.CI is set?
Update
I tried to run the script inside my CI environment and it seems to work out of the box ( I don't know how it sets the CI environment flag but the pipeline did not freeze )
- name: Install Playwright Browsers
run: npx playwright install --with-deps
- name: Check if e2e tests are passing
run: npm run test:e2e
If any test fails it exists with an error code
It's serving the html report and asking to press 'Ctr+C' to quite.You can disable it using below configuration.
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
};
export default config;
Refer - Report Doc
Issue - https://github.com/microsoft/playwright/issues/9702
To add to the answer above, you can set headless: true in the 'use' block of the config which is above the projects block. Anything set at that level will apply to all projects unless you specifically override the setting inside a project specific area:
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
reporter: [ ['html', { open: 'never' }] ],
use: {
headless: true,
},
projects: [
{
name: 'chromium',
use: {
browserName: 'chromium',
},
},
},
};
export default config;
I am trying to use GitLab CI to run some client-side unit test written using QUnit. Now to run the Qunit test I am using the grunt-contrib-qunit plugin. To run these tests in headless mode I am using this plugin which hosts it on a localhost server in a console and runs all unit tests. When running this project locally I am successfully able to run all the unit tests but when I checking in my code which kicks of the CI process, on GitLab, it fails on starting the phantomjs server and gives timeout error. I am also providing the jsbin link of the two text files which are basically the output of the unit test from my console. One file is of my local system and another is from the GitLab CI that runs on GitLab website when I check-in my code.
Local Console Output File Dump
Gitlab CI Output Dump
Adding my gitlab-ci.yaml file
image: node:4.2.2
before_script:
- dir
- cd sapui5_repo
- dir
- cd app-with-tests
build:
stage: build
script:
- npm i
- npm run test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
Also adding my gruntfile if that helps
/* global module */
module.exports = function (grunt) {
grunt.initConfig({
qunit: {
all: {
options: {
timeout: 9000,
urls: [
"http://localhost:9000/webcontent/test/unit/unitTests.qunit.html"
]
}
},
//all: ["webcontent/test/unit/unitTests.qunit.html"],
options: {
timeout: 2000,
}
},
connect: {
options: {
//open: true,
},
first: {
options: {
port: 9000,
//livereload: 3500,
base: "./"
}
},
second: {
options: {
open: {
target: "http://localhost:9000/webcontent"
},
keepalive: true,
port: 9000,
livereload: 3501,
base: "./",
}
}
},
});
grunt.loadNpmTasks("grunt-contrib-connect");
grunt.loadNpmTasks("grunt-contrib-qunit");
grunt.registerTask("test", [
"connect:first", "qunit"
]);
grunt.registerTask("default", [
"connect:second"
]);
};
How I should configure my TFS build to make it possible run protractor e2e test in browserstack, and return me some html report which test are failed? I am new in TFS. I can do it manually from my machine, but not sure have I can do it in TFS.
This is how my protractor config looks like:
var project = 'testProject',
build = 'build_4',
acceptSslCerts = 'true';
var HtmlScreenshotReporter = require('protractor-jasmine2-screenshot-reporter');
var reporter = new HtmlScreenshotReporter({
dest: './html-report/',
filename: 'my-report.html',
reportOnlyFailedSpecs: false,
captureOnlyFailedSpecs: true,
showSummary: true,
});
module.exports.config = {
framework: 'jasmine2',
seleniumAddress: 'http://hub.browserstack.com/wd/hub', /* 'http://localhost:4444/wd/hub', */
allScriptsTimeout: 40000,
specs: [ 'test-spec.js' ],
capabilities: {
browserName: 'chrome',
loggingPrefs: { driver: 'ALL', server: 'ALL', browser: 'ALL' },
'build' : 'version3',
'project' : 'newintropage',
'browserstack.user': 'browserstack.user',
'browserstack.key': 'browserstack.key',
'browser': 'Edge',
'browser_version': '13.0',
'os': 'Windows',
'os_version': '10',
'resolution': '1024x768',
'acceptSslCerts': acceptSslCerts
},
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 40000
},
// Setup the report before any tests start
beforeLaunch: function () {
return new Promise(function (resolve) {
reporter.beforeLaunch(resolve);
});
},
onPrepare: function () {
jasmine.getEnv().addReporter(reporter);
},
// Close the report after all tests finish
afterLaunch: function (exitCode) {
return new Promise(function (resolve) {
reporter.afterLaunch(resolve.bind(this, exitCode));
});
}
};
And that's how my tfs build looks like:
According to the screenshot, you are using vNext build, and you have chosen a default "Visual Studio" build template.
TFS vNext build system is task based, which is flexible. I'm not familiar with protractor e2e tests, but based on the description of Protractor, at least, you'll need to use npm to install two command line tools, protractor and webdriver-manager, so the default "Visual Studio" build template won't meet your requirement.
You need to customize your own build template by specifying your build steps. For example, you need to add npm step to install protractor and webdriver-manager, and add Command Line step to run protractor conf.js command.
I'm trying to use Karma + Mocha for the first time.
The main condition is to run the unit tests in a Docker container without any GUI. I don't have any browser, and my only choice is to use PhantomJS.
I have configured all necessary plugins, configs etc., but I received the following error:
ReferenceError: Can't find variable: exports
Before the test run, I built the JavaScript source code with Babel to the 'build' directory and then started Karma from there.
What could be causing this problem?
Here is my karma.conf.js:
// Karma configuration
// Generated on Mon Apr 25 2016 13:04:28 GMT+0300 (RTZ 2 (зима))
const path = require('path');
const webpack = require('webpack');
module.exports = function(config) {
var confBuilder = require('./webpack.config');
confBuilder.module.loaders.push({
test: /\.jsx$/,
loader: 'isparta',
include: path.resolve('src'),
});
confBuilder.devtool = 'inline-source-map';
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['mocha', 'chai'/*'jasmine' /*, 'requirejs'*/],
// list of files / patterns to load in the browser
files: [
{pattern: 'build/utils/*.js', load: false}
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'tests.webpack.js': ['webpack', 'sourcemap'],
'build/utils/*.js': ['babel']
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['mocha', 'coverage', 'jenkins'],
plugins: [
require('karma-webpack'),
require('karma-mocha'),
require('karma-chai'),
require('karma-mocha-reporter'),
require('karma-jenkins-reporter'),
require('karma-chrome-launcher'),
require('karma-sourcemap-loader'),
require('karma-coverage'),
require('karma-phantomjs-launcher'),
require('karma-requirejs'),
require('karma-babel-preprocessor')
],
webpack: confBuilder,
webpackServer: {
noInfo: true,
},
coverageReporter: {
check: {
global: {
statements: 86,
branches: 80,
functions: 95,
lines: 40,
},
},
},
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['PhantomJS'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true,
autoWatch: true,
})
}
First of all I know there are similar questions, but none of it works for me.
I have an Angular app, and I would like to test it with Karma, Jasmin, and RequireJs.
I have installed everything I need, and configured everything as it is written here: http://karma-runner.github.io/0.13/plus/requirejs.html
Below is how my files look like
karma.conf.js
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['jasmine', 'requirejs'],
// list of files / patterns to load in the browser
files: [
'src/test/Tests/*.js',
'src/test/Tests/directives/*.js',
//source code
{pattern: 'src/main/webapp/app/*.js', included: false},
{pattern: 'src/main/webapp/app/**/*.js', included: false},
{pattern: 'src/test/test-main.js', included: true}
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'test/Tests/*.js': 'coverage'
},
coverageReporter:{
type:'html',
dir:'../coverage/'
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress', 'coverage'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['PhantomJS'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: false,
// Concurrency level
// how many browser should be started simultanous
concurrency: Infinity
})
}
test-main.js
var TEST_REGEXP = /(spec|test)\.js$/i;
var allTestFiles = [];
// Get a list of all the test files to include
Object.keys(window.__karma__.files).forEach(function(file) {
if (TEST_REGEXP.test(file)) {
// Normalize paths to RequireJS module names.
// If you require sub-dependencies of test files to be loaded as-is (requiring file extension)
// then do not normalize the paths
var normalizedTestModule = file.replace(/^\/base\/|\.js$/g, '');
allTestFiles.push(normalizedTestModule);
}
});
require.config({
// Karma serves files under /base, which is the basePath from your config file
baseUrl: '/base',
//dynamically load all test files
deps: allTestFiles,
// we have to kickoff jasmine, as it is asynchronous
callback: window.__karma__.start
});
The output is in IntelliJ IDEA 14.1.5
Empty test suite.
Process finished with exit code 0
The server starts correctly. I have also tried with command line. $ karma start, and $ karma run, which also does nothing. There is no error message at all. The path is correct, there is no 404 error.
Do you have any guess what could be the problem?
Delete the module names from test files.
In my case, this was the solution.
According to RequireJS documentation (http://requirejs.org/docs/api.html#modulename) you can include a name for the module as you can see below:
//Explicitly defines the "foo/title" module:
define("foo/title",
["my/cart", "my/inventory"],
function(cart, inventory) {
//Define foo/title object in here.
}
);
But for me, Karma only executes the tests if I delete the "foo/title" name.