I'm running tests using tasks in VS Code and I've stacked with problem of parallel launching tests.
I want to pass two environments to env variable in order to start tests. With one environment, everything works perfect, but if I pass several -all tests start with default configuration in 4 threads.
Example:
var nightwatchOptions = {
config: './dist/dev/specs/e2e/nightwatch/nightwatch.json',
env: ['firefox', 'chrome'] ---> this one doesn't work
};
I also tried to pass it like that: env: 'firefox,chrome' and like that ['firefox,chrome'] - first one hangs, second one - default configuration.
env: 'chrome' ---> this one works perfect.
Any help will be appreciated!
Ok, I've just figured out how to avoid this issue. I decided to start it with the help of child_process of nodejs:
nightwatchOptions = ['node_modules/nightwatch/bin/runner.js',
'-c',
'path to config'];
var tests = child_process.spawn('node', nightwatchOptions, {
stdio: 'inherit'
});
And I've added test_workers to nightwatch config. Now it works fine for me.
Related
I have an application that I started in English from scratch. I had nightwatch tests and everything is working perfectly fine... but after adding 2 more languages, I want the main tests to run as they used to in English, then change the browser language (since that's the criteria on which I choose the language) so I can run the other tests in German or French... etc. Is there a way to start a test suite by changing the browser's language?
I looked into the documentation and found nothing in this area
In order to set the language and run the tests locally, I did the following:
chrome: {
desiredCapabilities: {
chromeOptions: {
prefs: {
intl: { accept_languages: "ss-ZA" }
}, args: []
}
}
}
this allows you to run the tests locally, on only that language. or you can manage setting that locale code to a variable in a way and try setting it at the beginning of your test suite.
I did not get that far, because in my case, I have to run my tests headless because they will run later on a remote git server that runs only headless tests. According to this issue on Nightwatch's github page, setting a browser language parameter for a headless test is not possible!
I am working on a TestCafe proof of concept. I have a few tests working in one test environment. I need a way to run the same tests in up to 3 different test environments that have different URLs. Is there a best practice for this scenario?
A solution is to add custom options on the testcafe command-line like for example : --env=uat.
Use minimist to read all custom options that you have added to the TestCafe command-line and export a config object like this:
import * as minimist from 'minimist';
const args = minimist(process.argv.slice(2));
// get the options --env=xxx --user=yyy from the command line
export const config = {
env: args.env,
user: args.user,
};
Then import that config object wherever you need it in the test code.
see How do I work with configuration files and environment variables? for more details.
In v1.20.0 and later, TestCafe offers a way to specify the baseUrl in the test run setup:
CLI
Program API runner.run({baseUrl})
Config file
You can use this approach along with environment variables or custom command line arguments to determine what url should be assigned to the baseUrl option.
Alternatively, you can have a different configuration file for each test run setup and switch between these files using the --config-file option.
I'm running tests with npm test - that actually runs a grunt task grunt casperjs:
casperjs:{
options:{},
files:
['./test_index.js',
'./test_map_regression.js',
'./test_index_get_gush.js'] /
},
using the grunt-casperjs-plugin in order to automate testing with slimerjs along with phantomjs, both running under casperjs in Travis-ci.
In order to do that, I need to pass the engine as a variable from the command line. something like:
casperjs --engine=slimerjs test_suite.js
Question: I can't find a way to pass the options from grunt cli (and I assume npm command line options would delegate to grunt. correctly?) to the files array.
I tried to add:
var engine = grunt.option('engine') || 'phantomjs';
engine = '--engine='+engine;
and then in the file array do:
files:['./test_index.js '+engine,
'./test_map_regression.js '+enging,
'./test_index_get_gush.js '+engine]
but seems that file array has to get real file names without the added args.
I'll be glad for any ideas on how to solve this through.
I haven't tested this, but looking at the grunt-casperjs source, it looks as though you would want to pass the engine as an option.
So, something like this should work:
casperjs:{
options: {
'engine': 'slimerjs'
},
files: [
'./test_index.js',
'./test_map_regression.js',
'./test_index_get_gush.js'
]
}
I have been playing with Intern and made my tests work in the browser. To better integrate with my current workflow I'm trying to run my tests through grunt with phantomjs. However all my attempts have failed. I've been looking at this question as a reference Intern WebDriver and PhantomJS but can't figure out all of the steps to make it work.
First of all: Is it even possible to run the tests through grunt and phantomjs?
A little bit of info:
I don't want/can't connect to Sauce Labs or a Selenium testing environment.
I want to test browser code while having jQuery as a shimmed dependency
I have Grunt 0.4.1 and phantomjs 1.9.1 installed
I'm not testing any ajax request (as the linked question is having problem with)
I'm not familiar with neither Selenium nor Sauce Lab
If it is possible to test through grunt and phanomjs, how would I go about doing it?
I guess that I have to start the GhostDriver
phantomjs --webdriver=8910
But then what are the important pieces of info in the Intern config to make this work?
define({
proxyPort: 9000,
proxyUrl: 'http://localhost:9000/',
environments: [
{
browserName: 'phantom',
version: '1.9.0',
platform: 'Linux'
}
],
webdriver: {
host: 'localhost',
port: 8910
},
maxConcurrency: 3,
useSauceConnect: false,
// ...
});
How does the environments browserName map to phantomjs? I've tried the browserNames 'phantom' as well as 'phanomjs' with different versions and platforms (running Mac 10.7)
My Gruntfile section for intern looks like:
intern: {
phantom: {
options: {
config: 'tests/intern',
reporters: ['webdriver']
}
}
}
Running this setup without any test-suite that includes browser code outputs
'ReferenceError: document is not defined' in 'node_modules/intern/lib/reporters/webdriver.js:41:19'
Adding browser tests gives 'ReferenceError: window is not defined' in 'src/vendor/jquery/jquery-1.9.1.js:9597:5'
Running it through node gives the same 'window is not defined' error.
node node_modules/intern/client.js config=tests/intern
What am I doing wrong/missing here?
There are two problems with your Gruntfile configuration:
The default run type for the Grunt task is 'client', which means it runs the Node.js client by default, not the test runner. You need to set runType: 'runner' in your Gruntfile options to run tests against your specified environments.
The webdriver reporter is for use in a browser client only and is specified automatically by the test runner when it is needed. You should typically never use it directly. The reporter you specify in the Gruntfile should be the one you want the test runner to use; the console default is usually appropriate.
These two things in combination mean that you’ve configured Intern to try to use a reporter that only works in a browser in conjunction with the test runner inside the Node.js client, hence the error. Once corrected, the remaining configuration should work properly to run on PhantomJS, assuming it is already running on localhost at port 8910.
I can run unit and midway tests, however when I want to run e2e tests, nothing happens. There is not tests found as the output of karma suggests:
C:\>karma start
INFO [karma]: Karma server started at http://localhost:9876/
INFO [launcher]: Starting browser IE
INFO [IE 10.0 (Windows)]: Connected on socket id 8lG9jAG8mloBmjFez5V9
IE 10.0 (Windows): Executed 0 of 0 SUCCESS (0.125 secs / 0 secs)
my karma.conf file is as followed:
files = [
JASMINE,
JASMINE_ADAPTER,
'Scripts/libs/jquery/jquery-*.js',
'Scripts/libs/angular/angular.js',
'Scripts/libs/angular/angular-mocks.js',
'Scripts/libs/angular/angular-resource.js',
'Scripts/libs/angular/angular-scenario.js',
'Scripts/sinon-1.7.3.js',
'app/**/index.js',
'app/**/*.js',
'app/*.js',
'test/unit/**/*.js',
'test/midway/**/*.js',
'test/e2e/*.js'
];
reporters = ['progress'];
port = 9876;
runnerPort = 9100;
colors = true;
logLevel = LOG_INFO;
autoWatch = true;
browsers = ['IE'];
captureTimeout = 60000;
singleRun = false;
proxies = {
'/': 'http://localhost:1506/portal.web'
};
Any idea what's wrong ?
I had the same problem - almost gave up - but followed http://blog.diniscruz.com/2013/06/running-karmas-angularjs-example.html - and it works!
AngularJS team recommends Protractor for E2E testing and Karma only for unit testing.
setting up karma for e2e testing
you will also need to comment out JASMINE AND JASMINE_ADAPTER inside files in the karma.conf.js and add frameworks = "jasmine" underneath this.
Hope this helps
// frameworks to use
frameworks: ['jasmine']
By adding these lines above in config file worked for me.
According to [https://github.com/ksunair/introtokarma][1]
frameworks: ['jasmine'],
files: [
//adapter - ignore deprecation warnings karma spits out -- you need this!
//JASMINE,
//JASMINE_ADAPTER,
ANGULAR_SCENARIO,
ANGULAR_SCENARIO_ADAPTER,
//includes
'client/common/vendor/angular.js',
//own files
'client/admin/views/admin.html',
//tests
//'test/client/*-unit-spec.js',
'test/client/*-e2e-spec.js'
]
(for ANGULAR_SCENARIO, you will have done npm install karma-ng-scenario --save-dev)
Jasmine is not E2E, E2E (although it looks much the same) is not Jasmine. You need to include either one or the other, never both.
See this answer for more.
FWIW, my versions:
karma: 0.10.8
karma-ng-scenario: 0.1.0
karma-jasmine: 0.1.5