I would like to add some e2e tests for my vue.js application and run them in the pipeline.
The corresponding part in my gitlab-ci.yml looks like this:
e2e:
image: node:8
before_script:
- npm install
services:
- name: selenium/standalone-chrome
alias: chrome
stage: testing
script:
- cd online-leasing-frontend
- npm install
- npm run test:e2e
And my nightwatch.js config:
{
"selenium": {
"start_process": false
},
"test_settings": {
"default": {
"selenium_port": 4444,
"selenium_host": "chrome"
}
}
}
Is “selenium_host”: “chrome” the correct way of setting the host to the selenium service?
I get the following error indicating that my e2e test can’t connect to the selenium service:
Connection refused! Is selenium server started?
Any tips?
The problem was that according to this issue, Gitlab CI is using the Kubernetes Executor instead of the Docker Executor which is mapping all Services to 127.0.0.1. After setting the selenium_host to this address, everything worked.
{
"selenium": {
"start_process": false
},
"test_settings": {
"default": {
"selenium_port": 4444,
"selenium_host": "127.0.0.1",
}
}
}
On the Selenium Repo it says:
"When executing docker run for an image with Chrome or Firefox please either mount -v /dev/shm:/dev/shm or use the flag --shm-size=2g to use the host's shared memory."
I don't know gitlab-ci so well, but I'm afraid it is not possible to add this as parameter to a service.
Related
I am running my whole system using docker-compose and I am trying to execute my end-to-end tests (written with Nightwatch) using a selenium/chrome service that is in the same network. My frontend is VueJS and the image is built from a node base image. I am running the tests with docker-compose exec frontend npm run test-selenium and getting the following output:
> frontend#0.1.0 test-selenium
> nightwatch -c nightwatch-selenium.conf.js
[Login Test] Test Suite
──────────────────────────────────────────────
ℹ Connected to chrome on port 4444 (1494ms).
Using: chrome (102.0.5005.61) on LINUX.
Running login:
───────────────────────────────────────────────────────────────────────────────────────────────────
ℹ Loaded url http://frontend:8090/login in 2049ms
NoSuchElementError: An error occurred while running .click() command on <Element [name=#loginBt]>: Timed out while waiting for element "#loginBt" with "css selector" to be present for 5000 milliseconds.
at Object.login (/app/tests/loginTest.js:8:8)
NoSuchElementError: An error occurred while running .setValue() command on <Element [name=#emailField]>: Timed out while waiting for element "input[name = "email"]" with "css selector" to be present for 5000 milliseconds.
at Page.fillUpData (/app/tests/page-objects/loginPage.js:13:21)
at Object.login (/app/tests/loginTest.js:9:8)
NoSuchElementError: An error occurred while running .setValue() command on <Element [name=#passField]>: Timed out while waiting for element "input[name = "password"]"
with "css selector" to be present for 5000 milliseconds.
at Page.fillUpData (/app/tests/page-objects/loginPage.js:13:21)
at Object.login (/app/tests/loginTest.js:10:8)
NoSuchElementError: An error occurred while running .click() command on <Element [name=#submitBt]>: Timed out while waiting for element "button[name = "submit"]" with
"css selector" to be present for 5000 milliseconds.
at Object.login (/app/tests/loginTest.js:11:8)
✖ Testing if the URL contains '/cards' in 5000ms - expected "contains '/cards'" but got: "http://frontend:8090/login" (5143ms)
at Object.login (/app/tests/loginTest.js:12:15)
FAILED: 1 assertions failed and 4 errors (28.22s)
When running tests locally, everything is working just fine. For local tests, I am using the chrome driver installed as dev dependency, accessing the frontend using http://localhost:8090/.
It is my first time dealing with E2E tests in docker, so I am not really sure what is going wrong. Any help is appreciated. Please find the source code below. Let me know if I need to add any more information.
package.json
"scripts": {
...
"test": "nightwatch",
"test-selenium": "nightwatch -c nightwatch-selenium.conf.js",
...
},
docker-compose.yml
services:
frontend:
build:
context: card-market-frontend/
dockerfile: Dockerfile
command: npm run serve
working_dir: /app
ports:
- "8090:8090"
chrome:
image: selenium/standalone-chrome:latest
hostname: chrome
privileged: true
shm_size: 2g
When running the tests with the test-selenium command I am using the nightwatch.conf below:
const chromedriver = require("chromedriver");
require("dotenv").config();
module.exports = {
src_folders: ["/tests"],
page_objects_path: ["tests/page-objects"],
test_workers: false,
selenium: {
start_process: false,
cli_args: {
"webdriver.chrome.driver": chromedriver.path,
},
},
webdriver: {
start_process: false,
},
test_settings: {
default: {
selenium_port: 4444,
selenium_host: "chrome",
screenshots: {
enabled: true,
path: "tests_output/",
on_failure: true,
},
desiredCapabilities: {
browserName: "chrome",
chromeOptions: {
w3c: false,
args: ["--no-sandbox"],
},
},
},
},
};
My page object loginPage.js
module.exports = {
url: `http://frontend:8090/login`,
elements: {
logoutBt: "#logoutBt",
loginBt: "#loginBt",
emailField: 'input[name = "email"]',
passField: 'input[name = "password"]',
submitBt: 'button[name = "submit"]',
},
commands: [
{
fillUpData(selector, data) {
return this.setValue(selector, data);
},
},
],
};
The test that is running loginTest.js
module.exports = {
login(browser) {
const page = browser.page.loginPage();
const email = "coolEmail#gmail.com";
const pass = "coolPass";
page
.navigate()
.click("#loginBt")
.fillUpData("#emailField", email)
.fillUpData("#passField", pass)
.click("#submitBt")
.assert.urlContains("/cards");
},
};
Your first error is:
NoSuchElementError: An error occurred while running .click() command on <Element [name=#logoutBt]>: Timed out while waiting for element "#logoutBt" with "css selector" to be present for 5000 milliseconds.
which indicates that NoSuchElementError was raised while searching for the Logout button.
However as per the test steps within loginTest.js, ideally you are not supposed to find the Logout button just after navigating to the page:
.navigate()
.click("#logoutBt")
You can find the Logout button only after logging in.
Solution
Ideally, the sequence of the commands will be:
page
.navigate()
.click("#loginBt")
.fillUpData("#emailField", email)
.fillUpData("#passField", pass)
.click("#submitBt")
.assert.urlContains("/cards");
.click("#logoutBt")
I am building a jenkins job that needs to run some tests against a selenium server. I have have defined a stage where I start the selenium server in 1 container, and after that I want to run my tests from another container, against the selenium server.
The selenium server seems to start fine, but after that the job just hangs, displaying a spinner:
This is what my pipeline script looks like:
agent {
kubernetes {
defaultContainer 'jnlp'
yaml """
apiVersion: v1
kind: Pod
spec:
containers:
- name: node
image: node:12.14.1
- name: selenium
image: vvoyer/selenium-standalone
"""
}
}
stages {
stage('Checkout codebase') {
// do checkout
}
stage('start selenium') {
steps {
container('selenium') {
sh '''
selenium-standalone install
selenium-standalone start //script hangs after this command
'''
}
}
}
stage('test') {
steps {
container('node') {
//build test project & run tests
}
}
}
}
}
Can anyone tell me what I am doing wrong here and how I can fix it?
Yeah Thanks, solved similar error by adding '&' at the end of shell script step inside Jenkinsfile.
ex:
sh label: '',script: 'docker-compose -f TestAutomation_UI_API/docker-compose-v3.yml up --scale chrome=3 &'
I have the following docker-compose.yml which starts a chrome-standalone container and a nodejs application:
version: '3.7'
networks:
selenium:
services:
selenium:
image: selenium/standalone-chrome-debug:3
networks:
- selenium
ports:
- '4444:4444'
- '5900:5900'
volumes:
- /dev/shm:/dev/shm
user: '7777:7777'
node:
image: node_temp:latest
build:
context: .
target: development
args:
UID: '${USER_UID}'
GID: '${USER_GID}'
networks:
- selenium
env_file:
- .env
ports:
- '8090:8090'
volumes:
- .:/home/node
depends_on:
- selenium
command: >
sh -c 'yarn install &&
yarn dev'
I'm running the containers as follows:
docker-compose up -d selenium
docker-compose run --service-ports node sh
and starting the e2e from within the shell.
When running the e2e tests, selenium can be reached from the node container(through: http://selenium:4444), but node isn't reachable from the selenium container.
I have tested this by VNC'ing into the selenium container and pointing the browser to: http://node:8090. (The node container is reachable on the host however, through: http://localhost:8090).
I first thought that docker-compose run doesn't add the running container to the proper network, however by running docker network inspect test_app I get the following:
[
{
"Name": "test_app_selenium",
"Id": "df6517cc7b6446d1712b30ee7482c83bb7c3a9d26caf1104921abd6bbe2caf68",
"Created": "2019-06-30T16:08:50.724889157+02:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.31.0.0/16",
"Gateway": "172.31.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"8a76298b237790c62f80ef612debb021549439286ce33e3e89d4ee2f84de3aec": {
"Name": "test_app_node_run_78427bac2fd1",
"EndpointID": "04310bc4e564f831e5d08a0e07891d323a5953fa936e099d20e5e384a6053da8",
"MacAddress": "02:42:ac:1f:00:03",
"IPv4Address": "172.31.0.3/16",
"IPv6Address": ""
},
"ef087732aacf0d293a2cf956855a163a081fc3748ffdaa01c240bde452eee0fa": {
"Name": "test_app_selenium_1",
"EndpointID": "24a597e30a3b0b671c8b19fd61b9254bea9e5fcbd18693383d93d3df789ed895",
"MacAddress": "02:42:ac:1f:00:02",
"IPv4Address": "172.31.0.2/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "selenium",
"com.docker.compose.project": "test_app",
"com.docker.compose.version": "1.24.1"
}
}
]
Which shows both containers running on the "selenium" network. I'm not sure however if the node container is properly aliased on the network and if this is proper behaviour.
Am I missing some config here?
Seems like docker-compose run names the container differently to evade the service namespace as noted in docker-compose.yml. http://node:8090 was therefore not reachable.
I solved this by adding a --name flag as follows:
docker-compose run --service-ports --name node node sh
EDIT:
It took me a while to notice, but I was overcomplicating the implementation by a lot. The above docker-compose.yml can be simplified by adding host networking. This simply exposes all running containers on localhost and makes them reachable on localhost by their specified ports. Considering that I don't need any encapsulation (it's meant for dev), the following docker-compose.yml sufficed:
version: '3.7'
services:
selenium:
image: selenium/standalone-chrome:3
# NOTE: port definition is useless with network_mode: host
network_mode: host
user: '7777:7777'
node:
image: node_temp:latest
build:
context: .
target: development
args:
UID: '${USER_UID}'
GID: '${USER_GID}'
network_mode: host
env_file:
- .env
volumes:
- .:/home/node
command: >
sh -c 'yarn install &&
yarn dev'
I am trying to use GitLab CI to run some client-side unit test written using QUnit. Now to run the Qunit test I am using the grunt-contrib-qunit plugin. To run these tests in headless mode I am using this plugin which hosts it on a localhost server in a console and runs all unit tests. When running this project locally I am successfully able to run all the unit tests but when I checking in my code which kicks of the CI process, on GitLab, it fails on starting the phantomjs server and gives timeout error. I am also providing the jsbin link of the two text files which are basically the output of the unit test from my console. One file is of my local system and another is from the GitLab CI that runs on GitLab website when I check-in my code.
Local Console Output File Dump
Gitlab CI Output Dump
Adding my gitlab-ci.yaml file
image: node:4.2.2
before_script:
- dir
- cd sapui5_repo
- dir
- cd app-with-tests
build:
stage: build
script:
- npm i
- npm run test
cache:
policy: push
paths:
- node_modules
artifacts:
paths:
- built
Also adding my gruntfile if that helps
/* global module */
module.exports = function (grunt) {
grunt.initConfig({
qunit: {
all: {
options: {
timeout: 9000,
urls: [
"http://localhost:9000/webcontent/test/unit/unitTests.qunit.html"
]
}
},
//all: ["webcontent/test/unit/unitTests.qunit.html"],
options: {
timeout: 2000,
}
},
connect: {
options: {
//open: true,
},
first: {
options: {
port: 9000,
//livereload: 3500,
base: "./"
}
},
second: {
options: {
open: {
target: "http://localhost:9000/webcontent"
},
keepalive: true,
port: 9000,
livereload: 3501,
base: "./",
}
}
},
});
grunt.loadNpmTasks("grunt-contrib-connect");
grunt.loadNpmTasks("grunt-contrib-qunit");
grunt.registerTask("test", [
"connect:first", "qunit"
]);
grunt.registerTask("default", [
"connect:second"
]);
};
i'm trying to configure the integration to run portractor tests.
I'm using grunt-protractor-runner task
with following configuration:
protractor: {
options: {
configFile: "protractor.conf.js", //your protractor config file
keepAlive: true, // If false, the grunt process stops when the test fails.
noColor: false, // If true, protractor will not use colors in its output.
args: {
// Arguments passed to the command
}
},
run: {},
chrome: {
options: {
args: {
browser: "chrome"
}
}
}
}
and here is grunt task which i use for running the protractor after the server is running:
grunt.registerTask('prot', [
'connect:test',
'replace:includemocks',//for uncommenting angular-mocks reference
'protractor:run',
'replace:removemocks',//for commenting out angular-mocks reference
]);
It is running well on my local machine, but at codeship i'm getting following error:
Error: Could not find chromedriver at /home/rof/src/bitbucket.org/myrepo/myFirstRepo/node_modules/grunt-protractor-runner/node_modules/protractor/selenium/chromedriver.exe
Which i guess, a result of not having this "chromedriver.exe" at this path.
How can i solve it in codeship environment?
Thanks forwards
Add postinstall to your package.json file and that way npm install will take care of placing the binaries for you ahead of time:
"scripts": {
"postinstall": "echo -n $NODE_ENV | \
grep -v 'production' && \
./node_modules/protractor/bin/webdriver-manager update || \
echo 'will skip the webdriver install/update in production'",
...
},
And don't forget to set NODE_ENV ... not setting it at all will result in echo 'will skip the webdriver install/update in production' piece running. Setting it to dev or staging will get desired results.
Short answer (pulkitsinghal gave the original solution):
./node_modules/grunt-protractor-runner/node_modules/protractor/bin/webdriver-manager update
I'm one of the founders at Codeship.
The error seems to be because you are trying to use the exe file, but we're on Linux on our system. Did you hardcode that executable?
Could you send us an in-app support request so we have a link to look at and can help you fix this?