I have a CakePHP Application with codeception-plugin for testing.'
Locally I run it in a ddev docker environment and everything works fine.
Trying to run automated tests with gitlab-ci gives me following error:
Running with gitlab-runner 11.1.0 (081978aa)
on shared runner 601c0f11
Using Docker executor with image kevinliteon/cakephp:php7 ...
Starting service mysql:latest ...
Pulling docker image mysql:latest ...
Using docker image sha256:6a834f03bd02bb88cdbe0e289b9cd6056f1d42fa94792c524b4fddc474dab628 for mysql:latest ...
Waiting for services to be up and running...
*** WARNING: Service runner-601c0f11-project-94-concurrent-0-mysql-0 probably didn't start properly.
Health check error:
service "runner-601c0f11-project-94-concurrent-0-mysql-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2018-10-04T12:12:18.904025613Z Initializing database
2018-10-04T12:12:18.925096235Z 2018-10-04T12:12:18.919745Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2018-10-04T12:12:18.925195518Z 2018-10-04T12:12:18.919970Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.12) initializing of server in progress as process 30
2018-10-04T12:12:50.330736417Z 2018-10-04T12:12:50.330487Z 5 [Warning] [MY-010453] [Server] root#localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
*********
Pulling docker image kevinliteon/cakephp:php7 ...
Using docker image sha256:bd4a83b02647ad93a356b343d2ce5ae3a9a1177aea2cd76c61b009abc7df8990 for kevinliteon/cakephp:php7 ...
Running on runner-601c0f11-project-94-concurrent-0 via d7f4a5e71b47...
Fetching changes...
Removing vendor/
HEAD is now at 92cb022 test
Checking out 92cb0223 as deployment...
Skipping Git submodules setup
Checking cache for default...
Successfully extracted cache
$ vendor/bin/codecept run Unit
Codeception PHP Testing Framework v2.3.9
Powered by PHPUnit 6.5.13 by Sebastian Bergmann and contributors.
In Db.php line 308:
Db: SQLSTATE[HY000] [2002] Connection refused while creating PDO connection
My gitlab-ci.yml (partly):
services:
- mysql:latest
variables:
MYSQL_ROOT_PASSWORD: mysql123456789
MYSQL_DATABASE: test_db
MYSQL_USER: db
MYSQL_PASSWORD: db
build:
...
codecept:Unit:
stage: test
script:
- vendor/bin/codecept run Unit
In my codeception.yml I configured the Db module:
modules:
config:
Db:
dsn: 'mysql:host=mysql;dbname=test_db'
user: 'db'
password: 'db'
cleanup: true # reload dump between tests
populate: true # load dump before all tests
reconnect: true
I also tryed using the root user - without success.
Problem is, that I can not connect to the DB for whatever reasons... Maybe the warnings while initializing the service container have something to do with that, but I could not figure out how to fix them or if this is the problem.
I really tried a lot of things without any success! Basically my code depends on the documentations of gitlab-ci and codeception so it should work.
Anybody implemented this scenario successfully or know what I'm doing wrong?
Thanks for any help!
I want to answer how I solved it:
First thing was, I had to add the env-varibale "db_dsn" like this:
export db_dsn="mysql://user:paswd#host/db"
Then I still got the health-check error. Only way I found to successfully setup was to use another docker image for the db-service. I choose "mariadb:latest" - and then it worked for me.
Related
I've got Gitlab installed on Kubernetes using Helm and try to get Auto DevOps working but I'm getting the following issue from the Auto DevOps pipeline:
Executing "step_script" stage of the job script 00:01
$ if [[ -z "$CI_COMMIT_TAG" ]]; then # collapsed multi-line command
$ /build/build.sh
Logging in to GitLab Container Registry with CI credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
Building Dockerfile-based application...
Attempting to pull a previously built image for use with --cache-from...
Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
No previously cached image found. The docker build will proceed without using a cached image
Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?
Cleaning up file based variables 00:00
ERROR: Job failed: command terminated with exit code 1
I've read abt the issues with Docker-in-Docker, but I couldn't get it to work.
I also tried setting the DOCKER_HOST: tcp://docker:2375/ variable, but I still get the same error.
Cannot connect to the Docker daemon at tcp://docker:2375/. Is the docker daemon running?
My Gitlab Runner Helm values looks like this:
gitlabUrl: https://gitlab.mydomain.com
privileged: true
rbac:
create: true
runnerRegistrationToken: mytoken
runners:
config: |
[[runners]]
executor = "docker"
privileged = true
environment = ["DOCKER_TLS_CERTDIR="]
[runners.docker]
tls_verify = false
privileged = true
I tried using the Auto-DevOps .gitlab-ci.yml template.
My cluster is integrated into Gitlab using the cluster certificate method and running simple CI/CD like
test_project:
stage: test
script:
- npm run test
Works fine
I tried docker:18.09.7, docker:19.03.1 and docker:latest (and the corresponding -dind services, with or without setting DOCKER_HOST: tcp://docker:2375/)
I don't know what I'm missing.
// Update
I got this issue resolved with help from the Gitlab Support.
My issue was that I had to use [runners.kubernetes] config and set privileged to true there. I was only setting it in [runners.docker] as I thought that Kubernetes Gitlab Runners would that config.
I've configured IntelliJ to use python via a stack I've defined in docker-compose. I'm configured my project to execute my pytest via docker-compose so that I can use the debugger. However, I've discovered that after the initial run, when I change my code and re-run my tests, pytest is not seeing my changes, but rather executing a cached version of the code.
The only way I've discovered to get around this is to invoke the File menu option Invalidate Caches and Restart. This is annoying.
This my compose file:
networks:
app: {}
services:
item-set-definitions:
build:
context: /Users/kudrykma/ghoildex/kudrykma/analytics/sa-item-set-definitions
target: build
command:
- /bin/bash
image: item-sets:test
networks:
app: {}
volumes:
- source: /Users/kudrykma/ghoildex/kudrykma/analytics/sa-item-set-definitions
target: /project
type: bind
version: '3.9'
In the pytest Run configuration I've tried adding -force-recreate option in the docker-compose Command and options field but IntelliJ won't recognize it.
Does anyone know how I can configure IntelliJ to not cache any of my source file so that pytest will see my changed code?
Thank you
I have a very basic integration configured for Gitlab-CI but it fails almost at the beginning when it has to clone the code.
My integration is this:
image: node:latest
stages:
- build
- test
cache:
paths:
- node_modules/
- dist/
build-prod:
stage: build
script:
- npm install
- npm run build-prod
artifacts:
paths:
- node_modules/
- dist/
test_with_karma:
stage: test
script: ng test
And the error that I get is this:
Running with gitlab-runner 11.7.0 (8bb608ff)
on fakehost 2eaf11ea
Using Docker executor with image node:latest ...
Pulling docker image node:latest ...
Using docker image sha256:8c67bfd7b95bdc535edc4a4144f5392b0f73efd6385fbcb47747d028d7059359 for node:latest ...
Running on runner-2eaf11ea-project-56-concurrent-0 via fakehost...
Cloning repository...
Cloning into '/builds/redacted/frontend'...
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#working-domain.com/redacted/frontend.git/': The requested URL returned error: 403
/bin/bash: line 65: cd: /builds/redacted/frontend: No such file or directory
ERROR: Job failed: exit code 1
What is the problem here?
Check if this is covered by gitlab-org/gitlab-ce issue 39469
YAY - it works for me. This problem seems to have multiple solutions.
The one that worked for me is #44855
To summarize. Being an Administrator on Gitlab does not mean you have the "access" to do whatever you want to do in Gitlab.
"Unable to access" permissions applies to the person who is logged into Gitlab and running the job.
To fix the problem - the person / account running the job must be a member (master) of the project.
This will apply to private projects.
It is not necessary to make a private project Public even though that appears to fix the problem. GITLAB suggests you must have https for the project to work you can use http.
SOLUTION - add your account to the project even if you are the Administrator
And:
Conrad has described it correctly.
You need to have rights to the project to run pipeline, however, as administrator, you can start any pipeline.
I've got the case when the user being Admin in Gitlab could push his commit from command line, although theoretically having no rights to project - and the pipeline has failed.
This inconsistency need to be fixed, either Admin user should not be able to push/start pipeline, having no rights for it, or he should authomatically be granted all rights to all projects. I'd prefer the first one, because it separates gitlab administration from project rights. Sometimes I prefer not having full rights, just like working as non-root under Linux.
I've been trying to get our tests to pass on our Gitlab CI, but can't. I'm using the stock pipelines config that comes with Gitlab. All I've had to do is provide the gitlab yaml file to config the CI.
This is what we're using
image: maven:3.5.0-jdk-8-alpine
services:
- postgres:latest
variables:
POSTGRES_DB: my_test_db
POSTGRES_USER: my_test_user
POSTGRES_PASSWORD: ""
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
ACTIVE_ENV: test
connect:
image: postgres
script:
# official way to provide password to psql: http://www.postgresql.org/docs/9.3/static/libpq-envars.html
- export PGPASSWORD=$POSTGRES_PASSWORD
- psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 'OK' AS status;"
stages:
- test
test:
stage: test
script:
- "mvn -Denvironments=test -B db-migrator:migrate; mvn -Denvironments=test -DACTIVE_ENV=test -B test"
Everything works perfectly up to the point where the tests run. Then they all error out with similar messages:
383 [main] WARN org.javalite.activeweb.DBSpecHelper - no DB connections are configured, none opened
456 [main] WARN org.javalite.activeweb.DBSpecHelper - no DB connections are configured, none opened
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.528 sec <<< FAILURE! - in app.models.RoleTest
validatePresenceOfUsers(app.models.RoleTest) Time elapsed: 0.071 sec <<< ERROR!
org.javalite.activejdbc.DBException: Failed to retrieve metadata from DB, connection: 'default' is not available
I have one database.properties file that is checked in and is for tests only (our dev and prod envs use jndi). It looks like so:
test.driver=org.postgresql.Driver
test.username=my_test_user
test.password=
test.url=jdbc:postgresql://postgres/edv_test
Again, migrations run using all this exact same config. I just can't figure out why the tests won't run. I understand why it's saying there's no default db, but I don't get why it's not seeing the test settings and configuring that connection as expected.
So you know, the Maven flag environments like this: mvn test -Denvironments=test only works for the DB-Migrator, and not for the tests. Any JavaLite application in a standard running mode or as a test, it will be looking at ACTIVE_ENV. If this is not set, it will assume development. In test mode, it will be looking at database.properties block development.test.xxx=yyy as in http://javalite.io/database_configuration#property-file-configuration.
Think of it as "development" environment, "test" mode.
Additionally, DbConfig is not involved in tests, as database connections in test have a special treatment (rollback transactions), see: http://javalite.io/testing_with_db_connection
Gitlab provides a .gitlab-ci.yml template for building and publishing images to its own registry (click "new file" in one of your project, select .gitlab-ci.yml and docker). The file looks like this and it works out of the box :)
# This file is a template, and might need editing before it works on your project.
# Official docker image.
image: docker:latest
services:
- docker:dind
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
build-master:
stage: build
script:
- docker build --pull -t "$CI_REGISTRY_IMAGE" .
- docker push "$CI_REGISTRY_IMAGE"
only:
- master
build:
stage: build
script:
- docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
- docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
except:
- master
But by default, this will publish to gitlab's registry. How can we publish to docker hub instead?
No need to change that .gitlab-ci.yml at all, we only need to add/replace the environment variables in project's pipeline settings.
1. Find the desired registry url
Using hub.docker.com won't work, you'll get the following error:
Error response from daemon: login attempt to https://hub.docker.com/v2/ failed with status: 404 Not Found
Default docker hub registry url can be found like this:
docker info | grep Registry
Registry: https://index.docker.io/v1/
index.docker.io is what I was looking for.
2. Set the environment variables in gitlab settings
I wanted to publish gableroux/unity3d images using gitlab-ci, here's what I used in Gitlab's project > Settings > CI/CD > Variables
CI_REGISTRY_USER=gableroux
CI_REGISTRY_PASSWORD=********
CI_REGISTRY=docker.io
CI_REGISTRY_IMAGE=index.docker.io/gableroux/unity3d
CI_REGISTRY_IMAGE is important to set.
It defaults to registry.gitlab.com/<username>/<project>
regsitry url needs to be updated so use index.docker.io/<username>/<project>
Since docker hub is the default registry when using docker, you can also use <username>/<project> instead. I personally prefer when it's verbose so I kept the full registry url.
This answer should also cover other registries, just update environment variables accordingly. 🙌
To expand on the GabLeRoux's answer,
I had issues on the pushing stage of the GitLab CI build:
denied: requested access to the resource is denied
By changing my CI_REGISTRY to docker.io (remove the index.) I was able to successfully push.