I have created gocd, server and agent using docker-compose.
agent image is gocd/gocd-agent-docker-dind:v22.3.0.
In my GOCD pipeline job i am running the command "docker run hello-world"
If I run my pipeline the hangs at:
Waiting for console logs…
and does not print anything
My initial error was "Please make sure [docker] can be executed on this agent." when i was using "gocd/gocd-agent-alpine-3.8:v19.1.0" image
then i changed to dind image, not it doesnt give any error but stays hanged
Related
I am a newbie in gitlabci. I want to understand why do we need docker dind image in order to build a docker image in GitLab CI jobs. Why can't we use the docker executor and run docker commands under scripts?
When we register docker executor gitlab runner, we choose one image..
Again inside gitlabci, we choose an image under image: or services: fields. So does that mean this GitLab CI job container runs inside the docker executor container?
why do we need docker dind image in order to build a docker image in GitLab CI jobs. Why can't we use the docker executor and run docker commands under scripts?
This partly depends on how you have configured your GitLab runner.
Why docker doesn't work inside containers
When you invoke docker commands, they are really talking to a docker daemon which is needed to perform builds and carry out other docker commands. Typically, jobs running under the docker executor do not have access to any docker daemon by default. It's the same kind of problem you would face if you tried to run docker inside of a docker container you started locally.
Even if I can run docker successfully on my host:
$ docker run --rm docker /bin/sh -c 'hello from container $HOSTNAME'
hello from container 2b51479b11b1
I cannot run docker inside the container
$ docker run --rm docker /bin/sh -c 'docker info'
errors pretty printing info
Client:
Context: default
Debug Mode: false
Server:
ERROR: error during connect: Get "http://docker:2375/v1.24/info": dial tcp: lookup docker on 192.168.65.5:53: no such host
The same error would happen trying to run any other significant docker command like build, run, etc.
An exception to this would be if you configured your GitLab runner to run containers in privileged mode and mount /var/run/docker.sock to all your jobs (this would not be advisable) in which case all your jobs could talk directly to the docker daemon on the host. Another exception might be if you use the shell executor instead and you have docker installed on the host where the runner is running.
How the dind service fixes this
The docker:dind service is a daemon that is created just for your job. This is incredibly important because it can prevent concurrent jobs from stepping on one another or being able to escalate access where they might not otherwise have it.
When the build starts, the GitLab runner will create two containers: your job container and the docker:dind container; they are linked together. When your job invokes docker commands, your job connects to the docker:dind container, which then carries out the requested commands.
Any containers created by your job (say, by invoking docker run or docker build as part of your job) are managed by the daemon running on the docker:dind container, not the host daemon. If you run docker ps inside the job, you'll notice that none of the containers run on the host daemon are listed, despite the fact that if you ran docker ps on the host, you would see the job container, the dind container, and any other running containers.
To clarify your other questions:
When we register docker executor gitlab runner, we choose one image
The image specified in your runner configuration is simply the default docker image to be used if a job doesn't declare any image: key. It does not affect how the runner runs in any way.
inside gitlabci, we choose an image under image: or services: fields
When the docker executor runs your job, it uses docker run to do so. The image: key determines which image is used to run your job. Similarly, services: define the image used for service containers -- service containers are siblings to the job container and are connected with links.
So does that mean this GitLab CI job container runs inside the docker executor container?
No. I'd also like to clear up: the runner/executor doesn't run in a container, necessarily. Runners might be installed as a Windows service, or simply even a process running directly on a system. You can use runners that happen to be inside containers, but it doesn't materially affect how jobs are run.
In any case, the containers where your job run are generally always going to be run directly by the host docker daemon.
I am trying to get selenium side runner to run some tests using docker, to include in our CI.
I am able to run the tests locally in my machine by running:
selenium-side-runner C:\path-to-tests\tests-selenium.side
This is windows host.
I am trying to do the same using docker locally, so afterwards I will migrate this to our Teamcity.
First I am running the selenium server container:
docker run -d -p 4444:4444 --name chromedriver selenium/standalone-chrome:3.4.0
Afterwards I run the selenium side runner container:
docker run -v C:\path-to-tests:/sides --link chromedriver:chromedriver nixel2007/docker-selenium-side-runner
I have to link the containers otherwise I get an error saying that the container can't connect to chromedriver:4444
I also have to mount the volume where my tests are.
When I do this and run, I get the following error:
Test suite failed to run
WebDriverError: Unable to parse new session response
What am I missing here?
UPDATE:
I also tried different versions of the selenium/standalone-chrome container, selenium/standalone-chrome:3.4.0, selenium/standalone-chrome:3.141.59-xenon and selenium/standalone-chrome:latest
All fail with different errors.
SECOND UPDATE:
I have been able to get the tests to run, both locally and in teamcity. One of the issues that I am facing right now is that docker-compose seems to hang. Not sure if this is container related, or docker-compose related.
When I run the tests, the selenium side runner container exits with code 1 and I do not get back to the host console prompt, it stays forever waiting for something to happen.
The error is this:
selenium_selenium-side-runner_1 exited with code 1
I have gotten the docker-compose file from here:
https://github.com/nixel2007/docker-selenium-side-runner/blob/master/docker-compose.yml
Any clues on what I might be missing?
On my local dev machine I use docker.from_env(version=auto) (docker-py) to get a docker client without any issues. If I run the same statement inside a gitlab-ci job which run inside a docker container (or with gitlab-runner exec docker ... during local debugging) to get a docker client I get a DockerException: Error while fetching server API version: ('Connection aborted.', error(2, 'No such file or directory')). How can I get a docker client with docker-py inside a gitlab-ci job which runs inside a docker container?
I think you'll need to build and deploy your own runner with the following binding '/var/sock/docker.sock:/var/sock/docker.sock.
I am running “dev mode” by leveraging pre-generated orderer and channel artifacts for a sample dev network
here cli require image: hyperledger/fabric-tools by default it is trying to pull latest tag image and showing errorlatest image. and it throwing error
Error response from daemon: manifest for hyperledger/fabric-tools:latest not found
so I pull image hyperledger/fabric-tools:x86_64-1.0.0, and rename it with hyperledger/fabric-tools:latest( not sure it is proper way or not ) by :
docker pull hyperledger/fabric-tools:x86_64-1.0.0
docker tag hyperledger/fabric-tools:x86_64-1.0.0 hyperledger/fabric-tools
My network is running successfully but unfortunately cli container is stopped running.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d10d170cd2fa hyperledger/fabric-tools:x86_64-1.0.0 "/bin/bash -c ./sc..." 29 seconds ago Exited (1) 27 seconds ago cli
163f494bb85f hyperledger/fabric-ccenv "/bin/bash -c 'sle..." 59 minutes ago Up About a minute chaincode
e96e86930d94 hyperledger/fabric-peer "peer node start -..." 59 minutes ago Up About a minute 0.0.0.0:7051->7051/tcp, 0.0.0.0:7053->7053/tcp peer
c568480e30d2 hyperledger/fabric-orderer "orderer" 59 minutes ago Up About a minute 0.0.0.0:7050->7050/tcp
You can use the tools container as the cli container.
docker exec -it d10d170cd2fa /bin/bash
Can you post logs of cli container by issuing command docker logs <containerId>? cli container exit doesn't necessarily mean there's any error about the e2e test.
If you started the services using docker-compose, you can run either of: docker-compose restart -f docker-compose-simple.yaml cli or docker-compose up -f docker-compose-simple.yaml cli.
However, if you started your network AFTER having tagged the fabric-tools image as above, you should examine the logs of your exited container with docker logs cli, to determine why it exited.
It can be because of previously running docker containers. In my case first time it worked correctly but it gave error in second time. Killing and removing created docker containers using
docker rm container_name
and starting containers again, solved the problem.
I am attempting to dockerize a GUI app and have had some success. If I build the dockerfile into an image and then perform a docker run --name testcontainer testimage it appears that the process begins but the abruptly stops. I then check the container with docker ps to confirm no containers are running. Then I check docker ps -a and can see that it exited with status code exit(0). Then if I run the command docker start testcontainer, it appears to start the ENTRYPOINT command again, but this time it is able to continue and the GUI pops up.
My best guess is that I think that when I run the docker run command, the process begins but might be forked into a background process, causing the container to exit since the foreground process has ended. Although that could be way off because you would think the docker start command would result in the same outcome. I was thinking of trying to force the process to stay in the foreground, but do not know how to do that. Any suggestions?
UPDATE: I editted my Dockerfile to use supervisord to manage the starting of the GUI app. Now my docker run command will start supervisor, which will start my GUI app, and it works. Some things to note about this are that supervisor shows:
INFO spawned: myguiapp with pid 7
INFO success: myguiapp entered RUNNING state
INFO: exited: myguiapp (exit status 0; expected)
Supervisor and the container are still running at this point, which seems to indicate that the main process kicks off a child process. Since supervisor is still running, my container stays up and the GUI app does show up and I can use it. When I close the GUI, supervisor reports:
CRIT reaped unknown pid 93
Supervisor remains running, causing the container NOT to close. So I have to CTRL-C to kill supervisor. I'd rather not use supervisor, but if I need to, I would like for supervisor to close itself gracefully when that child process ends. If I could figure out how to get my container or supervisor to track child processes of the main process, then I think this would be solved.
The first issue is probably because your application requires a tty and you are not allocating a pseudo tty. Try running your container like this:
docker run -t --name testcontainer testimage
When you do a docker start the second time around it somehow allocates the pseudo-tty and the process keeps on running. I tried it myself. I couldn't find this info anywhere in the Docker docs though.
Also, if your UI is interactive you would want:
docker run -t -i --name testcontainer testimage