I mirrored a private repository from Github to run builds with gitlab runner. My project has private gems hosted on Github and the build is failing with
Fetching git#github.com:private/gem.git
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Retrying `git clone 'git#github.com:private/gem.git' "/builds-ci/gitlab/repo/vendor/ruby/2.4.0/cache/bundler/git/gem-a356dd016736a58b8b77677e8d7df689f8f43ada" --bare --no-hardlinks --quiet` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone 'git#github.com:private/gem.git' "/builds/gitlab-ci/repo/vendor/ruby/2.4.0/cache/bundler/git/gem-a356dd016736a58b8b77677e8d7df689f8f43ada" --bare --no-hardlinks --quiet` in directory /builds/gitlab-ci/repo has failed.Host key verification failed.
fatal: Could not read from remote repository.
I used personal Github access token to mirror private repositories from Github to Gitlab. There is a way to use Github access token to clone private gems with bundler without the need to set up SSH keys:
export BUNDLE_GITHUB__COM=x-access-token:<token>
Do I need to create a separate access token and paste it into .gitlab-ci.yml
image: ruby:2.4.1
variables:
BUNDLE_GITHUB__COM=x-access-token:<token>
...
Or I can use the token which I used to mirror repositories from Github? Like this
variables:
BUNDLE_GITHUB__COM=x-access-token:$SOME_GITLAB_JOB_ACCESS_TOKEN
?
I managed to do it with adding BUNDLE_GITHUB__COM to .gitlab-ci.yml commiting it to the source code for now:
# .gitlab-ci.yml
variables:
BUNDLE_GITHUB__COM=x-access-token:<token>
...
Related
I'm trying to run a CircleCI test job locally by running
circleci local execute --job test
However, I'm getting this error message:
go: github.com/some/repo#v0.0.0-20180921204022-800easdf7ec: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /go/pkg/mod/cache/vcs/52f8e69c46f5a1cc77e6bf: exit status 128:
fatal: could not read Username for 'https://github.com': terminal prompts disabled
I would basically like to do the equivalent of https://circleci.com/docs/2.0/add-ssh-key/ for the local CircleCI environment, but there is no way to go to Project Settings -> Checkout SSH Keys as described in that documentation. I've read the documentation at https://circleci.com/docs/2.0/local-cli/#run-a-job-in-a-container-on-your-machine but wasn't able to find a way to do this.
Any idea how I can check out code from private Github repos in the local CircleCI environment?
Have you tried adding the key to your SSH keychain?
ssh-add (location of ssh key)
This should add it to the keychain and CircleCI local should pick it up.
You can use --checkout-key argument
circleci local execute build --checkout-key id_rsa
Note: id_rsa should be in the same folder
I have a very basic integration configured for Gitlab-CI but it fails almost at the beginning when it has to clone the code.
My integration is this:
image: node:latest
stages:
- build
- test
cache:
paths:
- node_modules/
- dist/
build-prod:
stage: build
script:
- npm install
- npm run build-prod
artifacts:
paths:
- node_modules/
- dist/
test_with_karma:
stage: test
script: ng test
And the error that I get is this:
Running with gitlab-runner 11.7.0 (8bb608ff)
on fakehost 2eaf11ea
Using Docker executor with image node:latest ...
Pulling docker image node:latest ...
Using docker image sha256:8c67bfd7b95bdc535edc4a4144f5392b0f73efd6385fbcb47747d028d7059359 for node:latest ...
Running on runner-2eaf11ea-project-56-concurrent-0 via fakehost...
Cloning repository...
Cloning into '/builds/redacted/frontend'...
remote: You are not allowed to download code from this project.
fatal: unable to access 'https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx#working-domain.com/redacted/frontend.git/': The requested URL returned error: 403
/bin/bash: line 65: cd: /builds/redacted/frontend: No such file or directory
ERROR: Job failed: exit code 1
What is the problem here?
Check if this is covered by gitlab-org/gitlab-ce issue 39469
YAY - it works for me. This problem seems to have multiple solutions.
The one that worked for me is #44855
To summarize. Being an Administrator on Gitlab does not mean you have the "access" to do whatever you want to do in Gitlab.
"Unable to access" permissions applies to the person who is logged into Gitlab and running the job.
To fix the problem - the person / account running the job must be a member (master) of the project.
This will apply to private projects.
It is not necessary to make a private project Public even though that appears to fix the problem. GITLAB suggests you must have https for the project to work you can use http.
SOLUTION - add your account to the project even if you are the Administrator
And:
Conrad has described it correctly.
You need to have rights to the project to run pipeline, however, as administrator, you can start any pipeline.
I've got the case when the user being Admin in Gitlab could push his commit from command line, although theoretically having no rights to project - and the pipeline has failed.
This inconsistency need to be fixed, either Admin user should not be able to push/start pipeline, having no rights for it, or he should authomatically be granted all rights to all projects. I'd prefer the first one, because it separates gitlab administration from project rights. Sometimes I prefer not having full rights, just like working as non-root under Linux.
I'm trying to automatically deploy my app to digital ocean through bitbucket pipelines. Here are the steps my deployment is following:
connect to the remote digital ocean droplet using ssh
clone my repository by running a git clone with ssh
launch my application with docker-compose
I have successfully setup ssh access to my remote. I have also configured ssh access to my repository and can successfully execute git clone from my remote server.
However, in the pipeline, while connection to the remote server is successfull, the git clone command fails with the following error.
git#bitbucket.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Anybody has an idea of what is going on here?
Here is my bitbucket-pipelines.yml
image: atlassian/default-image:latest
pipelines:
default:
- step:
deployment: production
script:
- cat deploy.sh | ssh $USER_NAME#$HOST
- echo "Deploy step finished"
And the deployment script deploy.sh
#!/usr/bin/env sh
git clone git#bitbucket.org:<username>/<my_repo>.git
cd my_repo
docker-compose up -d
Logs for the git clone ssh commands within the droplet and from the pipeline
Git uses the default ssh key by default.
You can overwrite the SSH command used by git, by setting the GIT_SSH_COMMAND environment variable. You can add the -i argument to use a different SSH key.
export GIT_SSH_COMMAND="ssh -i ~/.ssh/<key>"
git clone git#bitbucket.org:<username>/<my_repo>.git
From the git documentation:
GIT_SSH
GIT_SSH_COMMAND
If either of these environment variables is set then git fetch and git push will use the specified command instead of ssh when they need to connect to a remote system. The command-line parameters passed to the configured command are determined by the ssh variant. See ssh.variant option in git-config[1] for details.
$GIT_SSH_COMMAND takes precedence over $GIT_SSH, and is interpreted by the shell, which allows additional arguments to be included. $GIT_SSH on the other hand must be just the path to a program (which can be a wrapper shell script, if additional arguments are needed).
Usually it is easier to configure any desired options through your personal .ssh/config file. Please consult your ssh documentation for further details.
In official docs we can see:
# docker build github.com/creack/docker-firefox
It just works fine to me. docker-firefox is a repository and has Dockerfile within root dir.
Then I want to buid redis image and exact version 2.8.10 :
# docker build github.com/docker-library/redis/tree/99c172e82ed81af441e13dd48dda2729e19493bc/2.8.10
2014/11/05 16:20:32 Error trying to use git: exit status 128 (Initialized empty Git repository in /tmp/docker-build-git067001920/.git/
error: The requested URL returned error: 403 while accessing https://github.com/docker-library/redis/tree/99c172e82ed81af441e13dd48dda2729e19493bc/2.8.10/info/refs
fatal: HTTP request failed
)
I got error above. What's the right format with build docker image from github repos?
docker build url#ref:dir
Git URLs accept context configuration in their fragment section,
separated by a colon :. The first part represents the reference that
Git will check out, this can be either a branch, a tag, or a commit
SHA. The second part represents a subdirectory inside the repository
that will be used as a build context.
For example, run this command to use a directory called docker in the
branch container:
docker build https://github.com/docker/rootfs.git#container:docker
https://docs.docker.com/engine/reference/commandline/build/
The thing you specified as repo URL is not a valid git repository. You will get error when you will try
git clone github.com/docker-library/redis/tree/99c172e82ed81af441e13dd48dda2729e19493bc/2.8.10
Valid URL for this repo is github.com/docker-library/redis. So you may want to try following:
docker build github.com/docker-library/redis
But this will not work too. To build from github, docker requires Dockerfile in repository root, howerer, this repo doesn't provide this one. So, I suggest, you only have to clone this repo and build image using local Dockerfile.
One can use the following example which sets up a Centos 7 container for testing ORC file format. Make sure to escape the # sign:
$ docker build https://github.com/apache/orc.git\#:docker/centos7 -t orc-centos7
I met this problem when I was trying to config Jenkins to fetch the repository on the server of a fellow(university course).
Started by user anonymous
Building in workspace /var/lib/jenkins/jobs/TEST1/workspace
$ hg clone --rev default --noupdate ssh://XXX#partch.anu.edu.au/XXX/XXX /var/lib/jenkins/jobs/TEST1/workspace
remote: Host key verification failed.
abort: no suitable response from remote hg!
ERROR: Failed to clone ssh://XXX#partch.anu.edu.au/XXX/XXX
ERROR: Failed to clone ssh://XXX#partch.anu.edu.au/XXX/XXX
Finished: FAILURE
Then I tried to do the same command in terminal, with the different error
abort: Permission denied: /var/lib/jenkins/jobs/TEST1/workspace
I tried to change the location for the command:
hg clone --rev default --noupdate ssh://XXX#partch.anu.edu.au/XXX/XXX /home/administrator/TEST
And it works
According to all these, and the fact that this is no workspace(Error: no workspace)--I know it's because of no successful build ever.
Is it a problem with the workspace configuration of Jenkins? but don't know what to do.
Thank you very much if anyone gives me some advice!
Your error doesn't mention anything about a workspace. There error clearly states that it comes from remote hg and it is an authentication failure.
Also, are you using the Mercurial Plugin to perform the SCM checkout or trying to replicate it with a Build Shell build step?