I'm trying to run a CircleCI test job locally by running
circleci local execute --job test
However, I'm getting this error message:
go: github.com/some/repo#v0.0.0-20180921204022-800easdf7ec: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /go/pkg/mod/cache/vcs/52f8e69c46f5a1cc77e6bf: exit status 128:
fatal: could not read Username for 'https://github.com': terminal prompts disabled
I would basically like to do the equivalent of https://circleci.com/docs/2.0/add-ssh-key/ for the local CircleCI environment, but there is no way to go to Project Settings -> Checkout SSH Keys as described in that documentation. I've read the documentation at https://circleci.com/docs/2.0/local-cli/#run-a-job-in-a-container-on-your-machine but wasn't able to find a way to do this.
Any idea how I can check out code from private Github repos in the local CircleCI environment?
Have you tried adding the key to your SSH keychain?
ssh-add (location of ssh key)
This should add it to the keychain and CircleCI local should pick it up.
You can use --checkout-key argument
circleci local execute build --checkout-key id_rsa
Note: id_rsa should be in the same folder
Related
I have been using the docker build --ssh flag to give builds access to my keys from ssh-agent.
When I try the same thing with podman it does not work. I am working on macOS Monterey 12.0.1. Intel chip. I have also reproduced this on Ubuntu and WSL2.
❯ podman --version
podman version 3.4.4
This is an example Dockerfile:
FROM python:3.10
RUN mkdir -p -m 0600 ~/.ssh \
&& ssh-keyscan github.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh git clone git#github.com:ruarfff/a-private-repo-of-mine.git
When I run DOCKER_BUILDKIT=1 docker build --ssh default . it works i.e. the build succeeds, the repo is cloned and the ssh key is not baked into the image.
When I run podman build --ssh default . the build fails with:
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Error: error building at STEP "RUN --mount=type=ssh git clone git#github.com:ruarfff/a-private-repo-of-mine.git": error while running runtime: exit status 128
I have just begun playing around with podman. Looking at the docs, that flag does appear to be supported. I have tried playing around with the format a little, specifying the id directly for example but no variation of specifying the flag or the mount has worked so far. Is there something about how podman works that I may be missing that explains this?
Adding this line as suggested in the comments:
RUN --mount=type=ssh ssh-add -l
Results in this error:
STEP 4/5: RUN --mount=type=ssh ssh-add -l
Could not open a connection to your authentication agent.
Error: error building at STEP "RUN --mount=type=ssh ssh-add -l": error while running runtime: exit status 2
Edit:
I belive this may have something to do with this issue in buildah. A fix has been merged but has not been released yet as far as I can see.
The error while running runtime: exit status 2 does not to me appear to be necessarily related to SSH or --ssh for podman build. It's hard to say really, and I've successfully used --ssh like you are trying to do, with some minor differences that I can't relate to the error.
I am also not sure ssh-add being run as part of building the container is what you really meant to do -- if you want it to talk to an agent, you need to have two environment variables being exported from the environment in which you run ssh-add, these define where to find the agent to talk to and are as follows:
SSH_AUTH_SOCK, specifying the path to a socket file that a program uses to communicate with the agent
SSH_AGENT_PID, specifying the PID of the agent
Again, without these two variables present in the set of exported environment variables, the agent is not discoverable and might as well not exist at all so ssh-add will fail.
Since your agent is probably running as part of the set of processes to which your podman build also belongs to, at the minimum the PID denoted by SSH_AGENT_PID should be valid in that namespace (meaning it's normally invalid in the set of processes that container building is isolated to, so defining the variable as part of building the container would be a mistake). Similar story with SSH_AUTH_SOCK -- the path to the socket file dumped by starting the agent program, would not normally refer to a file that exists in the mount namespace of the container being built.
Now, you can run both the agent and ssh-add as part of building a container, but ssh-add reads keys from ~/.ssh and if you had key files there as part of the container image being built you wouldn't need --ssh in the first place, would you?
The value of --ssh lies in allowing you to transfer your authority to talk to remote services defined through your keys on the host, to the otherwise very isolated container building procedure, through use of nothing else but an SSH agent designed for this very purpose. That removes the need to do things like copying key files into the container. They (keys) should also normally not be part of the built container, especially if they were only to be used during building. The agent, on the other hand, runs on the host, securely encapsulates the keys you add to it, and since the host is where you'd have your keys that's where you're supposed to run ssh-add at to add them to the agent.
I'm trying to automatically deploy my app to digital ocean through bitbucket pipelines. Here are the steps my deployment is following:
connect to the remote digital ocean droplet using ssh
clone my repository by running a git clone with ssh
launch my application with docker-compose
I have successfully setup ssh access to my remote. I have also configured ssh access to my repository and can successfully execute git clone from my remote server.
However, in the pipeline, while connection to the remote server is successfull, the git clone command fails with the following error.
git#bitbucket.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Anybody has an idea of what is going on here?
Here is my bitbucket-pipelines.yml
image: atlassian/default-image:latest
pipelines:
default:
- step:
deployment: production
script:
- cat deploy.sh | ssh $USER_NAME#$HOST
- echo "Deploy step finished"
And the deployment script deploy.sh
#!/usr/bin/env sh
git clone git#bitbucket.org:<username>/<my_repo>.git
cd my_repo
docker-compose up -d
Logs for the git clone ssh commands within the droplet and from the pipeline
Git uses the default ssh key by default.
You can overwrite the SSH command used by git, by setting the GIT_SSH_COMMAND environment variable. You can add the -i argument to use a different SSH key.
export GIT_SSH_COMMAND="ssh -i ~/.ssh/<key>"
git clone git#bitbucket.org:<username>/<my_repo>.git
From the git documentation:
GIT_SSH
GIT_SSH_COMMAND
If either of these environment variables is set then git fetch and git push will use the specified command instead of ssh when they need to connect to a remote system. The command-line parameters passed to the configured command are determined by the ssh variant. See ssh.variant option in git-config[1] for details.
$GIT_SSH_COMMAND takes precedence over $GIT_SSH, and is interpreted by the shell, which allows additional arguments to be included. $GIT_SSH on the other hand must be just the path to a program (which can be a wrapper shell script, if additional arguments are needed).
Usually it is easier to configure any desired options through your personal .ssh/config file. Please consult your ssh documentation for further details.
versiont Control > GitHub > "Test Successufl"
IMG1
Git version : 2.18.0
SSH ececutable: Native
IMG2
BUT!!
git clone fail
IMG3
I don't know why this happens any help would be appreciated.
You are cloning via HTTP. SSH executable is not related.
Check if it works in the command line first. It could happen there is git credential.helper that somehow saved wrong credentials and git is trying to use them.
You could use SSH instead, but make sure SSH keys are registered on GitHub and, since you want to use native SSH client, the key is added to ssh-agent or does not have a passphrase, because IntelliJ is not a terminal and cannot handle interactive prompts for passphrases.
Everybody Thanks~
I resolved it in the following way.
Step 1. Create SSH Key (in Local PC)
My PC use Windows.
So, I use Git Bash.
Step 2. Register GitHub
Personal settings > SSH and GPG Keys
Nes SSH Key > XXX_rsa.pub (This contens is Created SSH Key File in Local PC)
Step 3. In intellij, Git Clone used "Use SSH" instead of "Use HTTPS".
I installed Gitlab 6 on my MBP running OS X 10.8.5 and works fine. I can create projects, users, commit, push ... But only from there.
I try to push project from my iMac ( I do git config, init, add, commit), I generated also the ssh keys.
When a test the connection : ssh -T git#my_server it gives "Welcome to Gitlab, Anonymous".
but when I issue the push -u origin master I've got :
Blockquote
Access denied
Fatal: Could not read from remote repository
Please make sure you have the correct access rights and the repository exists.
Blockquote
Is it something relating with SSH or with Gitlab itself ?
In Gitlab 6 a added a 2d key to the project, my public key (generated on the iMac).
On my MBP I added the content of the id_rsa.pub (iMac) to the authorised_keys and known_hosts files (MBP)
Thank.
I'm setting up a git environment on Windows XP (msysGit 1.7.11, TortoiseGit 1.7.14) and trying to achieve following points :
ssh connection on a port different than default 22
ssh authentification handled by ssh-agent
So I create a ~/.ssh/config file :
Host gitbox
User gitolite
Hostname XX.XX.XX.XX
Port 154
PreferredAuthentications publickey
IdentityFile "/c/Documents and Settings/kraymer/.ssh/id_rsa"
When using the git bash CLI, everything works as intended.
I'm struggling with TortoiseGit.
I first installed TortoiseGit with Plink and using Pageant to load ssh private key. The automatic authentication (Pageant) worked but setup was a fail as TortoiseGit don't recognize git repos url formatted as gitolite#gitbox/repo.git.
I then decided to install TortoiseGit using openssh client, so the config file can be read by the ssh client, and to mimic the git CLI setup.
I picked the ssh.exe shipped with msys git as ssh client in TortoiseGit settings.
When doing a git pull, the remote url is now resolved but the passphrase password is asked while I expect ssh-agent automatic authentication to occur.
Is it possible to make TortoiseGit work with ssh-agent ?
Or make TortoiseGit (Plink) aware of .ssh/config ?
Edit #1
Following #VonC advice I configured my $HOME variable.
When I click Show environment variables in TortoiseGit I now have :
HOME=C:\Documents and Settings\kraymer
HOMEDRIVE=C:
HOMEPATH=\Documents and Settings\kraymer
But git pull still require I enter passphrase.
No tweaks needed.
Just make TortoiseGit point to the same ssh client used by git itself, see the screenshot:
This should be C:\Program Files\Git\usr\bin\ssh.exe in latest version of Git as mentioned by Aleksey Kontsevich in the comments.
I first installed TortoiseGit with Plink and using Pageant to load ssh private key. The automatic authentication (Pageant) worked but setup was a fail as TortoiseGit don't recognize git repos url formatted as gitolite#gitbox/repo.git.
I finally found a workaround which consist to create a PuTTY session with the same name that the ssh alias (ie gitbox in the question).
This way I can clone as git clone gitbox/monrepo in the CLI and the origin syntax is correctly handled by TortoiseGit.
Windows10 System
#TortoiseGit
In Network Section
From : C:\Program Files\TortoiseGit\bin\TortoiseGitPlink.exe
To : "C:\Users{user}\AppData\Local\Programs\Git\usr\bin\ssh.exe"
There seems to be a whole variety of options to solve this. As none of the above have worked for me, I tought I'd share what helped for me.
In Settings... -> Network -> SSH -> SSH client, set the client to C:\Program Files\TortoiseGit\bin\TortoiseGitPlink.exe. Using Pageant, you're then automatically authenticated as expected, otherwise you are prompted for your private key passphrase. Make sure the "Autoload Putty Key" option is checked in the push dialog.
Cygwin
Use approach described in the following article:
https://help.github.com/articles/working-with-ssh-key-passphrases
Password will be asked only once on the cygwin session startup.
!!! Before exiting cygwin session don't forget to kill ssh-agent process (use ps for find process PID and kill -9).
We are using separate approach for cygwin, because cygwin by some reason doesn't see processes started externally in windows environment.
2, 3) MsysGit, TortoiseGit
Useful link:
http://dogbiscuit.org/mdub/weblog/Tech/WindowsSshAgent
Install MsysGit.
Install TortoiseGit (check openssh instead of plink during installation).
!!! Check systems variables. If there is GIT_SSH variable present - remove it.
Go to TortoiseGit->Settings->General
Set Git exe Path to /bin
Set External dll path to /mingw/bin
Go to TortoiseGit->Settings->Network
Set SSH Client property to /bin/ssh.exe
Define system variable SSH_AUTH_SOCK=C:\temp.ssh-socket
Start cmd.exe and execute following commands(since we installed MsysGit all following commands are accessible in cmd - /bin is added to system PATH variable):
# following command is required to execute for avoiding Address already bind message when ssh-agen is not started yet but .ssh-socket exists after previous agent session
rm "%SSH_AUTH_SOCK%"
# Starting ssh-agent
ssh-agent -a "%SSH_AUTH_SOCK%"
# Adding our openssh key
ssh-add "%USERPROFILE%\.ssh\id_rsa"
# Type password for your key
That's it. From that moment you can execute git push, git pull from TortoiseGit and MsysGit without prompting passphrase.
When ssh-agent is no longer required you can kill it through windows task manager.
None of the above answers worked for me.
I created this batch file to solve the problem.
CALL "C:\Program Files\Git\cmd\start-ssh-agent.cmd"
SETX SSH_AUTH_SOCK "%SSH_AUTH_SOCK%"
SETX SSH_AGENT_PID "%SSH_AGENT_PID%"
Run this once, and enter your passphrase.
Then you can use tortoisegit with openssh without having to enter your passphrase for every operation.
Make sure to launch your TortoiseGit in an environment where HOME is defined, and reference the parent directory of .ssh.
This is important since, on Windows, HOME isn't defined by default.
See as an example: "Auth fails on Windows XP with git and tortoisegit".
(Other possible sources: "How to I tell Git for Windows where to find my private RSA key?")
Since the explanations here are a bit outdated, I decided to post my solution.
I am using Git Bash and TortoiseGit 2.8.0 in Windows 10, which are common nowadays.
I set ssh.exe as SSH client in Settings->Network as explained in previous posts.
I created a script with the following commands, as explained in a previous comment. You might also want to set a HOME environment variable, in case your system does not do it automatically. Assuming your home is in drive H:\ you can add the following lines:
SETX HOME /h
CALL "C:\Program Files\Git\cmd\start-ssh-agent.cmd"
SETX SSH_AUTH_SOCK "%SSH_AUTH_SOCK%"
SETX SSH_AGENT_PID "%SSH_AGENT_PID%"
I added the script using Win logo+R shell:startup to the startup folder. Alternatively, you can add the script to the registry to guarantee that it runs before other processes:
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run
Make sure to type exit to close the console and allow for the variable to be set for future processes.
If you use RSA keys in repositories, add at the end of the script as described above^
CALL "C:\Program Files\Git\cmd\start-ssh-agent.cmd"
...
"C:\Program Files\Git\usr\bin\ssh-add" ~/.ssh/myid.rsa
Works with Git 2.24.0, TortoiseGit 2.9.0, Windows 10 and no any Putty using.