How to deploy from CircleCI to Google Compute Engine? - ssh

I want to deploy CircleCi's artifacts to Google Compute Engine after CI test,
but I got stuck in SSH key issues.
When gcloud compute copy-files ran, gcloud will ask for ssh key generating during CircleCI's deployment process, which I don't know how to handle.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: The private SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
I've looked into document here and here, but none of them mentioned SSH related topics. So what SSH key should I use and how can I pass it to gcloud securely?
[UPDATED]
Here's my commands in the deployment step.
- echo $GCLOUD_SERVICE_KEY | base64 --decode --ignore-garbage > ${HOME}/gcloud-service-key.json
- sudo /opt/google-cloud-sdk/bin/gcloud --quiet components update
- sudo /opt/google-cloud-sdk/bin/gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json
- sudo /opt/google-cloud-sdk/bin/gcloud config set project $GCLOUD_PROJECT_ID
- sudo /opt/google-cloud-sdk/bin/gcloud compute copy-files ./dist instance-1:/tmp/ --zone $GCLOUD_INSTANCE_ZONE

First error:
Enter passphrase (empty for no passphrase):
You have to add the --quiet flag.
gcloud compute copy-files
This command ensures that the user's public SSH key is present in the project's metadata. If the user does not have a public SSH key, one is generated using ssh-keygen(1) (if the --quiet flag is given, the generated key will have an empty passphrase).
https://cloud.google.com/sdk/gcloud/reference/compute/copy-files
Second error:
Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
add a user before the instance name 338234243-compute#instance-1:/tmp
remove the last / of the instance path instance-1:/tmp
Check the owner of the tmp directory (user you choose should have a R&W right)
It seems that the user by default of my generated ssh key was root, and GCC sshd is not allowing root connection...

I had the same issue, and upon tinkering more and reading the google cloud docs, I realized I needed to generate an ssh key.
I use CircleCI (this also works with TravisCI) and this was the command I added to my circle.yml file to generate the ssh key:
- ssh-keygen -q -N "" -f ~/.ssh/google_compute_engine

Related

Copying an SSH key to the server even though you are already using SSH

I have already copied an SSH key to a server with ssh-copy-id -i ~/.ssh/skynet_key.pub.
Now I am building the gitlab-ci pipeline and have generated an new SSH key pair on my computer. I saved the private key as a variable in my gitlab project account.
Now I want to upload the new public key to the server.
My question is can I do this again from my local computer with ssh-copy-id?
Because I already have an SSH key pair distributed on both systems.
Would there be any problems here?
If so, how can I transfer the public key to the server?
Manually add the pubKey to authorized_keys?
As mentioned here, you can use the -o option (passed to SSH) to specify an existing key, in order to copy your second key:
ssh-copy-id -i ~/.ssh/<your-new-id-to-install> \
-o 'IdentityFile ~/.ssh/<your-already-existing-id>' \
<servername>`

Clone with SSH Issues

I am trying to clone a repository on git.dscoe.org
This is the situation:
-Created key using keygen in terminal
-Added key to Gitlab
Getting this error:
git#git.dscoe.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I've tried changing the ownership of the .ssh file, and the result from ls -al is attached in the image.
This is how I have my config file
Debugging output
Any help would be so amazing! Thank you so much!!
Make sure you have added:
the public key content (as one line) the the GitLab hosted on git.dscoe.org (and not to your gitlab.com account)
set the right permissions to ~/.ssh, keys and config
add the key to ssh-agent if it is protected by a passphrase
or try and generate a new one with:
ssh-keygen -t rsa -m PEM -P ""

Automate SSH key pair creation/generation for another user

i am trying to automate the process to order a linux instance and also handle the ssh keys on the instance-level.
Is it possible to generate a ssh key file for another user with the gcloud command line (without ssh'ing to it, that auto generate keys).
For Windows instances it looks like this:
I automate the instance creation
I automate generating windows password for windows instances
I email the newly generated password to the requesting user
For Linux:
I have automated linux instance creation
But what do i do next to generate a ssh key for another specific username so that i can attach in the email to the requesting user. The user does not have access to the GCE dashboard.
With AWS its simple because then i create the keys before the instance and can attach those, but dont know how to solve this automation issue with GCE.
Help?!
Thanks
Take a look at the instructions for "Adding and Removing SSH Keys", summarized here:
$ # Creating a new SSH key-pair with the correct format (`USERNAME` is your Google username
$ ssh-keygen -t rsa -f ~/.ssh/[KEY_FILE_NAME] -C [USERNAME]
$ # Edit the file. It should look like the following line:
$ # [USERNAME]:ssh-rsa [KEY_VALUE] [USERNAME]
$ vim ~/.ssh/[KEY_FILE_NAME]
$ # Get the existing metadata for the instance:
$ gcloud compute instances describe [INSTANCE]
$ # Look for the "metadata" -> "ssh-keys" entry and merge your new SSH key in.
$ vim all_keys.txt # This is where the merged key list goes
$ gcloud compute instances add-metadata [INSTANCE_NAME] \
--metadata-from-file ssh-keys=all_keys.txt
The link contains advanced instructions for adding an expiration time, adding the key to the entire project, blocking project-wide keys from working on an instance, using the Cloud Console instead of gcloud, doing this on Windows, etc.
That said, I'd urge you to use caution when emailing SSH private keys around.

How to specify a different location for ssh keys loading during rhc setup?

I am using rhc cli tool for OpenShift projects. I have encountered a problem with default rhc ssh key.
On any ssh related action (setup, app-create, etc..) rhc creates ~/.ssh/id_rsa key if it does not exist. I do not like that behaviour, and I would like it to use something like ~/.ssh/OpenShift-SSH-Keys/my_id_rsa.
Because during rhc setup, it did not ask me from which location I wanted to load the keys. Thus I also looked in ~/.openshift/express.conf and I only saw the configurations for ssl; not ssh.
So I found on the internet this following configuration line to add to ~/.openshift/express.conf:
ssh_key_file='~/.ssh/OpenShift-SSH-Keys/my_id_rsa'
I added it and also modified my ~/.ssh/config file:
# Openshift *.rhcloud.com ssh-key config
Host *.rhcloud.com
IdentityFile ~/.ssh/OpenShift-SSH-Keys/my_id_rsa
IdentitiesOnly yes
VerifyHostKeyDNS yes
StrictHostKeyChecking no
PasswordAuthentication no
UserKnownHostsFile ~/.ssh/known_hosts
To finish I setup my account like that:
rhc setup --config ~/.openshift/express.conf -l myusername#gmail.com
Output of this command line:
OpenShift Client Tools (RHC) Setup Wizard
This wizard will help you upload your SSH keys, set your application namespace, and check that other programs like Git are
properly installed.
If you have your own OpenShift server, you can specify it now. Just hit enter to use the server for OpenShift Online:
openshift.redhat.com.
Enter the server hostname: |openshift.redhat.com|
You can add more servers later using 'rhc server'.
Using myusername#gmail.com to login to openshift.redhat.com
RSA 1024 bit CA certificates are loaded due to old openssl compatibility
Password: ************************
OpenShift can create and store a token on disk which allows to you to access the server without using your password. The
key is stored in your home directory and should be kept secret. You can delete the key at any time by running 'rhc
logout'.
Generate a token now? (yes|no) yes
Generating an authorization token for this client ... RSA 1024 bit CA certificates are loaded due to old openssl compatibility
lasts 29 days
Saving configuration to /Users/theuser/.openshift/express.conf ... done
No SSH keys were found. We will generate a pair of keys for you.
Created: /Users/theuser/.ssh/id_rsa.pub
Your public SSH key must be uploaded to the OpenShift server to access code. Upload now? (yes|no) no
You can upload your public SSH key at a later time using the 'rhc sshkey' command
Checking for git ... found git version 2.5.0
Checking common problems .. done
Checking for a domain ... mydomainz1955
Checking for applications ... found 1
myapp http://myapp-mydomainz1955.rhcloud.com/
You are using 2 of 3 total gears
The following gear sizes are available to you: small
Your client tools are now configured.
As you can see in the output of the command line: No SSH keys were found. We will generate a pair of keys for you., although I specified in the ~/.openshift/express.conf that I already had ssh keys generated, rhc setup did not take them in consideration or did not find them.
So according to you guys, is it possible to somehow specify a different location for ssh keys loading during rhc setup?
Note: I know how to add additional ssh key, but I would like to stop rhc creating/using ~/.ssh/id_rsa
As far as I see you just want rhc to not use your default ssh key. So here is how you create a separate key and configure rhc to use it instead of the default one.
Key points are that:
you select no to generating and uploading ssh key during rhc
setup
you add your key separately with rhc sshkey add
you configure ssh to use the different key for that domain as you
list in your original example
Does this cover your concerns?
[crackit#koTapaH ~]$ mkdir /home/crackit/my_key_location
[crackit#koTapaH ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/crackit/.ssh/id_rsa): /home/crackit/my_key_location/key.rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/crackit/my_key_location/key.rsa.
Your public key has been saved in /home/crackit/my_key_location/key.rsa.pub.
The key fingerprint is:
c5:20:15:fb:17:96:86:8f:88:28:18:17:2a:b8:eb:51 crackit#koTapaH
The key's randomart image is:
+--[ RSA 2048]----+
| . ..+. |
|.. . . + . . |
|= . . + = |
|.= . . + = . |
|o .E. . S o o |
| ... . |
|.. |
|. . |
| . |
+-----------------+
[crackit#koTapaH ~]$ rhc setup
OpenShift Client Tools (RHC) Setup Wizard
This wizard will help you upload your SSH keys, set your application namespace,
and check that other programs like Git are properly installed.
If you have your own OpenShift server, you can specify it now. Just hit enter to
use the server for OpenShift Online: openshift.redhat.com.
Enter the server hostname: |openshift.redhat.com|
You can add more servers later using 'rhc server'.
Login to openshift.redhat.com:
Login to openshift.redhat.com: asdfgg#example.com
Password: *************
OpenShift can create and store a token on disk which allows to you to access the
server without using your password. The key is stored in your home directory and
should be kept secret. You can delete the key at any time by running 'rhc
logout'.
Generate a token now? (yes|no) yes
Generating an authorization token for this client ... lasts about 1 month
Saving configuration to /home/crackit/.openshift/express.conf ... done
No SSH keys were found. We will generate a pair of keys for you.
Created: /home/crackit/.ssh/id_rsa.pub
Your public SSH key must be uploaded to the OpenShift server to access code.
Upload now? (yes|no)
no
You can upload your public SSH key at a later time using the 'rhc sshkey'
command
Checking for git ... found git version 2.1.0
Checking common problems .. done
Checking for a domain ... foobar
Checking for applications ... found 2
jenkins http://jenkins-foobar.rhcloud.com/
tmp http://tmp-foobar.rhcloud.com/
You are using 2 of 3 total gears
The following gear sizes are available to you: small, medium
Your client tools are now configured.
[crackit#koTapaH ~]$ rhc sshkey add mykey my_key_location/key.rsa.pub
RESULT:
SSH key my_key_location/key.rsa.pub has been added as 'mykey'
[crackit#koTapaH ~]$ vi .ssh/config
<.. do your modifications here ..>
[crackit#koTapaH ~]$ rhc ssh tmp
Connecting to 550000a0e0b8cdca4c000040#tmp-foobar.rhcloud.com ...
*********************************************************************
You are accessing a service that is for use only by authorized users.
If you do not have authorization, discontinue use at once.
Any use of the services is subject to the applicable terms of the
agreement which can be found at:
https://www.openshift.com/legal
*********************************************************************
Welcome to OpenShift shell
This shell will assist you in managing OpenShift applications.
!!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
Shell access is quite powerful and it is possible for you to
accidentally damage your application. Proceed with care!
If worse comes to worst, destroy your application with "rhc app delete"
and recreate it
!!! IMPORTANT !!! IMPORTANT !!! IMPORTANT !!!
Type "help" for more info.
[tmp-foobar.rhcloud.com 550000a0e0b8cdca4c000040]\> exit
exit
Connection to tmp-foobar.rhcloud.com closed.
[crackit#koTapaH ~]$
Update: I didn't notice keys are generated. But I am sure that the generated keys during rhc setup are not actually used. First because the keys from default location are never added to openshift. And you can see a quick proof below. Another way to see is rhc sshkeys list.
Another thing is that if you already have keys in default location, then no keys are generated (in which case you still select no to not upload them). But it is actually a minor bug IMO in rhc that ssh keys are generated without asking the user. It might be a very rare use case - you don't have default key and you want to use a key from non-standard location (this is not your use case where you have a key in standard location, just don't want to use it) but still IMO one shouldn't generate something user did not request. So here's how I show you that only my desired custom key is used:
[crackit#koTapaH ~]$ rm -rf .ssh/id_rsa*
[crackit#koTapaH ~]$ rhc ssh tmp
Connecting to 550000a0e0b8cdca4c000040#tmp-foobar.rhcloud.com ...
<...>
Type "help" for more info.
[tmp-foobar.rhcloud.com 550000a0e0b8cdca4c000040]\> exit
exit
Connection to tmp-foobar.rhcloud.com closed.
[crackit#koTapaH ~]$ ls .ssh/
config known_hosts
[crackit#koTapaH ~]$
Update 2 Of course token cannot help you with ssh:
[crackit#koTapaH ~]$ rm -rf my_key_location
[crackit#koTapaH ~]$ rhc ssh tmp
Connecting to 550000a0e0b8cdca4c000040#tmp-foobar.rhcloud.com ...
no such identity: /home/crackit/my_key_location/key.rsa: No such file or directory
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
SSH key is used for ssh. Token is used for API requests. These are different use cases. rhc uses the ssh executable underneath so using a custom key means to edit ~/.ssh/config to set your default key to another location or set different keys for different hosts. This is not handled well by rhc setup. But once you have your key set, you don't have to run rhc setup anymore.

How to resolve Permission denied (publickey,gssapi-keyex,gssapi-with-mic)?

This question may have been asked before but I don't understand the concept. Can you please help me here?
Weird issue from this morning .. see i just push my file to google cloud computing then showing below error.. I don't know where to look that error.
ri#ri-desktop:~$ gcloud compute --project "project" ssh --zone "europe-west1-b" "instance"
Warning: Permanently added '192.xx.xx.xx' (ECDSA) to the list of known hosts.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
This occurs when your compute instance has PermitRootLogin no in it's SSHD config and you try to login as root. You can change the login user by adding username# before the instance name. Here is a complete example:
gcloud compute instances create my-demo-compute \
--zone us-central1-f \
--machine-type f1-micro \
--image-project debian-cloud \
--image-family debian-8 \
--boot-disk-size=10GB
gcloud --quiet compute ssh user#hostname --zone us-central1-f
In the example above, gcloud will set the correct credentials and will make sure you login. You can add the --quiet to ignore the ssh-password question.
One possible cause is that someone else in your project set the per-instance metadata for sshKeys (which overrides the project-wide metadata). When you run gcloud compute instances describe your-instance-name do you see a key called sshKeys in the metadata items?
It would also be helpful to see the contents of the latest log in ~/.config/gcloud/logs/. However, please make sure to scrub it of sensitive information.
I have a MacBook after facing with same problem, I re-created my SSH key in this format and works fine.
Generate your key with:
ssh-keygen -t rsa -C your_username
Copy the key and paste the ssh key under compute Engine metadata:
cat ~/.ssh/id_rsa.pub
It should work fine