How do I store keypair in s3? - amazon-s3

I referred this stackoverflow answer:
ssh key of newly created ec2 instance using boto
where if the keypair doesn't exist, it creates a new one and saves the private keypair.
However, I wish to save the keypair in s3 or some common place between mac and windows users. I know the directory hierarchy is different for both OS so how do I hard code a value that works for both?
My current code:
# Check to see if specified keypair already exists.
# If we get an InvalidKeyPair.NotFound error back from EC2,
# it means that it doesn't exist and we need to create it.
try:
key = ec2.get_all_key_pairs(keynames=[keypair_name])[0]
except ec2.ResponseError, e:
if e.code == 'InvalidKeyPair.NotFound':
print 'Creating keypair: %s' % keypair_name
# Create an SSH key to use when logging into instances.
key = ec2.create_key_pair(keypair_name)
key.save('https://s3.amazonaws.com/' + bucket_name)
else:
raise
The key.save line makes the url something like "https://s3.amazonaws.com/bucket_name\launch.pem"

The boto.ec2.keypair.KeyPair object's .save function saves the material (the unencrypted PEM encoded RSA private key) of a newly created KeyPair to a local file.
It is invalid syntax to use key.save('https://s3.amazonaws.com/' + bucket_name). Amazon S3 is not a local filesystem.
To store the file on S3, first download it to the local computer, then upload it via the S3 API calls.
It's worth mentioning that EC2 Key Pairs are provided for initial access to EC2 instances. Once you login to an instance, you should implement your own security for the instance, such as creating users and adding your own keypairs, or connecting to an LDAP or Active Directory server.

Related

Can't connect to SFTP (with private key file) in Copy Data Tool

I am trying to copy data from SFTP to blob but got stuck when creating SFTP source.
I have the connection details and can easily connect on Filezilla or WinSCP. However, I am unable to get it to work in Azure data factory.
I am not using code but the user interface.
The connection details on the page creating the SFTP source:
Connect via integration runtime: AutoResolveIntegrationRuntime (default)
Host: xyz
Port: 22 (can't remove it as it doesn't like it)
SSH Host Key Validation: Enable SSH Host Key Validation
SSH Host Key Finger-print: taken from WinSCP - Session - Server/protocol information
Authentication type: SSH Public Key Authentication -can't use basic as the private key holds the security info
User name:XXX
Private Key Type: Use Key Content
Private key content: loaded the .ppk file, tried also tried loading the .pem file and got different errors
Pass Phrase: none
When setting up this sftp in WinSCP or FileZilla it automatically converted the provided .pem file into .ppk.
When I loaded the .ppk file into ADF I got an error: Invalid Sftp credential provided for 'SshPublicKey' authentication type.
When I loaded the .pem file I got: Meet network issue when connect to Sftp server 'spiderftp.firstgroup.com', SocketErrorCode: 'TimedOut'.
I have also tried 'Disable SSH Host Key Validation' in SSH Host Key Validation and made no difference.
I have also opened the .ppk file in PuttyGen and used that host key finger print and still no luck.
Only getting these 2 errors depending on which file I load.
Can't find anything about this online so would be grateful for some advice.
Have you read this note in this doc?
https://learn.microsoft.com/en-us/azure/data-factory/connector-sftp#using-ssh-public-key-authentication
SFTP connector supports RSA/DSA OpenSSH key. Make sure your key file content starts with "-----BEGIN [RSA/DSA] PRIVATE KEY-----". If the private key file is a ppk-format file, please use Putty tool to convert from .ppk to OpenSSH format.
Got this working today. Like you, could connect using WinSCP and failed when using ADF.
The link Fang Liu shared contains our answers, but my issue was not the private key. I suspect Fang's suggestion resolved your problem and I'm sharing my answer here to help others who may encounter similar.
My issue:
When using Private Key Authentication in ADF the password becomes a Pass Phrase and you no longer have the ability to supply a password. To overcome the problem we disabled password authentication for the user and the SFTP connection started working.
As stated in the documentation. The Pass Phrase is used to decrypt the private key if it is encrypted.
Also worth noting:
If you store the contents of the private key in Key Vault you need
to base64 encode the entire contents of the exported key and use
that string. This includes "-----BEGIN RSA PRIVATE KEY-----" and the
end. The same applies if you want to paste the value into the
textbox of the SFTP linked service edit screen.
I did not try to manually edit the JSON of the Linked Service to explicitly provide a password and this could be workaround for someone to test if they are unable to disable the password.
I used PuTTYGen to export the PPK to a private key and had the same fingerprint issue too so I just disabled cert validation. Funnily you can use the fingerprint provided by the error and it passes validation so not sure where the bug lies. :-)

SSH suddenly asking for DSA

After allowing an application to install ssh keys on my local machine, I can no longer connect to my Digital Ocean shell. (The app is not related to DO; totally different.) I get a
Permission denied (public key)
error that, with -vvv, debugs to
Trying private key: /Users/macbook/.ssh/id_dsa
no such identity: path/.ssh/id_dsa: No such file or directory
My keys are rsa. I have no idea why the machine is asking for dsa.
My .ssh/config file (which I have never read until now) has only information about the application I allowed access. Maybe it was overwritten. If this is the case, could you tell me how to rewrite my .ssh/config file?
My keys on my cloud server (accessed through a DO gui shell) and on my machine still match up. The folder permissions are ok. I've been using this for months with no trouble until now.
Any suggestions?
Edit:
This was probably a result of the third party application overwriting my .ssh/config file. Because the writing of this file was always automated for me, I never took any notice of it. If you try:
regenerating new keys
appending the new public key to the authorized_keys file on your server on a new line
writing a new .ssh/config file as so:
-
Host 111.11.11.1
User bob
IdentityFile ~/.ssh/new_file.pem
then you might make it. Somehow now I can't get a passphrase to work on the new keys.

how to download ssh keypair from fi-lab cloud more than once?

I am starting with fi-lab Cloud at at http://lab.fi-ware.org, then I created security key pair before start to create an instance. Associating the floating ip address to an instance and using this key pair under a ssh session is the only way to have terminal access to the created instance:
1 - tab security > key pairs > create
2 - given a name to keypair "kpname"
3- resulting in: The keypair "kpname" should download automatically. If not use the link below.
Download Keypair
Well, the "automatically download" doesn't started on google chrome browser (windows 7 and mac os 10.8.5)
No problem, I clicked on the provided link (blob:https%3A//cloud.lab.fi-ware.org/850xxxsnipedxxxxecd4), get the "kpname.pem" file, changed the file mode permission (chmod 600) and used with ssh command.
The problem is when I need to use another computer and not keep the key file stored on my hands (pendrive, dropbox, email...). This way I cannot access my virtual machine, neither creating a new keypair from fi-lab cloud site, because there is no function to associate a new key pair in a created instance. The only place to associate the keypair is on the creation instance workflow. That is, creating a new instance.
I dont know where to click to download again the same keypair. If not exists I believe the platform could offer download keypair in the context menu (action) from each created key pair.
The only "action" from each keypair is to delete a selected keypair.
thanks for help.

How to change EC2 Public Key?

I'm trying to change my EC2 public key for myself, the rest of my team who uses AWS, and also for running instances.
So the old key was (for example) 'my_key', and I made a new key called 'my_key2'.
I made the new key, saved the .pem file alongside the old one, so both 'my_key.pem' and 'my_key2.pem' are residing in my ~/.ssh folder.
Now, I'm trying to ssh into other instances to change they authorization_keys to share my public key, but when I try to ssh, I get
amy$ ssh ***-**-**-***-***.foo.com
Permission denied (publickey).
I tried taking the new 'my_key2' file out of the .ssh folder, and the same problem occurs. I didn't have this problem before I started mucking around. Any tips on getting ssh to recognize my old certificates so that I can get everyone going on the new keys?
You will need to stop the instance (if EBS-backed instance) in order to change the key pair to be used for the instance. If not EBS-backed volume you will just need to start another instance from your AMI with the the new key pair.
If you are not able to easily restart the instance or create a new instance, there is this method which purports to being able to change public keys on a running instance, but I haven;t tried it. Really the right way to do it is by stopping/starting or creating new instances.
http://seabourneinc.com/2011/01/19/change-key-pairs-on-aws-ec2-instance/
http://seabourneinc.com/2011/01/19/change-key-pairs-on-aws-ec2-instance/
This solution works. I checked it.
In simple steps.
login to AWS console, create a new pem key, it will ask you to save it to local, save it.
run the following command.
ssh-keygen -y | tee my_key.pub
type the path to where you have saved the new pem key file along with the file name.
the above command will display the contents on the screen, copy it, and paste into the
authorized_keys file of the EC2 instance, save and close.
now you can use the saved/downloaded new pem key file to access the EC2 instance.

SSH to Amazon EC2 instance using PuTTY in Windows

I am a newbie to Amazon web services, was trying to launch an Amazon instance and SSH to it using putty from windows. These are the steps I followed:
Created a key pair.
Added a security group rule for SSH and HTTP.
Launched and instance of EC2 using the above key pair and security group.
Using PuTTYgen converted the *.pem file to *.ppk
Using putty tried connecting to the public DNS of the instance and provided the *.ppk file.
I logged in using 'root' and 'ec2-user', and created the PPK file using SSH1 and SSH2, for all these attempts I get the following error in putty,
"Server refused our key"
Can you guys please help, any suggestions would be greatly appreciated.
I assume that the OP figured this out or otherwise moved on, but the answer is to use ubuntu as the user (if the server is ubuntu).
1) Make sure you have port 22 (SSH) opened in Security Group of EC2 Instance.
2) Try connecting with Elastic IP instead of public DNS name.
I hope you have followed these steps Connecting EC2 from a Windows Machine Using PuTTY
Another situation where I got the "Server refused our key" error when using putty, from windows, to ssh to an EC2 instance running ubuntu:
The private key was wrongly converted from .pem to .ppk.
puttygen has two options for "converting keys".
Load your .pem file into puttygen using the File->Load Private Key option and then save as .ppk file using the Save Private Key Button.
DO NOT use the menu option Conversions->Import Key to load the .pem file generated by EC2.
See the puttygen screenshots below, with the two menu options marked.
Check the username, it should be "ubuntu" for your machine.
Check if traffic is enabled on port 22 in Security group.
Check if you are using the correct url i.e ubuntu#public/elasticip
Maybe worth of checking one more thing. Go to AWS console, right mouse click on the instance and choose "Connect...". It will show you the DNS name that you want to use. If you restarted that instance at some point, that DNS name could have changed.
I had a similar problem when I tried to connect an instance created automatically by the Elastic Beanstalk service (EBS). But, once I linked my existing key name to the EBS (under Environment Details -> Edit Configuration -> Server Tab -> Existing Key Pair), I was able to login with 'ec2-user' and my existing key file (converted to .ppk) with putty.
This, however, terminates the running instance and rebuilds a new instance with access through the key pair named above.
Just in case it helps anyone else, I encountered this error after changing the permissions on the home folder within my instance. I was testing something and had executed chmod -R 777 on my home folder. As soon as this had occurred, once I had logged out I was effectively locked out.
You won't face this error if you SSH AWS directly using ".pem" file instead of converted ".ppk" file.
1) Use Git Bash instead of putty. Since you can run all the Linux commands in Git Bash. By installing Git you get to access Git Bash Terminal
2) Right click from the folder where you have ".pem" and select "Git Bash Here".
3) Your key must not be publicly viewable for SSH to work. So run "chmod 400 pemfile.pem".
4) Connect to your instance using its Public DNS - "ssh -i "pemfile.pem" ec2-user#ec2-x-x-x-x.us-west-1.compute.amazonaws.com"
5) Make sure to whitelist your Network IP for SSH in your_instance->security_group->inbound_rules
I assume you're following this guide, and connecting using the instructions on the subsequent page. Verify a couple of things:
You converted the key correctly, e.g. selected the right .pem file, saved as private key, 1024-bit SSH-2 RSA
The Auth settings (step 4 in the connection tutorial) are correct
I was having the same trouble (and took the same steps) until I changed the user name to 'admin' for the debian AMI I was using.
You should lookup the user name ofthe AMI you are using. The debian AMI is documented here
http://wiki.debian.org/Cloud/AmazonEC2Image/Squeeze
I have had this same problem. The AMI you are using is the one that is also used by the "Cloud Formation" templating solution.
In the end I gave up with that, and created a Red Hat instance. I was then able to connect by SSH fine using the user root.
The instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html work fine using a Red Hat instance but not using an Amazon Linux instance. I assume they have some username that I didn't think to try (root, ec2-user, and many other obvious ones, all were refused)
Hope that helps someone!
I use Debain AMI and I try ec2-user, root but correct login is 'admin'.
I was getting the same error when I tried to create a new key pair and tried to use that new pem/ppk file. I noticed that the Key Pair Name field on the instance was still the old one and in poking around. Apparently, you can't change a key pair. So I went back to the original key pair. Fortunately, I didn't delete anything so this was easy enough.
Try an alternative SSH client, like Poderosa. It accepts pem files, so you will not need to convert the key file.
If you already have a key pair, follow these steps:
Convert *.pem to *.ppk using PuTTYgen (Load pem file key then Save ppk)
Add ppk auth key file to Putty SSH>Auth options
Enter "Host Name (or IP address)" field: ubuntu#your-ip-address-of-ubuntu-ec2-host))