CloverETL : How to connect to a server using ssh key - ssh-keys

I have a clover job that used to connect and transfer files from sftp using File URL = sftp://${SFTP_USERNAME}:${SFTP_PASSWORD}#${SFTP_SERVER}/${SFTP_SOURCE_PATH}${SOURCE_FILE_MASK}, but now I need to change the approach and connect to the location using a ssh key. How the parameters should be passed?

You need to put ssh key into specific folder 'ssh-keys' into your project (key need to have '.key' suffix and be in OpenSSH format). Then you will omit password parameter (eg. :${SFTP_PASSWORD} portion).
You can have multiple keys in the same folder, Clover loops through all of them and tries to connect.
Here is a link to forum where they are solving it https://forum.cloverdx.com/viewtopic.php?t=6799

Related

Verifying SSH host key in Tectia and WinSCP SFTP clients

I have two scripts to upload files into SFTP server:
sftp2 --identity-key-file=D:\test_ssh2 -B d:\test.txt testUser#xxx.xxx.xxx.xxx
open sftp://testUser#xxx.xxx.xxx.xxx -privatekey=D:\test.ppk -hostkey="ssh-rsa 1536 xxxxxxxxxxxxxxxxxxxxxxxxxxx"
Please help to explain:
What is the difference between the two scripts?
Why the first script doesn't need any host key, while the second script needs it?
Can I call SFTP with private key without using any hostkey?
What is the difference between the two scripts?
The first is batch file using Tectia sftp2.exe client.
The second is WinSCP script file.
Why the first script doesn't need any host key, while the second script needs it?
Most SFTP/SSH clients will prompt you to verify host key fingerprint on the first connect. If you verify the host key, it gets cached and you won't get prompted the next time.
Tectia sftp2.exe and WinSCP are no different.
If your Tectia sftp2.exe script works as it is, you must have verified the host key in this client before manually.
While your WinSCP script contains an explicit fingerprint of the host key. So the script will work, even if you have not verified the host key manually with WinSCP before running the script. This is recommended use of WinSCP, as it makes your script portable (not dependent on a hidden cache of host keys). But if you prefer, you can verify the host key manually with WinSCP, the same way you have done with Tectia.
Can I call SFTP with private key without using any hostkey?
Private key has nothing to do with host key. So this question makes little sense. See WinSCP article on SSH key pairs to understand.

Ansible: Change SSH key

I have an inventory of multiple servers. SSH access to these servers is secured using PEM key files. I would like to periodically change the PEM key used by my servers. So, I would like to use Ansible to do the following:
Generate a new PEM key file
For each server in my inventory, connect to the server using old PEM key file
Install new PEM key file
Test to ensure SSH with new key works and old key does not work
What is the best way to do this via Ansible?
You should split this in three playbooks.
The first to generate a new PEM key. This will run locally. See: https://docs.ansible.com/ansible/playbooks_delegation.html#local-playbooks
The second one will do the rollout. So it copies the key to all servers. You can use authorized_key or copy depending on what your preferred workflow is. But thats another question.
The third step then would be a testing playbook, maybe with an assert statement or just using ping to ensure the connection works.
When you have all this playbooks combine them in a single with include or add this three plays in one playbook in the right order. See: https://docs.ansible.com/ansible/playbooks_intro.html

SSH to Amazon EC2 instance using PuTTY in Windows

I am a newbie to Amazon web services, was trying to launch an Amazon instance and SSH to it using putty from windows. These are the steps I followed:
Created a key pair.
Added a security group rule for SSH and HTTP.
Launched and instance of EC2 using the above key pair and security group.
Using PuTTYgen converted the *.pem file to *.ppk
Using putty tried connecting to the public DNS of the instance and provided the *.ppk file.
I logged in using 'root' and 'ec2-user', and created the PPK file using SSH1 and SSH2, for all these attempts I get the following error in putty,
"Server refused our key"
Can you guys please help, any suggestions would be greatly appreciated.
I assume that the OP figured this out or otherwise moved on, but the answer is to use ubuntu as the user (if the server is ubuntu).
1) Make sure you have port 22 (SSH) opened in Security Group of EC2 Instance.
2) Try connecting with Elastic IP instead of public DNS name.
I hope you have followed these steps Connecting EC2 from a Windows Machine Using PuTTY
Another situation where I got the "Server refused our key" error when using putty, from windows, to ssh to an EC2 instance running ubuntu:
The private key was wrongly converted from .pem to .ppk.
puttygen has two options for "converting keys".
Load your .pem file into puttygen using the File->Load Private Key option and then save as .ppk file using the Save Private Key Button.
DO NOT use the menu option Conversions->Import Key to load the .pem file generated by EC2.
See the puttygen screenshots below, with the two menu options marked.
Check the username, it should be "ubuntu" for your machine.
Check if traffic is enabled on port 22 in Security group.
Check if you are using the correct url i.e ubuntu#public/elasticip
Maybe worth of checking one more thing. Go to AWS console, right mouse click on the instance and choose "Connect...". It will show you the DNS name that you want to use. If you restarted that instance at some point, that DNS name could have changed.
I had a similar problem when I tried to connect an instance created automatically by the Elastic Beanstalk service (EBS). But, once I linked my existing key name to the EBS (under Environment Details -> Edit Configuration -> Server Tab -> Existing Key Pair), I was able to login with 'ec2-user' and my existing key file (converted to .ppk) with putty.
This, however, terminates the running instance and rebuilds a new instance with access through the key pair named above.
Just in case it helps anyone else, I encountered this error after changing the permissions on the home folder within my instance. I was testing something and had executed chmod -R 777 on my home folder. As soon as this had occurred, once I had logged out I was effectively locked out.
You won't face this error if you SSH AWS directly using ".pem" file instead of converted ".ppk" file.
1) Use Git Bash instead of putty. Since you can run all the Linux commands in Git Bash. By installing Git you get to access Git Bash Terminal
2) Right click from the folder where you have ".pem" and select "Git Bash Here".
3) Your key must not be publicly viewable for SSH to work. So run "chmod 400 pemfile.pem".
4) Connect to your instance using its Public DNS - "ssh -i "pemfile.pem" ec2-user#ec2-x-x-x-x.us-west-1.compute.amazonaws.com"
5) Make sure to whitelist your Network IP for SSH in your_instance->security_group->inbound_rules
I assume you're following this guide, and connecting using the instructions on the subsequent page. Verify a couple of things:
You converted the key correctly, e.g. selected the right .pem file, saved as private key, 1024-bit SSH-2 RSA
The Auth settings (step 4 in the connection tutorial) are correct
I was having the same trouble (and took the same steps) until I changed the user name to 'admin' for the debian AMI I was using.
You should lookup the user name ofthe AMI you are using. The debian AMI is documented here
http://wiki.debian.org/Cloud/AmazonEC2Image/Squeeze
I have had this same problem. The AMI you are using is the one that is also used by the "Cloud Formation" templating solution.
In the end I gave up with that, and created a Red Hat instance. I was then able to connect by SSH fine using the user root.
The instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html work fine using a Red Hat instance but not using an Amazon Linux instance. I assume they have some username that I didn't think to try (root, ec2-user, and many other obvious ones, all were refused)
Hope that helps someone!
I use Debain AMI and I try ec2-user, root but correct login is 'admin'.
I was getting the same error when I tried to create a new key pair and tried to use that new pem/ppk file. I noticed that the Key Pair Name field on the instance was still the old one and in poking around. Apparently, you can't change a key pair. So I went back to the original key pair. Fortunately, I didn't delete anything so this was easy enough.
Try an alternative SSH client, like Poderosa. It accepts pem files, so you will not need to convert the key file.
If you already have a key pair, follow these steps:
Convert *.pem to *.ppk using PuTTYgen (Load pem file key then Save ppk)
Add ppk auth key file to Putty SSH>Auth options
Enter "Host Name (or IP address)" field: ubuntu#your-ip-address-of-ubuntu-ec2-host))

SSH basics - do you use a new key for each server you're accessing?

I couldn't find any basic info for designers (on a mac) for how SSH keys work - so thought I'd ask them here.
If I want to connect my work workstation to:
Github
A DEV server
A LIVE server
Do I generate one ssh key on the workstation and add it to all those servers or do I generate multiple keys - one for each server?
Once I've generated a key (or keys), do I copy it into the id_rsa file in my user account on that server (I realize I may have to create the id_rsa file)?
And if I now want to access the same server but from my home laptop, do I add the laptop's ssh key to the same id_rsa file on the server or do I create a new file?
If I need to create a new file, does it matter what the file is called - laptop_rsa?
I basically want to disable root login on my servers but I don't really understand how SSH applies to multiple machines and multiple servers.
Any help or pointers in the right direction would be much appreciated.
Cheers
You only need one key for the local machine that you are connecting
to all three servers.
For the DEV server and the LIVE server, you can add the contents of
your id_rsa.pub file to the
authorized_keys file on each of the target servers.
This file will be in the ~/.ssh directory. You will
need to create the file if it's not there (touch
~/.ssh/authorized_keys). Adding your public key to this file
will let you login with your passphrase rather than a password.
Place all authorized keys (i.e. your laptops id_rsa.pub) in the same
authorized_keys file on the target server.
Adding your keys to authorized_keys doesn't affect root login (that is a separate setting), however, it will prevent people from attempting to brute-force your password if you then turn off password login.

How to get Hudson CI to check out CVS projects over SSH?

I have my Hudson CI server setup. I have a CVS repo that I can only checkout stuff via ssh. But I see no way to convince Hudson to check out via ssh. I tried all sorts of options when supplying my connection string.
Has anyone done this? I gotta think it has been done.
If I still remember CVS, I thought you have to set CVS_RSH environment variable to ssh. I suspect you need to set this so that your Tomcat process gets this value inherited.
You can check Hudson system information to see exactly what environment variables the JVM is seeing (and passes along to the build.)
I wrote up an article that tackles this you can find it here:
http://www.openscope.net/2011/01/03/configure-ssh-authorized-keys-for-cvs-access/
Essentially you want to set up passphraseless ssh keys for your build user. This will allow authentication to occur without the need to work out some kind of way to key in your password.
<edit> i.e. Essentially the standard .ssh key client & server install/exchange.
http://en.wikipedia.org/wiki/Secure_Shell#Key_management
for the jenkins user account:
install user key (public & private part) in ~/.ssh (generate it fresh or use existing user key)
on cvs server:
install user key (public part) in ~/.ssh
add to authorized_keys
back on jenkins user account:
access cvs from command-line as jenkins user and accept remote host key (to known_hosts)
* note any time remote server changes key/ip you will need to manually access cvs and accept key again *
</edit>
There's another way to do it but you have to manually log from the build machine to your cvs server and keep the ssh session open so hudson/jenkins can piggyback the connection. Seemed kinda pointless to me though since you want your CI server to be as hands off as possible.