I'm testing out Ansible and am already stuck on a fairly simple thing. I configured my /etc/ansible/hosts to contain the IP of my server:
[web]
1.2.3.4
Now, connecting to it with ansible all -vvvv -m ping fails since my ~/.ssh/config for the specified server uses a custom port, a key file not in the default location, etc. How do I tell Ansible to reuse my SSH configuration for the connection?
It's a little esoteric, so it's understandable you missed it. This is from the Ansible Inventory page:
If you have hosts that run on non-standard SSH ports you can put the port number after the hostname with a colon. Ports listed in your SSH config file won’t be used with the paramiko connection but will be used with the openssh connection.
To make things explicit, it is suggested that you set them if things are not running on the default port
So, in your case:
[web]
1.2.3.4:9000
(Using 9000 as your alt port, of course)
Ansible uses paramiko on systems with a dated version of ssh, such as CentOS/RHEL.
What tedder42 said plus there are other slightly more advanced ways of defining your ssh config on a per host basis.
[web]
1.2.3.4 ansible_ssh_port=9000
This only makes sense if you're also using the other ansible_ssh special variables like ansible_ssh_user and ansible_ssh_host.
ansible_ssh_host is helpful if you want to be able to refer to a server by a name of your choosing instead of its IP.
[web]
apache ansible_ssh_host=1.2.3.4 ansible_ssh_port=9000
If you end up with multiple hosts with the same alternative ssh port you can makes use of Ansible's group variable function.
[web]
1.2.3.4
5.6.7.8
9.10.11.12
[web:vars]
ansible_ssh_port=9000
Now Ansible will use port 9000 on all three of the hosts In the web group.
Understanding how to organize your inventory data goes a long way to your success with Ansible.
Related
I have PyCharm pro and I need to set up a remote repo on it via ssh. There are instructions available for doing this but my requirements are slightly different than what I have seen available. My organisation has ssh set up via cloudfare and all those configs are already put in .ssh/config. It includes hostname, IdentityFile, username, and so on. If I need to access my remote machine, I just do ssh <HOST-NAME> from the terminal. My config looks like this -
Host <HOST-NAME>
HostName <HOST-NAME>.<ORGANISATION>.net
ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
IdentityFile ~/.ssh/id_rsa
User <USER-NAME>
The way to set up PyCharm usually includes instructions like specifying host, port, username etc so that PyCharm can essentially run
ssh username#host_ip_address. However, I just need it to run ssh <HOST-NAME> so that it can access everything else using the config file that is already set up.
Is there a way to make this happen?
I use Pycharm Pro with my own .ssh/config and Jump Proxy pseudo hostname and certificates so slightly different. I "ssh pseudo-hostname" so no user as well.
In Pycharm I set a valid username and port (default 22) cause in my case it will get passed on.
Your setup ignore username and port no? If so you can use BS values?
I have a problem with the configuration of a jumper server in ssh_config.
So far I was able to configure the connection for a server as follows:
Host myhost*
ProxyCommand ssh -q myjumpserver nc %h %p
So I could reach via ssh myhost1 or ssh myhost2 directly my two servers.
Now unfortunately something has changed and I should call my servers with the following syntax:
ssh user#newjump#myhost1
(yes 2x # is correct and only this notation works)
Newjump is not a jumpserver in the real sense, so I can't use the syntax above.
Does anyone have an idea how I can configure this?
Since I have >30 servers and these groupwise always have the same naming scheme, it would be nice if I could generalize it as above. But in case of doubt I would also implement a solution for each individual server.
What is important to me is that I only have to type ssh myhost1 to reach this host. A solution via alias is out of the question, because then I would have to adapt numerous scripts.
I solved it by
Host name1
Hostname newjump
User user#host1
I have enabled two factor authentication for ssh using duosecurity (using this playbook https://github.com/CoffeeAndCode/ansible-duo ).
How can I use ansible to manage the server now. The SSH calls fail at gathering facts because of this. I want the person running the playbook to enter the two factor code before the playbook is run.
Disabling two factor for the deployment user is a possible solution but creates a security issue which I would I like to avoid.
It's a hack, but you can tunnel a non-2fac Ansible SSH connection through a 2fac-enabled SSH connection.
Overview
We will setup two users: ansible will be the user Ansible will use. It should be authenticated in a way that's supported by Ansible (i.e., not 2fac). This user will be restricted so it cannot connect from anywhere but 127.0.0.1, so it is not accessible from outside the machine.
The second user, ansible_tunnel will be open to the outside world, but will be authenticated by two factors, and will only allow tunneling of SSH connections to the local machine.
You must be able to configure 2-factor authentication only for some users (not all).
Some info on SSH tunnels.
On the target machine:
Create two users: ansible and ansible_tunnel
Put your public key in ~/.ssh/authorized_keys of both users
Set the shell of ansible_tunnel to /bin/false, or lock the user - it will be used for tunneling exclusively, not running commands
Add the following to /etc/ssh/sshd_config:
AllowTcpForwarding no
AllowUsers ansible#127.0.0.1 ansible_tunnel
Match User ansible_tunnel
AllowTcpForwarding yes
PermitOpen 127.0.0.1:22
ForceCommand echo 'This account can only be used for tunneling SSH sessions'
Setup 2-factor authentication only for ansible_tunnel
Restart sshd
On the machine running Ansible:
Before running Ansible, run the following (on the Ansible machine, not the target):
ssh -N -L 8022:127.0.0.1:22 ansible_tunnel#<host>
You will be authenticated using two factors.
Once the tunnel is up (check with netstat), run Ansible with ansible_ssh_user=ansible, ansible_ssh_port=8022 and ansible_ssh_host=localhost.
Recap
Only ansible_tunnel can connect from the outside, and it will be authenticated using two factors
Once the tunnel is set up, connecting to port 8022 on the local machine is the same as connecting to sshd on the remote machine
We're allowing ansible to connect over SSH only when it is done through the localhost, so only connections that are tunneled are allowed
Scale
This will not scale well for multiple server, due to the need to open a separate tunnel for each machine, which requires manual action. However, if you've chosen 2-factor authentication for your servers you're already willing to do some manual action to connect to each server, and this solution will only add a little overhead with some script-wrapping.
[EDITED TO ADD]
Bonus
For convenience, we may want to log into the maintenance account directly to do some manual work, without going through the process of setting up a tunnel. We can configure SSH to require 2fac authentication in this case, while maintaining the ability to connect without 2fac through the tunnel:
# All users must authenticate using two factors
AuthenticationMethods publickey,keyboard-interactive
# Allow both maintenance user and tunnel user with no restrictions
AllowUsers ansible ansible_tunnel
# The maintenance user is allowed to authenticate using a single factor only
# when connecting from a local address - it should be impossible to connect to
# this user using a single factor from the outside (the only way to do that is
# having an existing access to the machine, or use the two-factor tunnel)
Match User ansible Address 127.0.0.1
AuthenticationMethods publickey
I can use ansible with ssh and 2FA using the ControlMaster feature of ssh and ansible.
My local ssh client is configured to dump a ControlPath socket for multiplexing connection. Ansible is configured to use the same socket.
Local ssh client
This configuration enable multiplexing for all connections. I personally store this configuration in `~/.ssh/config:
Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r#%h:%p.socket
ControlPersist 1m
When a connection is established, a socket appears in the $HOME/.ssh directory. This socket persists during one minute after disconnection.
Configure ansible
Ansible is configured to re-use the local socket.
Add this in your ansible configuration file (for instance, ~/.ansible.cfg):
[ssh_connection]
control_path=~/.ssh/master-%%r#%%h:%%p.socket
Note the double % for variable substitution.
Usage
Connect to your server using ssh regular command (ssh user#server), and perform 2FA;
Launch your ansible command as usual.
The step 2 must be performed within the ControlPersist configuration, or keep an ssh connection in a terminal when you launch ansible command in another one.
You can also force to close connection when you do not need it, using: ssh -O exit user#server.
Note that, if you open a third terminal and run ssh user#server, you will not be asked for credentials: the connection established in 1. will be re-used.
Drawbacks
In case of bad network conditions
Sometimes, when you loose connection, the socket persists. Every further connection hangs. You must manually disconnect this connection, using ssh -O exit user#server. This is the only known drawback for this method.
References:
Ansible parameter ANSIBLE_SSH_CONTROL_PATH
About multiplexing ssh (a very old blog post which makes me discover ssh multiplexing: https://blog.scottlowe.org/2015/12/11/using-ssh-multiplexing/)
Solution using a Bastion Host
Even using an ssh bastion host it took me quite a while to get this working. In case it helps anyone else, here's what I came up with. It uses the ControlMaster ssh config options and since ansible uses regular ssh it can be configured to use the same ssh features and re-use the connection to the bastion host regardless of how many connections it opens to remote hosts. I've seen these Control options recommended in general (presumably for performance reasons if you have a lot of hosts) but not in the context of 2FA to a bastion host.
With this approach you don't need any sshd config changes, so you'll want AuthenticationMethods publickey,keyboard-interactive as the only authentication method setting on the bastion server, and publickey only for all your other servers that you're proxying through the bastion to get to. Since the bastion host is the only one that accepts external connections from the internet, it's the only one that requires 2FA, and internal hosts rely on agent forwarding for public key authentication but don't use 2FA.
On the client, I created a new ssh config file for my ansible environment in the top-level directory that I run ansible from (so sibling of ansible.cfg) called ssh.config. It contains:
Host bastion-persistent-connection
HostName <bastion host>
ForwardAgent yes
IdentityFile ~/.ssh/my-key
ControlMaster auto
ControlPath ~/.ssh/ansible-%r#%h:%p
ControlPersist 10m
Host 10.0.*.*
ProxyCommand ssh -W %h:%p bastion-persistent-connection -F ./ssh.config
IdentityFile ~/.ssh/my-key
Then in ansible.cfg I have:
[ssh_connection]
ssh_args = -F ./ssh.config
A few things to note:
My private subnet in this case is 10.0.0.0/16 which maps to the host wildcard option above. The bastion proxies all ssh connections to servers on this subnet.
This is a bit brittle in that I can only run my ssh or ansible commands in this directory, because of the ProxyCommand passing the local path to this config file. Unfortunately I don't think there's an ssh variable that maps to the current config file being used so that I could pass the same config file to the ProxyCommand automatically. Depending on your environment it might be better to use an absolute path for this.
The one gotcha is it makes running ansible more complex. Unfortunately, from what I can tell ansible has no support whatsoever for 2FA. So if you have no existing ssh connection to the bastion, ansible will print out Verification code: once for every private server it's connecting to, but it's not actually listening for the input so no matter what you do the connections will fail.
So I first run: ssh -F ssh.config bastion-persistent-connection
This creates the socket file in ~/.ssh/ansible-*, and the ssh agent locally will close & remove that socket after the configurable time (what I have set to 10m).
Once the socket is open I can run ansible commands like normal, e.g. ansible all -m ping and they succeed.
I know how to forward SOCKS proxy on the command like below
ssh -D port_number user#host
This works well but I want to be able to put that forwarding into my SSH config file. But I am not able to locate any useful information or tutorial about.
I have bunch of normal SSH profiles in the config so I prefer to have the forwardings attached to the SSH profiles.
Use the config setting "DynamicForward" Here is a quick example of what it should look like:
Host example.com
User username
DynamicForward 8080
If the DynamicForward option is only given a port number, then it will bind to localhost:port.
You can add a specific IP to get it to bind to an address other than the localhost. Using "*:8080" will bind the proxy to all IP addresses on the box.
To use an IPv6 address enclose the address in square brackets:
[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:8080
For details, please see the ssh_config man page (type man ssh_config).
I do not recommend use socat because it only support socks4
But you can use ncat
install ncat
add this in your ssh config file ProxyCommand ncat --proxy-type socks5 --proxy 127.0.0.1:1080 %h %p
You may need to check ncat options if it does not work.
This is how it is done:
Host server-fwd
Hostname a.b.c.d
User username
Port 22
LocalForward localhost:AAAA localhost:DD
LocalForward localhost:BBBB localhost:EEE
LocalForward localhost:CCCC localhost:FFFF
Change the "server-fwd" to whatever name you like, change "a.b.c.d" to the IP you're connecting to, change "username" to whatever your account is, maybe change the port number if necessary.
The LocalForward lines are the ones you have been looking for. The middle column (i.e. AAAA, BBBB and CCCC) are the ports on the system you are running the ssh command from. The right column (i.e. DD, EEE and FFFF) are the ports on the server you're connecting to. It's localhost in both cases because in the first case it's when the ssh command is run locally and in the second case it is relative to the server you just logged into.
Yes, I use this a lot. ;)
I am trying to setup an SSH tunnel to access Beanstalk (to bypass an annoying proxy server).
I can get this to work, but with one caveat: I have to map my Beanstalk host URL (username.svn.beanstalkapp.com) in my hosts file to 127.0.0.1 (and use the ip in place of the domain when setting up the tunnel).
The reason (I think) is that I am creating the tunnel using the local SSH instance (on Snow Leopard) and if I use localhost or 127.0.0.1 when talking to Beanstalk, it rejects the authorisation credentials. I believe this is because Beanstalk use the hostname specified in a request to determine which account the username / password combination should be checked against. If localhost is used, I think this information is missing (in some manner which Beanstalk requires) from the requests.
At the moment I dig the IP for username.svn.beanstalkapp.com, map username.svn.beanstalkapp.com to 127.0.0.1 in my hosts file, then for the tunnel I use the command:
ssh -L 8080:ip:443 -p 22 -l tom -N 127.0.0.1
I can tell Subversion that the repo. is located at:
https://username.svn.beanstalkapp.com:8080/repo-name
This uses my tunnel and the username and password are accepted.
So, my question is if there is an option when setting up the SSH tunnel which would mean I wouldn't have to use my hosts file workaround?
I would add an entry to your hosts file that maps 127.0.0.1 to the hostname you need and then use the hostname to connect to your tunnel.
Update
The hosts file is IMO your best option.