Trying to test run playbook. Getting permission denied - ssh

I am trying to do a "dry-run" of a playbook. The machine I am targeting I am able to ssh into and vice versa. When I run the ansible all -m ping -vvv this is the output.
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/core/system/ping.py
<192.168.4.136> ESTABLISH SSH CONNECTION FOR USER: hwaraich207970
<192.168.4.136> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=username -o ConnectTimeout=10 -o ControlPath=/home/username/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.4.136 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1604952591.08-32914241935398 ` " && echo ansible-tmp-1604952591.08-32914241935398="` echo ~/.ansible/tmp/ansible-tmp-1604952591.08-32914241935
398 `" ) && sleep 0'"'"''
192.168.4.136 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n",
"unreachable": true

This could happen even if you have made sure the passwordless ssh between System A and System B (say using either ssh-copy-id command or by manually copying the public key i.e content of the idrsa.pub file on System A to .ssh/authorizedkeys file on System B. If this is happening, one of the reason could be the user home directories.
On System A user home directory is say /home/tester and on System B, it is /users/tester, then passwordless ssh might not work. Make sure both users have the same home directory solves this issue. I observed this case in CentOS machines and on making sure the home directories for users same, the issue resolved.

Ansible typically works when ssh public keys of the controller node are added to authorized keys of the remote node. This enables ansible to ssh into the remote node from the controlled node without the need for a password.
There is an alternate way to make ansible work without sharing public keys using sshpass. In this case, you need to input the password of the remote users via the ansible_ssh_pass variable. This can be done via inventory file, group_vars, or the extra-vars.
Regarding the error shared by you. It says, "Permission denied", meaning there is something wrong related to either ssh key sharing or password setting.
msg": "Failed to connect to the host via ssh: Permission denied (publickey,password).\r\n",
Debug mode provides more info related to the issue:
SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=username -o ConnectTimeout=10 -o ControlPath=/home/username/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.4.136 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1604952591.08-32914241935398 ` " && echo ansible-tmp-1604952591.08-32914241935398="` echo ~/.ansible/tmp/ansible-tmp-1604952591.08-32914241935
Some relevant information you can extract from the above snippet:
-o User=username: This means playbook is trying to execute from username user ID.
-o PasswordAuthentication=no: This would force ansible to use public keys over password.
This authentication failure is happening for 192.168.4.136.
Please check this for official info regarding connections for ansible.
Check this for generating and sharing ssh keys between the nodes.

Related

kubespary:ansible can't send data with ssh to a node in ansible-playbook command

in step 10 of tutorial
https://dzone.com/articles/kubespray-10-simple-steps-for-installing-a-product
for deploying a production ready kubernetes cluster with kubespray, an error occured when running ansible-playbook command.error is:
ERROR! SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh
ssh passwordless is active between nodes and i can run ssh from each nodes without password.
can anyone help me?
thanks
this is my command and it's output:
master-node#master-node:~/kubespray$ sudo ansible all -i inventory/mycluster/hosts.ini -m ping -vvv
ansible 2.7.8
config file = /home/master-node/kubespray/ansible.cfg
configured module search path = [u'/home/master-node/kubespray/library']
ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.12 (default, Nov 12 2018, 14:36:49) [GCC 5.4.0 20160609]
Using /home/master-node/kubespray/ansible.cfg as config file
/home/master-node/kubespray/inventory/mycluster/hosts.ini did not meet host_list requirements, check plugin documentation if this is unexpected
/home/master-node/kubespray/inventory/mycluster/hosts.ini did not meet script requirements, check plugin documentation if this is unexpected
/home/master-node/kubespray/inventory/mycluster/hosts.ini did not meet yaml requirements, check plugin documentation if this is unexpected
Parsed /home/master-node/kubespray/inventory/mycluster/hosts.ini inventory source with ini plugin
META: ran handlers
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/system/ping.py
<192.168.1.107> ESTABLISH SSH CONNECTION FOR USER: worker-node
<192.168.1.107> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=worker-node -o ConnectTimeout=10 -o ControlPath=/home/master-node/.ansible/cp/e24ed02313 192.168.1.107 '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/system/ping.py
<192.168.1.142> ESTABLISH SSH CONNECTION FOR USER: master-node
<192.168.1.142> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=master-node -o ConnectTimeout=10 -o ControlPath=/home/master-node/.ansible/cp/01ac2924af 192.168.1.142 '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
master-node | UNREACHABLE! => {
"changed": false,
"msg": "SSH Error: data could not be sent to remote host \"192.168.1.142\". Make sure this host can be reached over ssh",
"unreachable": true
}
worker-node | UNREACHABLE! => {
"changed": false,
"msg": "SSH Error: data could not be sent to remote host \"192.168.1.107\". Make sure this host can be reached over ssh",
"unreachable": true
}

SSH ok but Ansible returns "unreachable"

My SSH using keys is set up properly.
ssh admin#192.168.1.111
admin#DiskStation:~$
But Ansible returns an error:
TASK [setup] *******************************************************************
<192.168.1.111> ESTABLISH SSH CONNECTION FOR USER: admin
<192.168.1.111> SSH: EXEC ssh -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=admin -o ConnectTimeout=10 -o ControlPath=/Users/Shared/Jenkins/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.1.111 '/bin/sh -c '"'"'( umask 22 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1479205446.3-33100049148171 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1479205446.3-33100049148171 `" )'"'"''
<192.168.1.111> PUT /var/folders/pd/8q63k3z93nx_78dggb9ltm4c00007x/T/tmpNJvc43 TO /var/services/homes/admin/.ansible/tmp/ansible-tmp-1479205446.3-33100049148171/setup
<192.168.1.111> SSH: EXEC sftp -b - -C -vvv -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=admin -o ConnectTimeout=10 -o ControlPath=/Users/Shared/Jenkins/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.1.111]'
fatal: [192.168.1.111]: UNREACHABLE! => {"changed": false, "msg": "SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh", "unreachable": true}
Can someone help me?
Ansible returns "unreachable" for the SFTP connection, not SSH.
Either enable SFTP on the target node (or a firewall in-between), or configure Ansible to use SCP in ansible.cfg:
scp_if_ssh = True
I had a similar "unreachable" error, but in my case it was because my playbook file specified the host this way:
[webservers]
ubuntu#123.456.789.111
This worked for us in the past, so presumably this works with some Ansible versions, but not with my version (2.0.0.2). Instead I changed this to what the documentation recommends:
[webservers]
123.456.789.111 ansible_user=ubuntu
and now the SFTP connection does not fail.
After many years of try and error, now I always have these setting on my ansible.cfg:
[defaults]
host_key_checking = false
[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=20
scp_if_ssh = True
[connection]
pipelining = true
The pipelining is my personal preference when dealing with multiple
hosts.
The ssh_args deals with hangs and timeouts, useful when your target remote has unstable connection.
Please check if python is installed on the target machines. It's a prerequisite.
sudo apt install python3
sudo apt install python
sudo apt install python-minimal

Ansible: "Failed to connect to the host via ssh" error

I'm trying to get set up with Ansible for the first time, to connect to a Raspberry Pi. Following the official 'getting started' steps, I've made an inventory file:
192.168.1.206
.. but the ping fails as follows:
$ ansible all -m ping -vvv
No config file found; using defaults
<192.168.1.206> ESTABLISH SSH CONNECTION FOR USER: pi
<192.168.1.206> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=pi -o ConnectTimeout=10 -o ControlPath=/Users/username/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.1.206 '/bin/sh -c '"'"'( umask 22 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1464128959.67-131325759126042 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1464128959.67-131325759126042 `" )'"'"''
192.168.1.206 | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh.",
"unreachable": true
}
This looks the same as this question, but adding password/user bits make no effect for me, shouldn't be necessary to ping, and aren't in the official example anyhow. In any case I'd prefer to configure Ansible to use a specific public/private key pair (as per ssh -i ~/.ssh/keyfile method..)
Grateful for assistance.
Oh and yes the Raspberry is available at that address:
$ ping 192.168.1.206
PING 192.168.1.206 (192.168.1.206): 56 data bytes
64 bytes from 192.168.1.206: icmp_seq=0 ttl=64 time=83.822 ms
Despite what its name could suggest, Ansible ping module doesn't make an ICMP ping.
It tries to connect to host and makes sure a compatible version of Python is installed (as stated in the documentation).
ping - Try to connect to host, verify a usable python and return pong on success.
If you want to use a specific private key, you can specify ansible_ssh_private_key_file in your inventory file:
[all]
192.168.1.206 ansible_ssh_private_key_file=/home/example/.ssh/keyfile
It works for me.
10.23.4.5 ansible_ssh_pass='password' ansible_user='root'
You can also troubleshoot by executing ssh in debug mode and compare the results when running:
ssh -v pi#192.168.1.206
with:
ansible all -m ping -vvvv

I can ssh just fine, but ansible says "no route to host"

I wrote a script to run up several vms using vagrant, which I have to then provision with ansible. Unfortunately my host is a windows machine, so I thought I could solve the issue by putting all the vms into a vpn and then provision them from another machine in the same vpn.
In theory, it works... I can ssh into the other machines without trouble. But when I run my ansible playbook, ansible fails.
At first I got the message "ssh: connect to host 10.1.2.100 [10.1.2.100] port 22: No route to host" when running ansible with -vvvv
This was in the evening, and I was very tired, and this error didn't recur the following morning. Not sure if it's got something to do with the vm I'm doing deployment from being rebooted in the meantime, or the receiving machine being destroyed and uped completely since then. In any case, the problem has not gone away.
results now, after recreating both vms:
# ansible-playbook -i vms -k -u vagrant vms.yml -vvvv
result:
<10.1.2.100> ESTABLISH SSH CONNECTION FOR USER: vagrant <10.1.2.100>
SSH: EXEC sshpass -d14 ssh -C -vvv -o ServerAliveInterval=50 -o
User=vagrant -o ConnectTimeout=10 -tt 10.1.2.100 '( umask 22 && mkdir
-p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455781388.36-25193904947084 )" && echo
"$( echo $HOME/.ansible/tmp/ansible-tmp-1455781388.36-25193904947084
)" )' fatal: [10.1.2.100]: FAILED! => {"failed": true, "msg": "ERROR!
Using a SSH password instead of a key is not possible because Host Key
checking is enabled and sshpass does not support this. Please add
this host's fingerprint to your known_hosts file to manage this
host."}
So far so clear. I ssh into the other instance to add it to the known hosts. This works without any trouble.
Back to ansible, I try the same command again. The result now is:
<10.1.2.100> ESTABLISH SSH CONNECTION FOR USER: vagrant <10.1.2.100>
SSH: EXEC sshpass -d14 ssh -C -vvv -o ServerAliveInterval=50 -o
StrictHostKeyChecking=no -o User=vagrant -o ConnectTimeout=10 -tt
10.1.2.100 '( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455782149.99-271768166468916 )" &&
echo "$( echo
$HOME/.ansible/tmp/ansible-tmp-1455782149.99-271768166468916 )" )'
<10.1.2.100> PUT /tmp/tmpXQKa8Z TO
/home/vagrant/.ansible/tmp/ansible-tmp-1455782149.99-271768166468916/setup
<10.1.2.100> SSH: EXEC sshpass -d14 sftp -b - -C -vvv -o
ServerAliveInterval=50 -o StrictHostKeyChecking=no -o User=vagrant -o
ConnectTimeout=10 '[10.1.2.100]' fatal: [10.1.2.100]: UNREACHABLE! =>
{"changed": false, "msg": "ERROR! SSH Error: data could not be sent to
the remote host. Make sure this host can be reached over ssh",
"unreachable": true}
Well, I made sure the host was reachable by ssh, thank you very much! Ansible still can't get through, and I'm about to get a brain tumor from thinking of things that might be the problem.
Any suggestions what might be the problem?
This issue was reported here, with some workarounds:
https://github.com/ansible/ansible/issues/15321
The consensus seems to be either to a. use ansible_password or b. use -u username in the connection parameters. However, any number of things can disrupt an SSH connection in ways that make it look "unreachable" to higher level apps, so I recommend going through each of the steps outlined in that ticket.

Ansible : SSH Error: ControlPath too long

I run a computer with Ubuntu 15.10 and I try to run Vagrant with Ansible.
Before start, I like to say that I don't have any idea about server management and especialy the Ansible.
The reason I am going to run my system this way, is because I have start working on a project that requires this installation.
After all, the problem I have is that while provisioning the Vagrant I get the following message
<aaa.dev> ESTABLISH CONNECTION FOR USER: vagrant
<aaa.dev> REMOTE_MODULE setup
<aaa.dev> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/media/merianos/Large Internal/Vagrant/ansible-project/.vagrant/machines/default/virtualbox/private_key -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/merianos/.ansible/cp/%h-%r" -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=30 aaa.dev /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1446622406.54-199921739516776 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1446622406.54-199921739516776 && echo $HOME/.ansible/tmp/ansible-tmp-1446622406.54-199921739516776'
fatal: [aaa.dev] => SSH Error: ControlPath too long
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
So, is it posible to help me someone with this issue ?
Just to say that I have try this article : https://github.com/ansible/ansible/issues/11536 and I changed the control_path in my ansible.cfg to control_path = %(directory)s/%%h-%%r but still not working.
Note My installation path contains a space that I can't remove it because are running many other projects on the same HDD and the configuration will be huge for all the projects. I don't know if that space is the problem, but just I say about it.
UPDATE #1
Result before I change anything:
<aaa.dev> ESTABLISH CONNECTION FOR USER: vagrant
<aaa.dev> REMOTE_MODULE setup
<aaa.dev> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/media/merianos/Large Internal/Vagrant/ansible-project/.vagrant/machines/default/virtualbox/private_key -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/merianos/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=30 aaa.dev /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1446628138.53-155680153347939 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1446628138.53-155680153347939 && echo $HOME/.ansible/tmp/ansible-tmp-1446628138.53-155680153347939'
fatal: [aaa.dev] => SSH Error: ControlPath too long
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
Result with control_path = %(directory)s/%%h-%%r :
<aaa.dev> ESTABLISH CONNECTION FOR USER: vagrant
<aaa.dev> REMOTE_MODULE setup
<aaa.dev> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/media/merianos/Large Internal/Vagrant/ansible-project/.vagrant/machines/default/virtualbox/private_key -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/merianos/.ansible/cp/%h-%r" -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=30 aaa.dev /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1446628320.4-231606404275563 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1446628320.4-231606404275563 && echo $HOME/.ansible/tmp/ansible-tmp-1446628320.4-231606404275563'
fatal: [aaa.dev] => SSH Error: ControlPath too long
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
UPDATE #2
After I set the ssh_args = -o ControlMaster=off I get the following result:
<aaa.dev> ESTABLISH CONNECTION FOR USER: vagrant
<aaa.dev> REMOTE_MODULE setup
<aaa.dev> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o IdentityFile=/media/merianos/Large Internal/Vagrant/ansible-project/.vagrant/machines/default/virtualbox/private_key -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/merianos/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ConnectTimeout=30 aaa.dev /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1446628489.4-10074395967553 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1446628489.4-10074395967553 && echo $HOME/.ansible/tmp/ansible-tmp-1446628489.4-10074395967553'
fatal: [aaa.dev] => SSH Error: ControlPath too long
It is sometimes useful to re-run the command using -vvvv, which prints SSH debug output to help diagnose the issue.
In general for each modification I did, the error message seems to be the same, and maybe the configuration it happens from some other level, but not the ansible.cfg.
Unfortunatelly I don't know where to find that location :(
I described the problem in similar question.
You need to change it to something shorter (if you have long hostname). For test case you can try just ./master, but for real use case, you should use at least ./s/%%h-%%r.
The very first sentence of the documentation for OpenSSH specific settings in Ansible says:
Under the [ssh_connection] header, the following settings are tunable for SSH connections.
So you need to place the ssh_args variable in [ssh_connection] section of the ansible.cfg, for example:
[defaults]
timeout = 600
[ssh_connection]
ssh_args = -o ControlMaster=off
In fact overriding the ssh_args with an empty value disables the defaults for ControlMaster/ControlPersistent/ControlPath in Ansible, so it should simply be:
[ssh_connection]
ssh_args =
Short Answers
Pass this as an argument to ansible or ansible-playbook commands:
-e "ansible_ssh_common_args='-o ControlPath=/tmp/ssh-%r#%h:%p'"
Or as env variable
export ANSIBLE_SSH_ARGS="-o ControlPath=/tmp/ssh-%r#%h:%p'"
Or as an argument to the host definition in the inventory file
[web_servers]
host ansible_ssh_common_args='-o ControlPath=/tmp/ssh-%r#%h:%p'
Avoid using /tmp directory as its not secure
Long Answer
ControlPath too long is the error that belongs to SSH. SSH creates a control (unix) socket to reuse a TCP connection. The Control path is where this socket is saved.
Ansible may use its local config directory as the location to the ControlPath and if this local config directory is long would raise this error. Alternatively if the home directory itself is at a long path this problem may raise.
Typically, this can be fixed at SSH end by simply using a shorter path in the local ssh config file (~/.ssh/config):
Host *
ControlPath /tmp/ssh-%r#%h:%p
This will create socket file in the /tmp directory (very short path and is world writeable) using the username (%r), hostname (%h), and port number (%p) from the SSH connection as part of the filename
/tmp is world writeable and is not safe. Hackers can use the socket file from the /tmp to login to target SSH server and is dangerous. Please prefer using home directory if it is at short path
Ansible typically should read the file ~/.ssh/config and load ControlPath settings. However in some cases if it isn't, then the following can be done:
Use ANSIBLE_SSH_CONFIG environment variable like:
export ANSIBLE_SSH_CONFIG=~/.ssh/config
OR use ansible configuration file. Create an ansible.cfg file in the ansible project directory and add this code to it:
[ssh_connection]
ssh_args = -F /full/path/to/ssh_config
For me, ~ or \~ in the cfg file didn't work. So ~/.ssh/config wouldn't work and we need to specify full path to the .ssh/config for ex: /home/xxxx/.ssh/config