What I am trying to accomplish overall is to ssh into systems which are untouched by ansible, and have them set up by ansible, including its account, and ssh keys, and adding to the dynamic inventory... and so on and so forth. In this case, it's via a proxy jump. Unfortunately this means having to ssh into them using the ssh command and the shell module, as well as storing a password. Keep in mind I am on ansible 2.9, and this is a build environment, so passwords can be copied to files during build for use and then deleted at the end of the run, so this isn't a problem. If this succeeds, we can set up accounts and ssh keys, then delete the build files and everyone is happy.
I don't need all that much I hope, I would just like to get one sticky piece of that working better. That part is the ssh options that are needed for a proxyjump connection. ansible-controller doesn't have direct access to host p0, but the ecc67 host does. I have it working in the shell command no problem, but for whatever reason, I can't shift it up to the ansible_ssh_common_args variable where it belongs.
Here is the working example of the task as it functions now:
- name: sshpass attempt with the raw module for testing.
shell: sshpass -p "{{ access_var.ansible_ssh_pass_ssn }}" ssh -o 'ProxyCommand=ssh -W %h:%p bob#ecc67 nc %h %p' bob#p0 "w; exit"
register: output_1
The above works just fine and uses an undefined ansible_ssh_common_args. The nc is the netcat binary and is simply being passed options through the proxy command. Then we have the below playbook in which I tried to complete my stated mission, however, it is not functional, and fails at the sshpass task:
- name: Play that is testing for a successful proxyjump connection to p0 through ecc67.
hosts: ansible-controller
remote_user: bob
become: no
become_method: sudo
gather_facts: no
vars:
ansible_connection: ssh
ansible_ssh_common_args: '-o "ProxyCommand=ssh -W %h:%p bob#ecc67 nc %h %p"'
tasks:
- name: Import the password file so that we have the bob account's password.
include_vars:
file: ~/project/copyable-files/dynamic-files/build/active-vars-repository/access.yml
name: access_var
- name: Set password for the bob account from the file value using previous operator input.
set_fact:
ansible_ssh_pass: "{{ access_var.ansible_ssh_pass_b }}"
ansible_become_password: "{{ access_var.ansible_ssh_pass_b }}"
cacheable: yes
- name: sshpass attempt with the raw module for testing.
shell: sshpass -p "{{ ansible_ssh_pass_b }}" ssh "{{ ansible_ssh_common_args }}" bob#p0 "hostname; exit"
register: output_1
- debug:
var: output_1
The error I get when I attempt to use the above playbook with the reworked task and variables is as follows:
TASK [sshpass attempt with the raw module for testing.] ***********************************************
fatal: [ansible-controller]: UNREACHABLE! => {"changed": false, "msg": "Invalid/incorrect password: Killed by signal 1.", "unreachable": true}
The password is not the issue despite the error stating it is, though it's possible it's accessing something I don't expect. Is there any way to do what I want, heck, is there even just a better way to go about it that I didn't think of? Any suggestions would be helpful thanks!
From your description I understand that there is an issue with special characters in variables, quoting, templating and debugging. Therefore I am explicit not addressing the question "Is there ... a better way to go?".
To address the different topics I've created the following minimal example playbook
---
- hosts: localhost
become: false
gather_facts: false
vars:
ansible_ssh_pass: !unsafe "P4$$w0rd!_%&"
ansible_ssh_common_args: !unsafe '-o "ProxyCommand=ssh -W %h:%p user#jump.example.com nc %h %p"'
tasks:
- name: Debug task to show command content
lineinfile:
path: ssh.file
create: true
line: 'sshpass -p {{ ansible_ssh_pass | quote }} ssh {{ ansible_ssh_common_args }} user#test.example.com "hostname; exit"'
resulting into an output of
sshpass -p 'P4$$w0rd!_%&' ssh -o "ProxyCommand=ssh -W %h:%p user#jump.example.com nc %h %p" user#test.example.com "hostname; exit"
... the content of ssh.file and what the shell would "see"
Further Documentation
Advanced playbook syntax - Unsafe or raw strings for usage of !unsafe
The most common use cases include passwords that allow special characters
Using filters to manipulate data
You can use YAML single quote escaping ... Escaping single quotes within single quotes in YAML is done by doubling the single quote.
Using filters to manipulate data - Manipulating strings for usage of quote
To add quotes for shell usage ... | quote
Templating (Jinja2)
Ansible uses Jinja2 templating to enable dynamic expressions and access to variables and facts.
I want to redirect all udp packet use tproxy in suse(kernel:3.0.101),I do this:
1.iptables -t mangle -N SHADOWSOCKS_UDP
2.iptables -t mangle -N SHADOWSOCKS_UDP_MARK
3.ip route add local default dev lo table 100
4.ip rule add fwmark 1 lookup 100
and iptables -t mangle -L shows :
enter image description here
5:iptables -t mangle -A SHADOWSOCKS_UDP -p udp -j TPROXY --on-port 3386 --tproxy-mark 0x01/0x01
and get error:
iptables: No chain/target/match by that name.
I do not know why? Thank you very much give me some advice.
and this is my tproxy config:
enter image description here
And those command works well in ubuntu(kernel version:4.4.0)
Yes. finsh it. it because suse dose not..have xt_TPROXY kernel module. you need to compile kernel module. and insmod it.
I'm trying to put a new rule into iptables. What I would like is to put the rule into the penultimate line number (ie just before the DENY ALL statement). eg.
[vagrant#controller ~]$ sudo iptables -L INPUT --line-numbers
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
2 ACCEPT icmp -- anywhere anywhere
3 ACCEPT all -- anywhere anywhere
4 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
5 REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
then,
sudo iptables -I INPUT 5 -t filter -d 192.168.33.10 -m state --state NEW -j ACCEPT
would be:
[vagrant#controller ~]$ sudo iptables -L --line-numbers
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
2 ACCEPT icmp -- anywhere anywhere
3 ACCEPT all -- anywhere anywhere
4 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
5 ACCEPT all -- anywhere 192.168.33.10 state NEW
6 REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
the issue is that I can not figure out how I can automate this with salt without explicitly knowing what the last line number is? Is there a way of running a command in salt stack and use the result of that command as a variable for the jinja template?
sudo iptables -L INPUT --line-numbers | tail -n1 | awk '{print $1}'
I'm not aware of any simple way to do this the way its described above. Some of the work-arounds that come to mind:
Define the last rule such that it depends on (i.e. requires) the one you need to insert. Salt would then make sure that they are in the right order.
Create an additional iptables chain. Insert a jump to the new chain somewhere into the original config, then alter the rules in the chain as needed. This gives you a lot of flexibility.
Write a shell-script which does what you want and call it with cmd.run. Quite inelegant but requires fewer changes to other parts of the config.
Neither of those options is perfect and may not suit your use-case. However, combining the first to methods will get you pretty far and also result in an over all cleaner setup.
Is there a way to ignore the SSH authenticity checking made by Ansible? For example when I've just setup a new server I have to answer yes to this question:
GATHERING FACTS ***************************************************************
The authenticity of host 'xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx)' can't be established.
RSA key fingerprint is xx:yy:zz:....
Are you sure you want to continue connecting (yes/no)?
I know that this is generally a bad idea but I'm incorporating this in a script that first creates a new virtual server at my cloud provider and then automatically calls my ansible playbook to configure it. I want to avoid any human intervention in the middle of the script execution.
Two options - the first, as you said in your own answer, is setting the environment variable ANSIBLE_HOST_KEY_CHECKING to False.
The second way to set it is to put it in an ansible.cfg file, and that's a really useful option because you can either set that globally (at system or user level, in /etc/ansible/ansible.cfg or ~/.ansible.cfg), or in an config file in the same directory as the playbook you are running.
To do that, make an ansible.cfg file in one of those locations, and include this:
[defaults]
host_key_checking = False
You can also set a lot of other handy defaults there, like whether or not to gather facts at the start of a play, whether to merge hashes declared in multiple places or replace one with another, and so on. There's a whole big list of options here in the Ansible docs.
Edit: a note on security.
SSH host key validation is a meaningful security layer for persistent hosts - if you are connecting to the same machine many times, it's valuable to accept the host key locally.
For longer-lived EC2 instances, it would make sense to accept the host key with a task run only once on initial creation of the instance:
- name: Write the new ec2 instance host key to known hosts
connection: local
shell: "ssh-keyscan -H {{ inventory_hostname }} >> ~/.ssh/known_hosts"
There's no security value for checking host keys on instances that you stand up dynamically and remove right after playbook execution, but there is security value in checking host keys for persistent machines. So you should manage host key checking differently per logical environment.
Leave checking enabled by default (in ~/.ansible.cfg)
Disable host key checking in the working directory for playbooks you run against ephemeral instances (./ansible.cfg alongside the playbook for unit tests against vagrant VMs, automation for short-lived ec2 instances)
I found the answer, you need to set the environment variable ANSIBLE_HOST_KEY_CHECKING to False. For example:
ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook ...
Changing host_key_checking to false for all hosts is a very bad idea.
The only time you want to ignore it, is on "first contact", which this playbook will accomplish:
---
- name: Bootstrap playbook
# Don't gather facts automatically because that will trigger
# a connection, which needs to check the remote host key
gather_facts: false
tasks:
- name: Check known_hosts for {{ inventory_hostname }}
local_action: shell ssh-keygen -F {{ inventory_hostname }}
register: has_entry_in_known_hosts_file
changed_when: false
ignore_errors: true
- name: Ignore host key for {{ inventory_hostname }} on first run
when: has_entry_in_known_hosts_file.rc == 1
set_fact:
ansible_ssh_common_args: "-o StrictHostKeyChecking=no"
# Now that we have resolved the issue with the host key
# we can "gather facts" without issue
- name: Delayed gathering of facts
setup:
So we only turn off host key checking if we don't have the host key in our known_hosts file.
You can pass it as command line argument while running the playbook:
ansible-playbook play.yml --ssh-common-args='-o StrictHostKeyChecking=no'
forward to nikobelia
For those who using jenkins to run the play book, I just added to my jenkins job before running the ansible-playbook the he environment variable ANSIBLE_HOST_KEY_CHECKING = False
For instance this:
export ANSIBLE_HOST_KEY_CHECKING=False
ansible-playbook 'playbook.yml' \
--extra-vars="some vars..." \
--tags="tags_name..." -vv
If you don't want to modify ansible.cfg or the playbook.yml then you can just set an environment variable:
export ANSIBLE_HOST_KEY_CHECKING=False
Ignoring checking is a bad idea as it makes you susceptible to Man-in-the-middle attacks.
I took the freedom to improve nikobelia's answer by only adding each machine's key once and actually setting ok/changed status in Ansible:
- name: Accept EC2 SSH host keys
connection: local
become: false
shell: |
ssh-keygen -F {{ inventory_hostname }} ||
ssh-keyscan -H {{ inventory_hostname }} >> ~/.ssh/known_hosts
register: known_hosts_script
changed_when: "'found' not in known_hosts_script.stdout"
However, Ansible starts gathering facts before the script runs, which requires an SSH connection, so we have to either disable this task or manually move it to later:
- name: Example play
hosts: all
gather_facts: no # gather facts AFTER the host key has been accepted instead
tasks:
# https://stackoverflow.com/questions/32297456/
- name: Accept EC2 SSH host keys
connection: local
become: false
shell: |
ssh-keygen -F {{ inventory_hostname }} ||
ssh-keyscan -H {{ inventory_hostname }} >> ~/.ssh/known_hosts
register: known_hosts_script
changed_when: "'found' not in known_hosts_script.stdout"
- name: Gathering Facts
setup:
One kink I haven't been able to work out is that it marks all as changed even if it only adds a single key. If anyone could contribute a fix that would be great!
You can simply tell SSH to automatically accept fingerprints for new hosts. Just add
StrictHostKeyChecking=accept-new
to your ~/.ssh/config. It does not disable host-key checking entirely, it merely disables this annoying question whether you want to add a new fingerprint to your list of known hosts. In case the fingerprint for a known machine changes, you will still get the error.
This policy also works with ANSIBLE_HOST_KEY_CHECKING and other ways of passing this param to SSH.
I know the question has been answered and it's correct as well, but just wanted to link the ansible doc where it's explained clearly when and why respective check should be added: host-key-checking
The most problems appear when you want to add new host to dynamic inventory (via add_host module) in playbook. I don't want to disable fingerprint host checking permanently so solutions like disabling it in a global config file are not ok for me. Exporting var like ANSIBLE_HOST_KEY_CHECKING before running playbook is another thing to do before running that need to be remembered.
It's better to add local config file in the same dir where playbook is. Create file named ansible.cfg and paste following text:
[defaults]
host_key_checking = False
No need to remember to add something in env vars or add to ansible-playbook options. It's easy to put this file to ansible git repo.
This one is the working one I used in my environment. I use the idea from this ticket https://github.com/mitogen-hq/mitogen/issues/753
- name: Example play
gather_facts: no
hosts: all
tasks:
- name: Check SSH known_hosts for {{ inventory_hostname }}
local_action: shell ssh-keygen -l -F {{ inventory_hostname }}
register: checkForKnownHostsEntry
failed_when: false
changed_when: false
ignore_errors: yes
- name: Add {{ inventory_hostname }} to SSH known hosts automatically
when: checkForKnownHostsEntry.rc == 1
changed_when: checkForKnownHostsEntry.rc == 1
local_action:
module: shell
args: ssh-keyscan -H "{{ inventory_hostname }}" >> $HOME/.ssh/known_hosts
Host key checking is important security measure so I would not just skip it everywhere. Yes, it can be annoying if you keep reinstalling same testing host (without backing up it's SSH certificates) or if you have stable hosts but you run your playbook for Jenkins without simple option to add host key if you are connecting to the host for a first time. So:
This is what we are using for stable hosts (when running the playbook from Jenkins and you simply want to accept the host key when connecting to the host for the first time) in inventory file:
[all:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=accept-new'
And this is what we have for temporary hosts (in the end this will ignore they host key at all):
[all:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
There is also environment variable or you can add it into group/host variables file. No need to have it in the inventory - it was just convenient in our case.
Used some other responses here and a co-worker solution, thank you!
Use the parameter named as validate_certs to ignore the ssh validation
- ec2_ami:
instance_id: i-0661fa8b45a7531a7
wait: yes
name: ansible
validate_certs: false
tags:
Name: ansible
Service: TestService
By doing this it ignores the ssh validation process
I am currently trying to secure a little my server before its release to the world. For now, there is just a Discourse instance running, that uses Mandrill as email smtp server.
There is an nginx server in front of that Discourse.
With no iptables rules, everything works fine. When I apply my rules, it brokes. I am still able to reach the Discourse and even send posts and everything, expect sending email.
With ./launcher mailtest app, it works. The Discourse error, however, is the following : ERREUR - getaddrinfo: Name or service not known.
I really try to find out myself what I should use. But I couldn't.
First, I was thinking a simple iptables -A OUTPUT -p tcp --dport 587 -j ACCEPT was enough, but I was proved the contrary.
Some other inputs :
iptables -F
iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP
I set Discourse to use port 587 of Mandrill.
Okay, so I just released... This is not the right SE forum for that question. I'm sorry for that.
However, since I finally found a solution (it's always when you post your question that the question hits you in the face), let me share it.
I was missing a FORWARD rule between docker0 and eth0.
iptables -A FORWARD -i docker0 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth0 -o docker0 -j ACCEPT
Sorry for the inconvenient.