How to get error output when ansible.systemd fails with restart? - error-handling

With the following ansible:
- name: Reload changes in configuration and restart docker service
systemd:
name: docker
enabled: true
daemon_reload: true
state: restarted
register: command_output
- name: Print to console
debug:
msg: "{{command_output.stdout}}"
I see the following error:
fatal: [xxxxxx]: FAILED! => {"changed": false, "msg": "Unable to start
service docker: Job for docker.service failed because the control
process exited with error code.\nSee "systemctl status
docker.service" and "journalctl -xe" for details.\n"}
This is of course a very useful error message when on the command line, but in ansible, not so much. My attempt at capturing the error output and dispalying it in debug has not been very fruitful. So the question is:
How can I get more details about the reason why ansible.systemd failed in this case?
Should I try to invoke journalctl -xe or systemctl status docker.service manually, or is there some other more ansible friendly way?

Whatever ansible module capture is already there stderr or stdout return values.. If you want to get more details of the error, you can try Block and Rescue ... Block and Resuce Documentation
block:
- name: Reload changes in configuration and restart docker service
systemd:
name: docker
enabled: true
daemon_reload: true
state: restarted
register: command_output
- name: Print to console
debug:
msg: "{{command_output.stdout}}"
rescue:
- name: get errors
shell: journalctl -xe # or systemctl status docker.service
register: err_msg
- name: Print error message to console
debug:
msg: "{{ err_msg.stdout }}"

Related

Fetch a file from task in same Ansible playbook

How do I transfer a file I have created from a previous task in my Ansible playbook? Here is what I got so far:
- name: Create Yum Report
shell: |
cd /tmp
yum history info > $(hostname -s)_$(date "+%d-%m-%Y").txt
register: after_pir
- name: Transfer PIR
fetch:
src: /tmp/{{ after_pir }}
dest: /tmp/
However, I receive this error message when I run my playbook.
TASK [Transfer PIR] ************************************************************************************************************
failed: [x.x.x.x] (item=after_pir) => {"ansible_loop_var": "item", "changed": false, "item": "after_pir", "msg": "the remote file does not exist, not transferring, ignored"}
I have tried to run different fetch, synchronzie and pull methods but I'm not sure what the issue is.
One way to do that:
- name: Create Yum Report
command: yum history info
register: yum_report
- name: Dump report on local disk for each host
copy:
content: "{{ yum_report.stdout }}"
dest: "/tmp/{{ inventory_hostname_short }}-{{ '%d-%m-%Y' | strftime }}"
delegate_to: localhost

How to deal with multiple when condition for registered variable in ansible

I have a playbook 3 raw task (or more) with sample commands like below:
Playbook mytest.yml
- hosts: remotehost
gather_facts: no
tasks:
- name: Execute command1
raw: "ls -ltr"
register: cmdoutput
when: remcmd == "list"
- name: Execute command2
raw: "hostname"
register: cmdoutput
when: remcmd == "host"
- name: Execute command3
raw: "uptime"
register: cmdoutput
when: remcmd == "up"
- hosts: localhost
gather_facts: no
tasks:
- debug:
msg: "Printing {{ hostvars['remotehost']['cmdoutput'] }}"
This is my nventory myhost.yml
[remotehost]
myserver1
Here is how I run the playbook:
ansible-playbook -i myhost.yml mytest.yml -e remcmd="host"
PLAY [remotehost] ***************************************************************************************************************
TASK [Execute command1] *********************************************************************************************************
Thursday 06 October 2022 07:06:06 -0500 (0:00:00.013) 0:00:00.013 ******
skipping: [myserver1]
TASK [Execute command2] *********************************************************************************************************
Thursday 06 October 2022 07:06:06 -0500 (0:00:00.023) 0:00:00.036 ******
changed: [myserver1]
TASK [Execute command3] *********************************************************************************************************
Thursday 06 October 2022 07:06:06 -0500 (0:00:00.521) 0:00:00.557 ******
skipping: [myserver1]
PLAY [localhost] ****************************************************************************************************************
TASK [debug] ********************************************************************************************************************
Thursday 06 October 2022 07:06:06 -0500 (0:00:00.032) 0:00:00.590 ******
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \"hostvars['remotehost']\" is undefined\n\nThe error appears to be in '/home/wladmin/mytest.yml': line 22, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n - debug:\n ^ here\n"}
PLAY RECAP **********************************************************************************************************************
localhost : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
myserver1 : ok=1 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
My requirement is no matter what value is passed for remcmd my localhost play should print stdoutlines of cmdoutput
Preliminary notes:
Using raw is evil.
Don't use raw unless to install prereqs (i.e. python) on the target host. Then switch to modules or at the very least command/shell
If you still intend to use raw, go back to point 1 above
In case your forgot to go back to point 1: using raw is evil
Don't register several tasks with the same var name (the last one always win, even if skipped). Don't create tasks you can avoid up-start.
As an illustration of the above principles
- hosts: remotehost
gather_facts: no
vars:
cmd_map:
list: ls -ltr
host: hostname
up: uptime
tasks:
- name: Make sure remcmd is known
assert:
that: remcmd in cmp_map.keys()
fail_msg: "remcmd must be one of: {{ cmd_map.keys() | join(', ') }}"
- name: Execute command
command: "{{ cmd_map[remcmd] }}"
register: cmdoutput
- name: Show entire result from above task
debug:
var: cmdoutput
my localhost play should print stdout_lines of cmdoutput
As far as I understand "How the debug module works", it can only print on the Control Node.
Therefore you could just remove three (3) lines in your example
- hosts: localhost
gather_facts: no
tasks:
and give it a try with
- hosts: remotehost
gather_facts: no
tasks:
- name: Execute command1
raw: "ls -ltr"
register: cmdoutput
when: remcmd == "list"
- name: Execute command2
raw: "hostname"
register: cmdoutput
when: remcmd == "host"
- name: Execute command3
raw: "uptime"
register: cmdoutput
when: remcmd == "up"
- debug:
msg: "Printing {{ cmdoutput }}"
and independently of which task became executed the result would be provided.
Apart from the answer about "How the debug module works" here, I like to recommended to proceed further with the answer of Zeitounator, since it will address your possible use case more complete.

How to alert via email in Ansible

I have setup a mail task in ansible to send emails if yum update is marked as 'changed'.
Here is my current working code:
- name: Send mail alert if updated
community.general.mail:
to:
- 'recipient1'
cc:
- 'recipient2'
subject: Update Alert
body: 'Ansible Tower Updates have been applied on the following system: {{ ansible_hostname }}'
sender: "ansible.updates#domain.com"
delegate_to: localhost
when: yum_update.changed
This works great, however, every system that gets updated per host group sends a separate email. Last night for instance I had a group of 20 servers update and received 20 separate emails. I'm aware of why this happens, but my question is how would I script this to add all the systems to one email? Is that even possible or should I just alert that the group was updated and inform teams of what servers are in each group? (I'd prefer not to take the second option)
Edit 1:
I have added the code suggested and am now unable to receive any emails. Here's the error message:
"msg": "The conditional check '_changed|length > 0' failed. The error was: error while evaluating conditional (_changed|length > 0): {{ hostvars|dict2items| selectattr('value.yum_update.changed')| map(attribute='key')|list }}: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'yum_update'\n\nThe error appears to be in '/tmp/bwrap_1073_o8ibkgrl/awx_1073_0eojw5px/project/yum-update-ent_template_servers.yml': line 22, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Send mail alert if updated\n ^ here\n",
I am also attaching my entire playbook for reference:
---
- name: Update enterprise template servers
hosts: ent_template_servers
tasks:
- name: Update all packages
yum:
name: '*'
state: latest
register: yum_update
- name: Reboot if needed
import_tasks: /usr/share/ansible/tasks/reboot-if-needed-centos.yml
- name: Kernel Cleanup
import_tasks: /usr/share/ansible/tasks/kernel-cleanup.yml
- debug:
var: yum_update.changed
- name: Send mail alert if updated
community.general.mail:
to:
- 'email#domain.com'
subject: Update Alert
body: |-
Updates have been applied on the following system(s):
{{ _changed }}
sender: "ansible.updates#domain.com"
delegate_to: localhost
run_once: true
when: _changed|length > 0
vars:
_changed: "{{ hostvars|dict2items|
selectattr('yum_update.changed')|
map(attribute='key')|list }}"
...
Ansible version is: 2.9.27
Ansible Tower version is: 3.8.3
Thanks in advance!
For example, the mail task below
- debug:
var: yum_update.changed
- community.general.mail:
sender: ansible
to: root
subject: Update Alert
body: |-
Updates have been applied to the following system:
{{ _changed }}
delegate_to: localhost
run_once: true
when: _changed|length > 0
vars:
_changed: "{{ hostvars|dict2items|
selectattr('value.yum_update.changed')|
map(attribute='key')|list }}"
TASK [debug] ***************************************************************
ok: [host01] =>
yum_update.changed: true
ok: [host02] =>
yum_update.changed: false
ok: [host03] =>
yum_update.changed: true
TASK [community.general.mail] **********************************************
ok: [host01 -> localhost]
will send
From: ansible#domain.com
To: root#domain.com
Cc:
Subject: Update Alert
Date: Wed, 09 Feb 2022 16:55:47 +0100
X-Mailer: Ansible mail module
Updates have been applied to the following system:
['host01', 'host03']
Remove the condition below if you want to receive also empty lists
when: _changed|length > 0
Debug
'ansible.vars.hostvars.HostVarsVars object' has no attribute 'yum_update'
Q: "What I could try?"
A: Some of the hosts are missing the variables yum_update. You can test it
- debug:
msg: "{{ hostvars|dict2items|
selectattr('value.yum_update.changed')|
map(attribute='key')|list }}"
run_once: true
Either make sure that the variable is defined on all hosts or use json_query. This filter tolerates missing attributes, e.g.
- debug:
msg: "{{ hostvars|dict2items|
json_query('[?value.yum_update.changed].key') }}"
run_once: true
Q: "The 'debug' task prior to the 'mail' task gives me the same output. But it fails when the 'mail' task is executed."
A: Minimize the code and isolate the problem. For example, in the code below you can see
Variable yum_update.changed is missing on host03
The filter json_query ignores this
The filter selectattr fails
- debug:
var: yum_update.changed
- debug:
msg: "{{ hostvars|dict2items|
json_query('[?value.yum_update.changed].key') }}"
run_once: true
- debug:
msg: "{{ hostvars|dict2items|
selectattr('value.yum_update.changed')|
map(attribute='key')|list }}"
run_once: true
gives
TASK [debug] **************************************************
ok: [host01] =>
yum_update.changed: true
ok: [host02] =>
yum_update.changed: false
ok: [host03] =>
yum_update.changed: VARIABLE IS NOT DEFINED!
TASK [debug] **************************************************
ok: [host01] =>
msg:
- host01
TASK [debug] **************************************************
fatal: [host01]: FAILED! =>
msg: |-
The task includes an option with an undefined variable.
The error was: 'ansible.vars.hostvars.HostVarsVars object'
has no attribute 'yum_update'
Both filters give the same results if all variables are present
TASK [debug] **************************************************
ok: [host01] =>
yum_update.changed: true
ok: [host02] =>
yum_update.changed: false
ok: [host03] =>
yum_update.changed: true
TASK [debug] **************************************************
ok: [host01] =>
msg:
- host01
- host03
TASK [debug] **************************************************
ok: [host01] =>
msg:
- host01
- host03

Ansible ssh error: mux_client_read_packet: read header failed: Broken pipe Received exit status from master

I have a script /wd/remoteuser/stopALL.sh on remotehost i.e 10.0.0.211 and takes 3 seconds to complete execution and has full permission 775 for remoteuser.
Note: /wd/remoteuser/stopALL.sh does not exist on the host where ansible runs.
I wish to trigger the stop script on remotehost from my ansiblehost.
Below is how i run my ansible playbook.
ansible-playbook /app/playbook/ovs.yml -i /app/playbook/ovs.hosts -t stop -f 5 -e Environment=PROD -e Country=SRILANKA -vvvv
cat /app/playbook/ovs.yml
---
- name: Play 1- check for login and mount point
hosts: "*{{ Country }}_{{ Environment }}"
user: "{{ USER }}"
any_errors_fatal: true
vars:
ansible_ssh_extra_args: -o StrictHostKeyChecking=no -o ConnectTimeout=90 -o ServerAliveInterval=50
ansible_ssh_private_key_file: /app/ssh_keys/id_rsa
gather_facts: false
tasks:
- name: Execute backup stop1 script
tags: stop,restart
script: "{{ stopscript }}"
args:
chdir: "{{ stopscript | dirname }}"
register: stopscriptoutput
- name: Debug stopscript
tags: stop,restart
debug:
msg: "{{ stopscriptoutput.stdout }}"
cat /app/playbook/ovs.hosts
[APP_SRILANKA_PROD]
10.0.0.211 USER=remoteuser stopscript=/wd/remoteuser/stopALL.sh countrydet=SRILANKA evt=PROD
Output:
<10.0.0.211> (0, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256#libssh.org,diffie-hellman-group14-sha1,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,gss-gex-sha1-,gss-group14-sha1-]\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 190236\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
fatal: [10.0.0.211]: FAILED! => {
"changed": false,
"msg": "Could not find or access '/wd/remoteuser/stopALL.sh' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"
}
NO MORE HOSTS LEFT *****************************************************************************************************************************************************
PLAY RECAP *************************************************************************************************************************************************************
10.0.0.211 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I get this ssh read header failed: Broken pipe error even if I use the shell module as shown below.
- name: Execute backup stop1 script
tags: stop
shell: "sleep 90; {{ stopscript }}; sleep 90"
register: stopscriptoutput
Kindly suggest how can I resolve the ssh broken pipe error and get the script to execute remotely?
Set the proxy in environment variable and it started working

How to detect unreachable target hosts in ansible

I wish to grab in a variable sshreachable if a target hosts all_hosts are reachable or not.
I wrote the below playbook for the same.
- name: Play 3- check telnet nodes
hosts: localhost
ignore_unreachable: yes
- name: Check all port numbers are accessible from current host
include_tasks: innertelnet.yml
with_items: "{{ groups['all_hosts'] }}"
cat innertelnet.yml
---
- name: Check ssh connectivity
block:
- raw: "ssh -o BatchMode=yes root#{{ item }} echo success"
ignore_errors: yes
register: sshcheck
- debug:
msg: "SSHCHECK variable:{{ sshcheck }}"
- set_fact:
sshreachable: 'SSH SUCCESS'
when: sshcheck.unreachable == 'false'
- set_fact:
sshreachable: 'SSH FAILED'
when: sshcheck.unreachable == 'true'
- debug:
msg: "INNERSSH1: {{ sshreachable }}"
Unfortunately, i get error like below:
Output:
TASK [raw] *********************************************************************
fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Shared connection to 10.9.9.126 closed.", "skip_reason": "Host localhost is unreachable", "unreachable": true}
TASK [debug] ***********************************************************************************************************************************************************
task path:
ok: [localhost] => {
"msg": "SSHCHECK variable:{'msg': u'Failed to connect to the host via ssh: Shared connection to 10.9.9.126 closed.', 'unreachable': True, 'changed': False}"
}
TASK [set_fact] ****************************************************************
skipping: [localhost]
TASK [set_fact] ****************************************************************
skipping: [localhost]
TASK [debug] *******************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'sshreachable' is undefined\n\nThe error appears to be in '/app/playbook/checkssh/innertelnet.yml': line 45, column 10, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - debug:\n ^ here\n"}
PLAY RECAP *********************************************************************
10.0.116.194 : ok=101 changed=1 unreachable=9 failed=0 skipped=12 rescued=0 ignored=95
localhost : ok=5 changed=0 unreachable=1 failed=1 skipped=4 rescued=0 ignored=0
Can you please suggest changes to my code to get this to work?
The error seems to indicate that sshreachable variable is not getting set as the when: condition does not match. I.e. sshcheck.unreachable might not be something returned by raw.
For this purpose, command module should be enough, and we can evaluate the return code of the command to set_fact.
You could do something like:
- block:
- command: ssh -o BatchMode=yes user#host1 echo success
ignore_errors: yes
register: sshcheck
- set_fact:
sshreachable: "{{ sshcheck is success }}"
- debug:
msg: "Host1 reachable: {{ sshreachable | string }}"
Update:
raw module seems to work the same way. Example (including #mdaniel's valuable input):
- block:
- raw: ssh -o BatchMode=yes user#host1 echo success
ignore_errors: yes
register: sshcheck
- set_fact:
sshreachable: SSH SUCCESS
when: sshcheck is success
- set_fact:
sshreachable: SSH FAILED
when: sshcheck is failed
- debug:
msg: "Host1 reachable: {{ sshreachable }}"