I have very simple script in monit:
check file reload_nginx with path /srv/www/site/shared/pids/reload_nginx
if not exist then exec "/bin/bash -c 'echo \"OK\"'" else if succeeded then exec "/bin/bash -c 'service nginx reload; rm /srv/www/site/shared/pids/reload_nginx'"
Monit shows that it's in "accessible" state.
But script does ... nothing.
File still exists on that path. No messages in syslog.
I've tried to change exec to alerts and echos - and still received nothing. I can't even say, whether checks are performed or not.
How to deal with it?
So, ok, i found the answer.
[if succeeded] branch in monit executed only when state of check CHANGED from "Failed" to "Succeeded". If check is succeeded at the start of monitoring - that branch wan't be called.
Related
I have a CI stage with the following command, which has to be executed remotely and checks if the mentioned file exists, if yes it creates a backup for it.
script: |
ssh ${USER}#${HOST} '([ -f "${PATH}/test_1.txt" ] && cp -v "${PATH}/test_1.txt" ${PATH}/test_1_$CI_COMMIT_TIMESTAMP.txt)'
The issue is, this job always fails whether the file exists or not with the following output:
ssh user#hostname '([ -f /etc/file/path/test_1.txt ] && cp -v /etc/file/path/test_1.txt /etc/file/path/test_1_$CI_COMMIT_TIMESTAMP.txt)'
Cleaning up project directory and file based variables
ERROR: Job failed: exit status 1
Running the same command manually, just works fine. So,
How can I make sure that this job succeeds as long as command logic is executed successfully and only fail incase there are some genuine failures?
There is no way for the job to know if the command you ran remotely worked or not. It can only know if the ssh instruction worked or not. You can force it to always succeed by appending || true to any instruction.
However, if you want to see and save the output of your remote instruction, you can do something like this:
ssh user#host command 2>&1 | tee ssh-session.log
I have a python script that I use as to check certain system conditions to alert on using monit.
This script prints a buch of stuff during execution to stdout which I want to capture in a log file.
How should I configure the monit conf script, such that I can capture both the stdout and stderr of this script and at the same time alerting on the exit status of the script. The monit alert should also included the stdout/stderr for the alert events.
This is what I tried
#/etc/monit/conf/myprogram.conf
check program my_program with path "/usr/bin/python -u /opt/program/my_program.py > my_prgoram.log 2&>1"
if status !=0 alert
But I see that the monit always thinks that the program is reporting status=0 even when then it exists with error code 1.
What am i doing wrong?
$MONIT_DESCRIPTION contains stderr
Try this:
#/etc/monit/conf/myprogram.conf
check program my_program with path "/bin/bash -c '/usr/bin/python -u /opt/program/my_program.py > my_prgoram.log 2&>1'"
if status !=0 alert
I have got a problem with incoming WALs directory in Barman - backup tool to postgresql databases
In my database server I have in postgresql.conf
wal_level = 'archive'
archive_mode = on
archive_command = 'rsync -a %p barman#mybarmanserverip:INCOMING_WALS_DIRECTORY/%f'
In my barman server when I make command "barman show-server myservername" I get, that my incoming_wals_directory is
/var/lib/barman/myservername/incoming
Command barman check myservername return "OK" in all points, but when I want to make backup in command barman backup myservername I see that first 3 points is correct but point "Asking PostgreSQL server to finalize the backup" never ends.
Where is my mistake?
I had this issue and that was a problem due to rsync.
To check if it's the case for you, try to rsync a random file :
rsync -zvh random_file user#remote_host:/tmp/test
if the output is something like:
protocol version mismatch -- is your shell clean?
then there is 2 possible reasons :
rsync versions are not the same on the two servers
some text is output when you ssh to the remote server, rsync does not like it
To fix the first issue, here is what I did :
be sure that rsync --version is the same on both machines :
on your local env run rsync --version
from your local (to the remote) run ssh login#remote_host "rsync --version"
(Install the correct version if they don't match.)
To fix the second issue, you must add something in your .bashrc file that prevent text output after ssh connection on non interactive session (i.e "Last login: Thu Sep..." - it makes rsync fail)
I put that at the top of my .bashrc file :
case $- in
*i*) ;;
*) return;;
esac
Then rsync works fine, and the initial barman backup command finnishes well
replace INCOMING_WALS_DIRECTORY with your incoming folder path which you can find using this command barman show-server main
archive_command = 'rsync -a %p barman#mybarmanserverip:/var/lib/barman/main/incoming/%f'
Make sure you change the INCOMING_WALS_DIRECTORY placeholder with the value returned by the barman show-server main command above.
Also make sure that postgres user can ssh to barman server correctly.
I am getting syntax errors when I try to check the exit status of a bash script using monit's "check program". If I run the bash script manually, it doesn't error. But if I do monit reload with my monit check program in the config, monit gives me errors.
Here's my current monit .conf file --
check program myscript with path "/etc/monit.d/script_to_run.sh"
if status != 0 then alert
If I change it to this to make sure monit uses bash to parse the file...
check program myscript with path "/usr/bash /etc/monit.d/script_to_run.sh"
if status != 0 then alert
Then I get this error
Warning: Program does not exist: '"/bin/bash /etc/monit.d/script_to_run.sh"'
Even a test script results in errors --
If I have this in script_to_run.sh:
#!/bin/bash
exit 0
Monit will complain
Error: syntax error 'exit'
If I have this in script_to_run.sh:
#!/bin/bash
echo "hello"
Monit will complain
Error: syntax error 'echo'
For reference -- I have monit 5.4, have followed the "check program" example in the man monit doc, and am able to configure/use monit "check process..." just fine on the same server.
ok, fixed. for future reference for anyone reading this -- never put .sh files in /etc/monit.d/
that was the problem. i moved the .sh file to a different directory and everything is fine now.
I am using Amazon AMI on AWS cloud. Unfortunately, by default Amazon is shipping older version of Monit - 5.2.5
https://forums.aws.amazon.com/thread.jspa?threadID=215645
They suggest to install version of monit otherwise. Amazon forum gave this workaround:
sudo yum install -y https://kojipkgs.fedoraproject.org//packages/monit/5.14/1.el6/x86_64/monit-5.14-1.el6.x86_64.rpm
I've been using monit for a little while, but I want to alarm if a file exists. This is the opposite use case from the main documentation.
Here's the doc says:
IF [DOES] NOT EXIST [[<X>] <Y> CYCLES] THEN action [ELSE IF SUCCEEDED [[<X>] <Y> CYCLES] THEN action]
action is a choice of "ALERT", "RESTART", "START", "STOP", "EXEC" or "UNMONITOR".
This gives me the recipe for "freak out if file is missing". But I want to "freak out if the file's there". And the choice of actions implies there's no "do nothing" action. I could shell out to a no-op, but that's really silly for the standard case of "do nothing".
I guessed some basic cases:
IF EXISTS THEN alarm
IF EXIST THEN ALARM
So, is there a standard way to do IF IT DOES EXIST?
I recently was looking for the same solution as you and unfortunately, I was unable to discover a way of doing this in monit.
My situation differs slightly from yours so I ended up alarming if the file did not exist, and executed a shell script if it did. Like you, I did not want to spawn a shell just because the file did not exist, and having "file does not exist" show up in /var/log/messages isn't a big deal for me.
I know you said that you could shell out to a no-op so you probably don't need the following but I am adding it for those who might have the same issue and not know how to do it.
check file testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alarm
Note that you must exec /bin/bash to write the output of echo to /dev/null or monit will literally echo "dne > /dev/null"
Edit: As it was brought to my attention by disasteraverted, newer versions of Monit use alert rather than alarm, so the check would look like this:
check file testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alert
since monit 5.21.0, alterting on existence is directly supported:
check file testfile with path /path/to/file
if exist then alert
see in changelog https://mmonit.com/monit/changes/#5.21.0
Please check with :
check program not_exist_file_root_test with path "/bin/ls /root/test"
if status = 0 then alert
or
check program not_exist_file_root_test with path /bin/sh -c "test -f /root/test"
if status = 0 then alert
My 2 cents
renab, your check should end with "then alert" not "then alarm" at least in my version (5.2.5).
testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alert