I have a python script that I use as to check certain system conditions to alert on using monit.
This script prints a buch of stuff during execution to stdout which I want to capture in a log file.
How should I configure the monit conf script, such that I can capture both the stdout and stderr of this script and at the same time alerting on the exit status of the script. The monit alert should also included the stdout/stderr for the alert events.
This is what I tried
#/etc/monit/conf/myprogram.conf
check program my_program with path "/usr/bin/python -u /opt/program/my_program.py > my_prgoram.log 2&>1"
if status !=0 alert
But I see that the monit always thinks that the program is reporting status=0 even when then it exists with error code 1.
What am i doing wrong?
$MONIT_DESCRIPTION contains stderr
Try this:
#/etc/monit/conf/myprogram.conf
check program my_program with path "/bin/bash -c '/usr/bin/python -u /opt/program/my_program.py > my_prgoram.log 2&>1'"
if status !=0 alert
Related
I have very simple script in monit:
check file reload_nginx with path /srv/www/site/shared/pids/reload_nginx
if not exist then exec "/bin/bash -c 'echo \"OK\"'" else if succeeded then exec "/bin/bash -c 'service nginx reload; rm /srv/www/site/shared/pids/reload_nginx'"
Monit shows that it's in "accessible" state.
But script does ... nothing.
File still exists on that path. No messages in syslog.
I've tried to change exec to alerts and echos - and still received nothing. I can't even say, whether checks are performed or not.
How to deal with it?
So, ok, i found the answer.
[if succeeded] branch in monit executed only when state of check CHANGED from "Failed" to "Succeeded". If check is succeeded at the start of monitoring - that branch wan't be called.
I was wondering if I could use bamboo's SSH task to run a script (this kicks off a small java message injector).
Then grep the logs for ERRORS. If any ERROR is present I would like to fail the build.
Something like this:
Is this a Bash question or is it really about Bamboo? Here is the Bash problem answer:
If you run
[[ ! $(grep ERROR /a/directory/log/*) ]]
the script will exit with an error if it finds the word "ERROR" anywhere in the files.
Bamboo should detect the task execution as failed.
(Note that if Bash is not the default shell on your target system you may need a #!/bin/bash on top of the script file.)
I am getting syntax errors when I try to check the exit status of a bash script using monit's "check program". If I run the bash script manually, it doesn't error. But if I do monit reload with my monit check program in the config, monit gives me errors.
Here's my current monit .conf file --
check program myscript with path "/etc/monit.d/script_to_run.sh"
if status != 0 then alert
If I change it to this to make sure monit uses bash to parse the file...
check program myscript with path "/usr/bash /etc/monit.d/script_to_run.sh"
if status != 0 then alert
Then I get this error
Warning: Program does not exist: '"/bin/bash /etc/monit.d/script_to_run.sh"'
Even a test script results in errors --
If I have this in script_to_run.sh:
#!/bin/bash
exit 0
Monit will complain
Error: syntax error 'exit'
If I have this in script_to_run.sh:
#!/bin/bash
echo "hello"
Monit will complain
Error: syntax error 'echo'
For reference -- I have monit 5.4, have followed the "check program" example in the man monit doc, and am able to configure/use monit "check process..." just fine on the same server.
ok, fixed. for future reference for anyone reading this -- never put .sh files in /etc/monit.d/
that was the problem. i moved the .sh file to a different directory and everything is fine now.
I am using Amazon AMI on AWS cloud. Unfortunately, by default Amazon is shipping older version of Monit - 5.2.5
https://forums.aws.amazon.com/thread.jspa?threadID=215645
They suggest to install version of monit otherwise. Amazon forum gave this workaround:
sudo yum install -y https://kojipkgs.fedoraproject.org//packages/monit/5.14/1.el6/x86_64/monit-5.14-1.el6.x86_64.rpm
I've been using monit for a little while, but I want to alarm if a file exists. This is the opposite use case from the main documentation.
Here's the doc says:
IF [DOES] NOT EXIST [[<X>] <Y> CYCLES] THEN action [ELSE IF SUCCEEDED [[<X>] <Y> CYCLES] THEN action]
action is a choice of "ALERT", "RESTART", "START", "STOP", "EXEC" or "UNMONITOR".
This gives me the recipe for "freak out if file is missing". But I want to "freak out if the file's there". And the choice of actions implies there's no "do nothing" action. I could shell out to a no-op, but that's really silly for the standard case of "do nothing".
I guessed some basic cases:
IF EXISTS THEN alarm
IF EXIST THEN ALARM
So, is there a standard way to do IF IT DOES EXIST?
I recently was looking for the same solution as you and unfortunately, I was unable to discover a way of doing this in monit.
My situation differs slightly from yours so I ended up alarming if the file did not exist, and executed a shell script if it did. Like you, I did not want to spawn a shell just because the file did not exist, and having "file does not exist" show up in /var/log/messages isn't a big deal for me.
I know you said that you could shell out to a no-op so you probably don't need the following but I am adding it for those who might have the same issue and not know how to do it.
check file testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alarm
Note that you must exec /bin/bash to write the output of echo to /dev/null or monit will literally echo "dne > /dev/null"
Edit: As it was brought to my attention by disasteraverted, newer versions of Monit use alert rather than alarm, so the check would look like this:
check file testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alert
since monit 5.21.0, alterting on existence is directly supported:
check file testfile with path /path/to/file
if exist then alert
see in changelog https://mmonit.com/monit/changes/#5.21.0
Please check with :
check program not_exist_file_root_test with path "/bin/ls /root/test"
if status = 0 then alert
or
check program not_exist_file_root_test with path /bin/sh -c "test -f /root/test"
if status = 0 then alert
My 2 cents
renab, your check should end with "then alert" not "then alarm" at least in my version (5.2.5).
testfile with path /path/to/file
if not exist then exec "/bin/bash -c 'echo dne > /dev/null'" else if succeeded then alert
I'm trying to make a push notification work on my debian vps (apace2, mysql).
I use a php script from this tutorial (http://www.raywenderlich.com/3525/apple-push-notification-services-tutorial-part-2).
Basically, the script is put in an infintive loop, that check a mysql table for new records every couple of seconds. The tutorial says it should be run as a background process.
// This script should be run as a background process on the server. It checks
// every few seconds for new messages in the database table push_queue and
// sends them to the Apple Push Notification Service.
//
// Usage: php push.php development &
So I have four questions.
How do I start the script from the terminal? What should I type? The script location on the server is:
/var/www/development_folder/scripts/push2/push.php
How can I kill it if I need to (without having to restart apace)?
Since the push notification is essential, I need a way to check if the script is running.
The code (from the tutorial) calls a function is something goes wrong:
function fatalError($message)
{
writeToLog('Exiting with fatal error: ' . $message);
exit;
}
Maybe I can put something in there to restart the script? But It would also be nice to have a cron job or something that check every 5 minute or so if the script is running, and start it if it doens't.
4 - Can I make the script automatically start after a apace or mysql restart? If the server crash or something else happens that need a apace restart?
Thanks a lot in advance
You could run the script with the following command:
nohup php /var/www/development_folder/scripts/push2/push.php > /dev/null &
The nohup means that that the command should not quit (it ignores hangup signal) when you e.g. close your terminal window. If you don't care about this you could just start the process with "php /var/www/development_folder/scripts/push2/push.php &" instead. PS! nohup logs the script output to a file called nohup.out as default, if you do not want this, just add > /dev/null as I've done here. The & at the end means that the proccess will run in the background.
I would only recommend starting the push script like this while you test your code. The script should be run as a daemon at system-startup instead (see 4.) if it's important that it runs all the time.
Just type
ps ax | grep push.php
and you will get the processid (pid). It will look something like this:
4530 pts/3 S 0:00 php /var/www/development_folder/scripts/push2/push.php
The pid is the first number you'll see. You can then run the following command to kill the script:
kill -9 4530
If you run ps ax | grep push.php again the process should now be gone.
I would recommend that you make a cronjob that checks if the php-script is running, and if not, starts it. You could do this with ps ax and grep checks inside your shell script. Something like this should do it:
if ! ps ax | grep -v grep | grep 'push.php' > /dev/null
then
nohup php /var/www/development_folder/scripts/push2/push.php > /dev/null &
else
echo "push-script is already running"
fi
If you want the script to start up after booting up the system you could make a file in /etc/init.d (e.g. /etc.init.d/mypushscript with something like this inside:
php /var/www/development_folder/scripts/push2/push.php
(You should probably have alot more in this file)
You would also need to run the following commands:
chmod +x /etc/init.d/mypushscript
update-rc.d mypushscript defaults
to make the script start at boot-time. I have not tested this so please do more research before making your own init script!