How to check response coming from a command while using SSHOperator?
t1 = SSHOperator(ssh_conn_id='conn_box2',
task_id='t1',
command='Rscript /code/demo.R',
do_xcom_push=True,
response_check=lambda response: True if "status:200" in response.text else False,
dag=dag
)
My R scripts returns status:200 if the execution goes well. And I want to track it. My task t1 should only complete if status is 200.
If R script returns status:300 its a failed one. But since the execution is completed without any error in UI task turns into green(which i don't want)
I code above is able to capture the response in xcom, but how do i validate it?
Try the following code:
bash_command = """
set -e;
Rscript /code/demo.R | grep 'status:200' &> /dev/null
if [ $? == 0 ]; then
echo "Task Successful"
else
echo "Task Failed"
exit 1
fi
"""
t1 = SSHOperator(ssh_conn_id='conn_box2',
task_id='t1',
command=bash_command,
dag=dag)
Alternatively, you can also use the following bash_command:
if Rscript /code/demo.R | grep -q 'status:200'; then
echo "Task Successful"
else
echo "Task Failed"
exit 1
fi
The SSHOperator does not have response_check parameter.
Airflow is unable to interpret exit command
[2021-09-07 06:36:58,164] {ssh.py:142} INFO - ps_count is 23, There might be some processes are running
[2021-09-07 06:36:58,169] {taskinstance.py:1455} ERROR - SSH operator error: error running cmd:
code:
set -e;
ps_count=jpsexec | grep -v execute | wc -l
if [ $ps_count -ne 0 ]
then
echo "ps_count is $ps_count, There might be some processes are running"
exit 1
else
echo "All processed were stopped..!"
fi
I have gammu-smsd up and running on a raspberry-pi with jessie. I am using runonreceive to process incoming texts. I have the following script working using runonreceive. In the script I am calling gammu sendsms instead of gammu-smsd-inject as the documentation states. All other references state gammu will not work while gammu-smsd daemon is running. The only reason I got this to work is after pulling my hair out trying to get gammu-smsd-inject to work. Can anyone explain what is going on?
RunOnReceive = /home/jaalfs/bin/sms_back.sh
sms_back.sh
#!/bin/bash
from=$SMS_1_NUMBER
echo "sms_back" >> /home/jaalfs/bin/sms_back.log
echo "Test from: $from" >> /home/jaalfs/bin/sms_back.log
echo -e "\n"
if [ "$from" != "+1310xxxxxxx" ]; then
echo -e "not accepted number \n" >> /home/jaalfs/bin/sms_back.log
exit 0
else
echo "accepted number" >> /home/jaalfs/bin/sms_back.log
echo "hello world!!!!!!" | sudo gammu sendsms TEXT "$from"
echo -e " text sent back \n" >> /home/jaalfs/bin/sms_back.log
exit 0
fi
exit 1
In POSIX awk, how do I get the exit status (return code) from command after processing its output via command | getline var? I want my awk script to exit 1 if command exited with a non-zero exit status.
For example, suppose I had an awk script named foo.awk that looks like this:
function close_and_get_exit_status(cmd) {
# magic goes here...
}
BEGIN {
cmd = "echo foo; echo bar; echo baz; false"
while ((cmd | getline line) > 0)
print "got a line of text: " line
if (close_and_get_exit_status(cmd) != 0) {
print "ERROR: command '" cmd "' failed" | "cat >&2"
exit 1
}
print "command '" cmd "' was successful"
}
then I want the following to happen:
$ awk -f foo.awk
got a line of text: foo
got a line of text: bar
got a line of text: baz
ERROR: command 'echo foo; echo bar; echo baz; false' failed
$ echo $?
1
According to the POSIX specification for awk, command | getline returns 1 for successful input, zero for end-of-file, and -1 for an error. It's not an error if command exits with a non-zero exit status, so this can't be used to see if command is done and has failed.
Similarly, close() can't be used for this purpose: close() returns non-zero only if the close fails, not if the associated command returns a non-zero exit status. (In gawk, close(command) returns the exit status of command. This is the behavior I'd like, but I think it violates the POSIX spec and not all implementations of awk behave this way.)
The awk system() function returns the exit status of the command, but as far as I can tell there's no way to use getline with it.
The simplest thing to do is just echo the exit status from shell after the command executes and then read that with getline. e.g.
$ cat tst.awk
BEGIN {
cmd = "echo foo; echo bar; echo baz; false"
mod = cmd "; echo \"$?\""
while ((mod | getline line) > 0) {
if (numLines++)
print "got a line of text: " prev
prev = line
}
status = line
close(mod)
if (status != 0) {
print "ERROR: command '" cmd "' failed" | "cat >&2"
exit 1
}
print "command '" cmd "' was successful"
}
$ awk -f tst.awk
got a line of text: foo
got a line of text: bar
got a line of text: baz
ERROR: command 'echo foo; echo bar; echo baz; false' failed
$ echo $?
1
In case anyone's reading this and considering using getline, make sure you read http://awk.freeshell.org/AllAboutGetline and FULLY understand all the caveats and implications of doing so first.
Not an ideal solution, but you can do:
"command || echo failure" | getline var; ... if( var == "failure" ) exit;
There is some ambiguity in that you have to select the string "failure" in such a way that command can never generate the same string, but perhaps this is an adequate workaround.
The following is horrifically complicated, but it:
is POSIX conformant (mostly -- fflush() isn't yet in the POSIX standard, but it will be and it's widely available)
is general (it works no matter what kind of output is emitted by the command)
does not introduce any processing delay. The accepted answer to this question makes a line available only after the next line has been printed by the command. If the command slowly outputs lines and responsiveness is important (e.g., occasional events printed by an IDS system that should trigger a firewall change or email notification), this answer might be more appropriate than the accepted answer.
The basic approach is to echo the exit status/return value after the command completes. If this last line is non-zero, exit the awk script with an error. To prevent the code from mistaking a line of text output by the command for the exit status, each line of text output by the command is prepended with a letter that is later stripped off.
function stderr(msg) { print msg | "cat >&2"; }
function error(msg) { stderr("ERROR: " msg); }
function fatal(msg) { error(msg); exit 1; }
# Wrap cmd so that each output line of cmd is prefixed with "d".
# After cmd is done, an additional line of the format "r<ret>" is
# printed where "<ret>" is the integer return code/exit status of the
# command.
function safe_cmd_getline_wrap(cmd) {
return \
"exec 3>&1;" \
"ret=$(" \
" exec 4>&1;" \
" { ( "cmd" ) 4>&-; echo $? >&4; } 3>&- |" \
" awk '{print\"d\"$0;fflush()}' >&3 4>&-;" \
");" \
"exec 3>&-;" \
"echo r${ret};"
}
# like "cmd | getline line" except:
# * if getline fails, the awk script exits with an error
# * if cmd fails (returns non-zero), the awk script exits with an
# error
# * safe_cmd_getline_close(cmd) must be used instead of close(cmd)
function safe_cmd_getline(cmd, wrapped_cmd,ret,type) {
wrapped_cmd = safe_cmd_getline_wrap(cmd)
ret = (wrapped_cmd | getline line)
if (ret == -1) fatal("failed to read line from command: " cmd)
if (ret == 0) return 0
type = substr(line, 1, 1)
line = substr(line, 2)
if (type == "d") return 1
if (line != "0") fatal("command '" cmd "' failed")
return 0
}
function safe_cmd_getline_close(cmd) {
if (close(safe_cmd_getline_wrap(cmd))) fatal("failed to close " cmd)
}
You use the above like this:
cmd = "ls no-such-file"
while (safe_cmd_getline(cmd)) {
print "got a line of text: " line
}
safe_cmd_getline_close(cmd)
If you have mktemp command, you could store the exit status in a temporary file:
#!/bin/sh
set -e
file=$(mktemp)
finish() {
rm -f "$file"
}
trap 'finish' EXIT
trap 'finish; trap - INT; kill -s INT $$' INT
trap 'finish; trap - TERM; kill $$' TERM
awk -v file="$file" 'BEGIN{
o_cmd="echo foo; echo bar; echo baz; false"
cmd = "("o_cmd "); echo $? >\""file"\""
print cmd
while ((cmd | getline) > 0) {
print "got a line of text: " $0
}
close(cmd)
getline ecode <file; close(file)
print "exit status:", ecode
if(ecode)exit 1
}'
I've got a problem with a bash script in which I'd like to execute SQL.
As an example if I simply write :
sqlplus -s << EOF
${USER}/${PASSWD}#DataBase
show user;
exit;
it works.
But as soon as I put it into a loop it doesn't work anymore.
For example :
while (condition)
do
echo $ANSWER
read -p '[y/n]' ANSWER
echo $ANSWER
if [ $ANSWER = 'y' ]
then
sqlplus -s << EOF
${USER}/${PASSWD}#DataBase
show user;
exit;
EOF
break
elif [ $ANSWER = 'n' ]
then
break
fi
done
echo $ANSWER
And the results I've got is : line 26: syntax error : unexpected end of file
(knowing that the line "echo $ANSWER" is the line 25...)
If anyone has an idea about why it doesn't want to work I will be really thankful for the help !
Your problem is that the shell expects the delimiter for the here document at the beginning of the line.
Your EOF is in the middle of the line, and therefore it isn't recognized as a delimiter anymore.
This works as expected:
while (true)
do
echo $ANSWER
read -p '[y/n]' ANSWER
echo $ANSWER
if [ $ANSWER = 'y' ]
then
sqlplus -s << EOF
${USER}/${PASSWD}#DataBase
show user;
exit;
EOF
break
elif [ $ANSWER = 'n' ]
then
break
fi
done
echo $ANSWER
I read this stackoverflow question...
Bash: check user input is correct
which does most of what I want however rather then checking it's just an integer I need to check it's an integer in a variable range....
The script looks for files in a directory and then assigns a number to them...
File 1
File 2
File 3
etc....
The user chooses the the number and the script then executes commands against that file.....the variable $FILELIST is the total number of files.
Taking the example from the previous stackoverflow I tried.....
FILENUM=""
while [[ ! ($FILENUM =~ ^[0-$FILELIST]+$) ]]; do
echo " "
echo "Please enter the file number: "
read -p "1 - $FILELIST" FILENUM < /dev/tty
done
echo "$FILENUM"
However this is throwing a syntax error: unexpected "(" (expecting "do") in the while line and I'm not sure why, I suspect $FILELIST has to be bracketed somehow but an explanation as to why the above works would help me understand the problem.
Thanks
bash-specific answers:
You don't need to reinvent the wheel: use the select builtin:
cd /path/to/directory
PS3="Select a file: "
select file in *; do
if [[ $file ]]; then break; fi
done
echo "You selected '$file'"
echo "You selected file number $REPLY"
To check a number is within a certain range, I'd write:
if (( 0 <= $number && $number <= $max )); then echo "in range"; fi
Since you're using ash you might use this as a reference: http://manpages.debian.net/cgi-bin/man.cgi?query=dash
while true; do
FILENUM=""
echo
echo "Please enter the file number: "
read -p "1 - $FILELIST" FILENUM < /dev/tty
if expr "$FILENUM" : '[0-9]\+$' &&
[ $FILENUM -gt 0 ] &&
[ $FILENUM -le $FILELIST ]
then
break
fi
done
echo "$FILENUM"