For some downloads lgogdownloader return always the exit code 141 after a successful download. Because of this the dlagent should catch this exit code.
I have already try some dlagents, but non of this works:
DLAGENTS+=('gogdownloader::/usr/bin/lgogdownloader --download-file=%u -o %o || /usr/bin/test $? -eq 141')
Error: unrecognised option '-eq'
DLAGENTS+=('gogdownloader::/usr/bin/bash -c \"lgogdownloader --download-file=%u -o %o || test $? -eq 141\"')
--download-file=gogdownloader://2146639313/en3installer0: -c: line 0: unexpected EOF while looking for matching `"'
--download-file=gogdownloader://2146639313/en3installer0: -c: line 1: syntax error: unexpected end of file
Your problem is that you incorrectly escape the " inside a ' string. This reproduces your 2nd error message:
$ eval '/usr/bin/bash -c \"echo hello world\"'
hello: -c: line 0: unexpected EOF while looking for matching `"'
hello: -c: line 1: syntax error: unexpected end of file
The 1st occurence of \" is treated as that literal sequence, while the 2nd occurence escapes the quotation, leaving you with an never-ending string. Removing the escape, you get the desired result:
$ eval '/usr/bin/bash -c "echo hello world"'
hello world
How msrd0 has mention you can use a separate script:
DLAGENTS+=("gogdownloader::./catch_gogdownloader_error141.sh %u %o")
#!/bin/bash
lgogdownloader --download-file=$1 -o $2 || test $? -eq 141
But I think this is more a workaround than a real solution.
Related
I can convert hexadecimal number stored in a variable to decimal number in shell using:
$ x=08BA
$ echo $x | awk '{print strtonum( "0x" $1 )}'
2234
But when i writing awk script in a file to perform same task using following code giving error.
program:
x=08BA
system("echo $x | print strtonum( "0x" $1 )");
Error message is:
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
kindly provide solution
I'm trying to create a script that removes my images that are not in DB
There is my code (Updated):
I have 1 problems:
Problem with the like syntax like '%$f%'
#!/bin/bash
db="intranet_carc_development"
user="benjamin"
for f in public/uploads/files/*
do
if [[ -f "$f" ]]
then
psql $db $user -t -v "ON_ERROR_STOP=1" \
-c 'select * from public.articles where content like "%'"$(basename "$f")"'%"' | grep . \
&& echo "exist" \
|| echo "doesn't exist"
fi
done
And I have the following error :
ERROR: column "%1YOLV3M4-VFb2Hydb0VFMw.png%" does not exist
LINE 1: select * from public.articles where content like "%1YOLV3M4-...
^
doesn't exist
ERROR: column "%wnj8EEd8wuJp4TdUwqrJtA.png%" does not exist
LINE 1: select * from public.articles where content like "%wnj8EEd8w...
EDIT : if i use \'%$f%\' for the like :
/purge_files.sh: line 12: unexpected EOF while looking for matching `"'
./purge_files.sh: line 16: syntax error: unexpected end of file
There are several issues with your code :
$f is public/uploads/files/FILENAME and i want only the FILENAME
You can use basename to circumvent that, by writing :
f="$(basename "$f")"
psql $db $user -c "select * from public.articles where content like '%$f%'"...
(The extra quotes are here to prevent issues if you have spaces and special characters in your file name)
your psql request will always return true even if no rows are found
your psql command will return true even if the request fails, unless you set the variable 'ON_ERROR_STOP' to 1
As shown in the linked questions, you can use the following syntax :
#!/bin/bash
set -o pipefail #needed because of the pipe to grep later on
db="intranet_carc_development"
user="benjamin"
for f in public/uploads/files/*
do
if [[ -f "$f" ]]
then
f="$(basename "$f")"
psql $db $user -t -v "ON_ERROR_STOP=1" \
-c "select * from public.articles where content like '%$f%'" | grep . \
&& echo "exist" \
|| echo "doesn't exist"
fi
done
I am trying to run select statement within case statement in Unix Shell script, but getting unexpected end of file error.
I want to run particular select statement depending on the output of previous sql script ran in shell script. The output from previous sql script is spooled to a log, required pattern is fetched into a variable, which is used in case statement.
My script
#!/usr/bin/sh
exec > check_company_details.log 2>&1
sqlplus username/password#database << EOF
#check_company_details.sql $1
exit;
EOF
pool=$(cat company.log | grep dbPool | awk {'print $5'})
#everything is working till above steps
#if sqlplus command is removed from below case statements, correct output of echo is returned.
case $pool in
dbpool1)
echo "DBPool is POOL1"
sqlplus username/password#database<<EOF
select name from v\$database;
exit;
EOF
;;
dbpool2)
echo "DBPool is POOL1"
sqlplus username/password#database<<EOF
select name from v\$database;
exit;
EOF
;;
dbpool3)
echo "DBPool is DC4POOL1"
sqlplus username/password#database<<EOF
select name from v\$database;
exit;
EOF
;;
*)
echo No Results
;;
esac
Error message:
*./check_company_details.sh: line 37: syntax error: unexpected end of file*
A here doc end string should not have leading whitespace. This means you should rewrite
dbpool3)
echo "DBPool is DC4POOL1"
sqlplus username/password#database<<EOF
select name from v\$database;
exit;
EOF
as
dbpool3)
echo "DBPool is DC4POOL1"
sqlplus username/password#database<<EOF
select name from v\$database;
exit;
EOF
and the same goes for the other cases.
You should also say fgrep dbPool company.log instead of needlessly using cat and instead of using grep when you are not feeding in a regex. You also have the quotes around your awk script in a weird place; it works but it's not what it should be.
pool=$(cat company.log | grep dbPool | awk {'print $5'})
becomes
pool=$(fgrep dbPool company.log | awk '{print $5}')
You should not expand $pool without quoting it, e.g. it should be case "$pool" in. Even if you think it won't have spaces in the variable you should do this for safety.
You should get in to the habit of checking all of your shell scripts with shellcheck whether they work or not.
I think you don't require a case block. You could use an if else statement with pool variable.
if [ "$pool" = "dbpool1" ] || [ "$pool" = "dbpool2" ] || [ "$pool" = "dbpool3" ]
then
echo "DBPool is ${pool}"
sqlplus username/password#database<<EOF
select name from v\$database;
exit
EOF
else
echo "No Results"
fi
I have following variables:
STR1="CD45RA_naive"
STR2="CD45RO_mem"
STR3="CD127_Treg"
STR4="IL17_Th_stim_MACS"
STR5="IL17_Th17_stim"
names=($STR1 $STR2 $STR3 $STR4 $STR5)
and the files with the same names as variables with a ".bed" extension.
in each file I need to add a following header:
track name='enhancer region' description='${names[$i]}'"
My idea was the following:
for ((i=0; i<${#names[#]}; i++))
do
echo "output/annotation/"${names[$i]}".bed"
sed -i '' "1 i \track name='enhancer region' description='"${names[$i]}"'" "output/annotation/"${names[$i]}".bed"
done
However I get an error:
sed: 1: "1 i \track name='enhanc ...": extra characters after \ at the end of i command
What is a proper way to add a corresponding header to each file (gawk,sed)?
You typically need the line that is inserted for the i command to be on its own line. In other words, you need to add a newline after the backslash. For example:
$ cat input
line 1
$ sed -e '1i\
foo' input
foo
line 1
$ sed -e '1i\ foo' input
sed: 1: "1i\ foo
": extra characters after \ at the end of i command
This is my hadoop job:
hadoop streaming \
-D mapred.map.tasks=1\
-D mapred.reduce.tasks=1\
-mapper "awk '{if(\$0<3)print}'" \ # doesn't work
-reducer "cat" \
-input "/user/***/input/" \
-output "/user/***/out/"
this job always fails, with an error saying:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `export TMPDIR='..../work/tmp'; /bin/awk { if ($0 < 3) print } '
But if I change the -mapper into this:
-mapper "awk '{print}'"
it works without any error. What's the problem with the if(..) ?
UPDATE:
Thank #paxdiablo for your detailed answer.
what I really want to do is filter out some data whose 1st column is greater than x, before piping the input data to my custom bin. So the -mapper actually looks like this:
-mapper "awk -v x=$x{if($0<x)print} | ./bin"
Is there any other way to achieve that?
The problem's not with the if per se, it's to do with the fact that the quotes have been stripped from your awk command.
You'll realise this when you look at the error output:
sh: -c: line 0: `export TMPDIR='..../work/tmp'; /bin/awk { if ($0 < 3) print } '
and when you try to execute that quote-stripped command directly:
pax> echo hello | awk {if($0<3)print}
bash: syntax error near unexpected token `('
pax> echo hello | awk {print}
hello
The reason the {print} one works is because it doesn't contain the shell-special ( character.
One thing you might want to try is to escape the special characters to ensure the shell doesn't try to interpret them:
{if\(\$0\<3\)print}
It may take some effort to get the correctly escaped string but you can look at the error output to see what is generated. I've had to escape the () since they're shell sub-shell creation commands, the $ to prevent variable expansion, and the < to prevent input redirection.
Also keep in mind that there may be other ways to filter depending on you needs, ways that can avoid shell-special characters. If you specify what your needs are, we can possibly help further.
For example, you could create an shell script (eg, pax.sh) to do the actual awk work for you:
#!/bin/bash
awk -v x=$1 'if($1<x){print}'
then use that shell script in the mapper without any special shell characters:
hadoop streaming \
-D mapred.map.tasks=1 -D mapred.reduce.tasks=1 \
-mapper "pax.sh 3" -reducer "cat" \
-input "/user/***/input/" -output "/user/***/out/"