Posix shell: distinguish between empty and not existing variable - variables

In pure /bin/sh how can I distinguish between an empty variable, an unset variable and a not existing (not defined) variable.
Here are the case:
# Case 1: not existing
echo "${foo}"
# Case 2: unset
foo=
echo "${foo}"
# Case 3: Empty
foo=""
echo "${foo}"
Now I would like to check for each of those three cases.
If case 2 and case 3 are actually the same, then I must at least be able to distinguish between them and case 1.
Any idea?
UPDATE
Solved thanks to Matteo
Here is how the code looks like:
#foo <-- not defined
bar1=
bar2=""
bar3="a"
if ! set | grep '^foo=' >/dev/null 2>&1; then
echo "foo does not exist"
elif [ -z "${foo}" ]; then
echo "foo is empty"
else
echo "foo has a value"
fi
if ! set | grep '^bar1=' >/dev/null 2>&1; then
echo "bar1 does not exist"
elif [ -z "${bar1}" ]; then
echo "bar1 is empty"
else
echo "bar1 has a value"
fi
if ! set | grep '^bar2=' >/dev/null 2>&1; then
echo "bar2 does not exist"
elif [ -z "${bar2}" ]; then
echo "bar2 is empty"
else
echo "bar2 has a value"
fi
if ! set | grep '^bar3=' >/dev/null 2>&1; then
echo "bar3 does not exist"
elif [ -z "${bar3}" ]; then
echo "bar3 is empty"
else
echo "bar3 has a value"
fi
And the results:
foo does not exist
bar1 is empty
bar2 is empty
bar3 has a value

I dont know about sh, but in bash and dash you can do echo ${TEST:?Error} for case 1 vs. case 2/3. And from quick glance at wikibooks, it seems like it should work for bourne shell too.
You can use it like this in bash and dash (use $? to get the error code)
echo ${TEST:?"Error"}
bash: TEST: Error
[lf#dell:~/tmp/soTest] echo $?
1
[lf#dell:~/tmp/soTest] TEST2="ok"
[lf#dell:~/tmp/soTest] echo ${TEST2:?"Error"}
ok
[lf#dell:~/tmp/soTest] echo $?
0
[lf#dell:~/tmp/soTest] dash
$ echo ${TEST3:?"Error"}
dash: 1: TEST3: Error
$ TEST3=ok
$ echo ${TEST3:?"Error"}
ok

You can use ${var?} syntax to throw an error if var is unset and ${var:?} to throw an error if var is unset or empty. For a concrete example:
$ unset foo
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
-bash: foo: unset
$ foo=
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
foo is empty
$ foo=bar
$ test -z "${foo?unset}" && echo foo is empty || echo foo is set to $foo
foo is set to bar

I don't see a correct answer here yet that is POSIX compliant. First let me reiterate William Pursell's assertion that the code foo= is indeed setting the variable foo to an empty value the same as foo="". For foo to be unset, it must either never be set or unset with unset foo.
Matteo's answer is correct, but there are caveats. When you run set in bash and posix mode is disabled, it also prints all of the defined functions as well. This can be suppressed like this:
isvar() (
[ -n "$BASH" ] && set -o posix
set | grep -q "^$1="
)
By writing it as a sub-shell function, we don't need to worry about what the state of bash's posix setting after we're done.
However, you can still get false-positives from variables whose values contain carriage returns, so understand that this is not 100% foolproof.
$ a="
> b=2
> "
$ set | grep ^b=
b=2
So for maximum correctness you can exploit bash's -v test, when available.
isvar() {
if [ -n "$BASH" ]; then
[ -v "$1" ]
else
set | grep -q "^$1="
fi
}
Perhaps somebody has a library somewhere that supports other shells' extensions as well. Essentially, this is a weakness in the POSIX specification and it just hasn't been seen as warranting amendment.

You can use set
If no options or arguments are specified, set shall write the names and values of all shell variables in the collation sequence of the current locale. Each name shall start on a separate line, using the format:
You can list all the variables (set) and grep for the variable name you want to check
set | grep '^foo='

Related

Bash purge script

I'm trying to create a script that removes my images that are not in DB
There is my code (Updated):
I have 1 problems:
Problem with the like syntax like '%$f%'
#!/bin/bash
db="intranet_carc_development"
user="benjamin"
for f in public/uploads/files/*
do
if [[ -f "$f" ]]
then
psql $db $user -t -v "ON_ERROR_STOP=1" \
-c 'select * from public.articles where content like "%'"$(basename "$f")"'%"' | grep . \
&& echo "exist" \
|| echo "doesn't exist"
fi
done
And I have the following error :
ERROR: column "%1YOLV3M4-VFb2Hydb0VFMw.png%" does not exist
LINE 1: select * from public.articles where content like "%1YOLV3M4-...
^
doesn't exist
ERROR: column "%wnj8EEd8wuJp4TdUwqrJtA.png%" does not exist
LINE 1: select * from public.articles where content like "%wnj8EEd8w...
EDIT : if i use \'%$f%\' for the like :
/purge_files.sh: line 12: unexpected EOF while looking for matching `"'
./purge_files.sh: line 16: syntax error: unexpected end of file
There are several issues with your code :
$f is public/uploads/files/FILENAME and i want only the FILENAME
You can use basename to circumvent that, by writing :
f="$(basename "$f")"
psql $db $user -c "select * from public.articles where content like '%$f%'"...
(The extra quotes are here to prevent issues if you have spaces and special characters in your file name)
your psql request will always return true even if no rows are found
your psql command will return true even if the request fails, unless you set the variable 'ON_ERROR_STOP' to 1
As shown in the linked questions, you can use the following syntax :
#!/bin/bash
set -o pipefail #needed because of the pipe to grep later on
db="intranet_carc_development"
user="benjamin"
for f in public/uploads/files/*
do
if [[ -f "$f" ]]
then
f="$(basename "$f")"
psql $db $user -t -v "ON_ERROR_STOP=1" \
-c "select * from public.articles where content like '%$f%'" | grep . \
&& echo "exist" \
|| echo "doesn't exist"
fi
done

How can i error out an entry if it already exists?

I'm writing a simple program to add a contact into a file called "phonebook", but if the contact already exists, i want it to return an echo saying " (first name last name) already exists", and not add it to the file. So far, i've gotten the program to add the user, but it wont return that echo and adds the duplicate entry anyway. How can i fix this?
#!/bin/bash
# Check that 5 arguments are passed
#
if [ "$#" -ne 5 ]
then
echo
echo "Usage: first_name last_name phone_no room_no building"
echo
exit 1
fi
first=$1
last=$2
phone=$3
room=$4
building=$5
# Count the number of times the input name is in add_phonebook
count=$( grep -i "^$last:$first:" add_phonebook | wc -l )
#echo $count
# Check that the name is in the phonebook
if [ "$count" -eq 1 ]
then
echo
echo "$first $last is already in the phonebook."
echo
exit 1
fi
# Add someone to the phone book
#
echo "$1 $2 $3 $4 $5" >> add_phonebook
# Exit Successfully
exit 0
Couple of things:
Should check if add_phonebook file exists before attempting to grep it, otherwise you get the grep: add_phonebook: No such file or directory output.
Your grep expression doesn't match the format of the file.
You are saving the file with space in between the fields, but searching with a colon(:) between the names. You can either update the file format to use a colon to separate the fields, or update the grep expression to search on space. In addition, you save first name, last_name, but search on last_name, first_name.
With space format:
count=$( grep -i "^$last[[:space:]]\+$first[[:space:]]" add_phonebook | wc -l )
Removed my tab separators from the echo line, used spaces, and now it can count properly

while loop only iterates once

I'm writing a unix script which does an awk and pipes to a while loop. For some reason, though, the while loop iterates only once. Can someone point out what I am missing?
awk '{ print $1, $2}' file |
while IFS=" " read A B
do
echo $B
if [ "$B" -eq "16" ];
then
grep -A 1 $A $1 | python unreverse.py
else
grep -A 1 $A
fi
done
"file" looks something like
cheese 2
elephant 5
tiger 16
Solution
The solution is to replace:
grep -A 1 $A
With:
grep -A 1 "$A" filename
Where filename is whatever file you intended grep to read from. Just guessing, maybe you intended:
grep -A 1 "$A" "$1"
I added double-quotes to prevent any possible word-splitting.
Explanation
The problem is that, without the filename, the grep command reads from and consumes all of standard input. It does this on the first run through the loop. Consequently, there is not input left for the second run and read A B fails and the loop terminates.
A Simpler Example
We can see the same issue happening with many fewer statements. Here is a while loop that is given two lines of input but only loops once:
$ { echo 1; echo 2; } | while read n; do grep "$n"; echo "n=$n"; done
n=1
Here, simply by adding a filename to the grep statement, we see that the while loop executes twice, as it should:
$ { echo 1; echo 2; } | while read n; do grep "$n" /dev/null; echo "n=$n"; done
n=1
n=2

Can you make it so that a bash command runs under certain conditions only?

I'm trying to do something like this (logic wise) but it's not working:
if (ls | wc -l ) >100; echo "too many files!"
else ls;
the point is adding this to my bashrc.
Any ideas?
Just an edit because I think I was slightly misunderstood. What I want is that when I type ls (or an alias that runs a modified ls) anywhere the files are only listed when there aren't a lot of them (I want something to add to my .bashrc). Being a bit of a moron, I sometimes type ls in directories where I have thousands of files so I'd like a way to circumvent that.
Rather than parsing ls, which is not best practice, you can do this with a bash array:
files=(*)
if ((${#files[#]} > 100)); then echo 'Too many files!'; else ls; fi
Probably in the actual problem you want a specific directory, and not the CWD; in that case, you might want something like this:
files=(/path/to/directory/*)
if ((${#files[#]} > 100)); then
echo 'Too many files!'
else (
cd /path/to/directory
ls
)
fi
Note that I've wrapped the cd into a parenthesized compound command, which means that the cd will be local. That assumes that you don't actually want the full path to appear in the output of ls.
You can do using find:
numFiles=$(find . -maxdepth 1 ! -name . -print0 | xargs -0 -I % echo . | wc -l)
(( numFiles > 100 )) && echo "too many files!" || ls
You can make this as function and put it in .bashrc
As others have pointed out, this is not an accurate way to count the number of files you have. It will miscount files that contain newlines for example and can have other issues.
It is, however, a perfectly good way to count the number of lines that ls will print and not show them if they're too many which is what you're presumably trying to do.
So, to answer your general question, to make one command depend on the result of another, you can use one of
command1 && command2
That will run command2 only if command1 was successful. If you want the second to be executed only if the first's results pass some test you can use:
[ command1 ] && command2
For your example, that would be:
[ $(ls | wc -l) -gt 100 ] && echo too many
To also execute ls again if the test is passed, use either
[ $(ls | wc -l) -gt 100 ] && echo too || ls
or
if [ $(ls | wc -l) -gt 200 ]; then echo 'too many files!'; else ls; fi
However, all of these are inelegant since they need to run the command twice. A better way might be to run the command once, save its output to a variable and then test the variable:
x=$(ls); [ $(wc -l <<<"$x") -gt 100 ] && echo 'too many!' || printf "%s\n" "$x"
Here, the output of ls is saved in the variable $x, then that variable is given as input to wc and if it has more than 100 lines, a message is printed. Else, the variable is.
For the sake of completeness, here's another safe approach that will count files correctly:
[ $(find -maxdepth 1 | grep -cF './') -gt 100 ] && echo 'too many!' || ls
A quick one liner:
test `find . -maxdepth 1 -type f|wc -l` -gt 100 && echo "Too Many Files"
A short one
[ $(ls | wc -l ) -gt 100 ] && echo too many
Combining some of the responses above - a simple Alias:
alias chkls='MAX=100 ; F=(*) ; if [[ ${#F[*]} -gt ${MAX} ]] ; then echo "## Folder: $(pwd) ## Too many files: ${#F[*]} ##" ; else ls ; fi '

How do you change a variable in a KSH if or case statement?

Does anyone know how to set a variable with global scope in a KSH if, case, or loop statement?
I am trying to run the following code but the script only echo's "H" instead of the actual value seen in the input file.
CFG_DIR=${WORK_DIR}/cfg
CFG_FILE=${CFG_DIR}/$1
NAME=$(echo $CFG_FILE | cut -f1 -d\.)
UPPER_BUS_NETWORK="H"
cat ${CFG_FILE} | grep -v ^\# |
while read CLINE
do
PROPERTY=$(echo $CLINE | cut -f1 -d\=)
VALUE=$(echo $CLINE | cut -f2 -d\=)
if [ ${PROPERTY} = "UpperBusService" ]; then
UPPER_BUS_SERVICE="${VALUE}"
fi
if [ ${PROPERTY} = "UpperBusNetwork" ]; then
UPPER_BUS_NETWORK="${VALUE}"
fi
done
echo ${UPPER_BUS_NETWORK}
Are you sure you're running that in ksh? Which version? Ksh93 doesn't set up a subshell in a while loop. Bash, dash, ash and pdksh do, though. I'm not sure about ksh88.
Compare
$ bash -c 'a=111; echo foo | while read bar; do echo $a; a=222; echo $a; done; echo "after: $a"'
111
222
after: 111
to
ksh -c 'a=111; echo foo | while read bar; do echo $a; a=222; echo $a; done; echo "after: $a"'
111
222
after: 222
Zsh gives the same result as ksh93.
Unfortunately, pdksh doesn't support process substitution and ksh93 does, but not when redirected into the done of a while loop, so the usual solution which works in Bash is not available. This is what it would look like:
# Bash (or Zsh)
while read ...
do
...
done < <(command)
Using a temporary file may be the only solution:
command > tmpfile
while read
do
...
done < tmpfile
Some additional notes:
Instead of cat ${CFG_FILE} | grep -v ^\# do grep -v ^\# "${CFG_FILE}"
Usually, you should use read -r so backslashes are handled literally
Instead of NAME=$(echo $CFG_FILE | cut -f1 -d\.) you should be able to do something like NAME=${CFG_FILE%%.*} and VALUE=${#*=}; VALUE=${VALUE%%=*}
Variables should usually be quoted on output, for example in each of your echo statements and your cat command
I recommend the habit of using lowercase or mixed case variable names to avoid conflict with shell variables (though none are present in your posted code)