Read one line at a time using shell - sql

Question

I'm trying to create a shell script. It will read one line at a time, assign the values to macros, then run a query with macros. After query is done, it will read the second line, create macros, run queries...
I developed the code below, but it will read all lines together then run the query. Should I use do i=1 to n ?
#!/bin/sh
$HOME/.profile
file=$1
OutputTable=$2
file=rule_flg.txt
cat $file|(
read flg table_num rule_num
while test "$flg" != ""
do
echo table_num is $table_num and rule_num is $rule_num
echo time is `date`
(here are some parameters of database)... -v flg=$flg -v table_num=$table_num -v rule_num=$rule_num -f query_1.sql &> query_1.log
read flg table_num rule_num
done
)
echo run finished!
exit 0

Answer 1

Instead of what you describe, what about reading the file line by line with the while read; do ... done < file syntax?
This way, every iteration will just contain the data from the current line.
while IFS= read -r flg table_num rule_num
echo "table_num is $table_num and rule_num is $rule_num"
echo "time is `date`"
(here are some parameters of database)... -v flg=$flg -v table_num=$table_num -v rule_num=$rule_num -f query_1.sql &> query_1.log
done < "$file"
You can find more details by reading How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?.
Note also that it is good to quote whatever variable you are working with, to prevent problems on formatting, etc. So say echo "$var" instead of echo $var unless you are very sure you don't want it.

Answer 2

So your question seems to be - "the script I wrote will read all the lines together then run the query, but I want it to execute the query once for each line it reads"
Since your script isn't complete or testable, I can't really debug it. :)
It is, however, a bit overly complex. Since you dont need to get variables or other shell data out side of the subshell, you can just do a cat $file | while read vars; do echo $vars;done type of thing.
the while read will read each line of input into a variable (or set of variables), and execute the inner part of the loop once for each line.
A hacked up version of your script to show it:
#!/bin/sh
file=rule_flg.txt
echo "1 2 3" > $file
echo "3 4 5" >> $file
echo "4 5 6" >> $file
cat $file | while read flg table_num rule_num
do
echo table_num is $table_num and rule_num is $rule_num
#do your other things here
done
produces
%% sh whileread.sh
table_num is 2 and rule_num is 3
table_num is 4 and rule_num is 5
table_num is 5 and rule_num is 6
Now, there are many other ways to approach this - and there's no handling for the case where you send more than 3 arguments to read (they'll get stuffed into $rule_num...) but assuming your input data is consistently whitespace/tab delimited into 3 columns it should be a good start.

Answer 3

I would take another approach here. As awk is part of POSIX, I dare to ignore your requirement of shell.
General pattern:
awk 'prog' inputfile | sh
So you write the repetitive statements with awk. When you are satisfied with the result, you pipe it to sh.
If you'd like some specifics, please post a sample of your input data and the commands that you want to execute in the end.