Using WHERE with multiple columns with different data types to satisfy a single input in bash and postgressql - sql

please assist with the following. i m trying to run a script that accepts one argument $1. The argument can either be a string or character or an integer. I want to use the argument in there where clause to search for the element in the database.
This is the table i want to search from:enter image description here
When i use the multiple conditions with OR , it works only when either the argument is a number or text.
This what my code looks like enter image description here
`
ELEMENT=$($PSQL "SELECT * FROM elements e FULL JOIN properties p USING(atomic_number) WHERE symbol = '$1' OR name = '$1' OR atomic_number = $1;")
`
and this is the results i get when i run with different aurgumentsenter image description here
Please help.
Thank you in advance

This will always fail on any non-numeric argument.
You are passing in H for hydrogen, but taking whatever was passed in and using it in the atomic_number comparison as an unquoted number, which the DB engine is trying to figure out what to do with. H isn't a number, and isn't a quoted string, so it must be the name of a column...but it isn't, so you are using invalid syntax.
I don't have a postgres available right now, but try something like this -
ELEMENT=$( $PSQL "
SELECT *
FROM elements e
FULL JOIN properties p USING(atomic_number)
WHERE symbol = '$1'
OR name = '$1'
OR atomic_number = CAST(`$1` as INTEGER); " )
Also, as an aside... avoid all-capital variable names.
As a convention, those are supposed to be system vars.
And please - please don't embed images except as helpful clarification.
Never rely on them to provide info if it can be avoided. Copy/paste actual formatted text people can copy/paste in their own testing.

An alternate way to construct the query: requires bash
looks_like_a_number() {
# only contains digits
[[ "$1" == +([[:digit:]]) ]]
}
sanitize() {
# at a minimum, handle embedded single quotes
printf '%s' "${1//\'/\'\'}"
}
if looks_like_a_number "$1"; then
field="atomic_number"
value=$1
elif [[ ${#1} -eq 1 ]]; then
field="symbol"
printf -v value "'%s'" "$(sanitize "$1")"
else
field="name"
printf -v value "'%s'" "$(sanitize "$1")"
fi
q="SELECT *
FROM elements e
FULL JOIN properties p USING(atomic_number)
WHERE $field = $value;"
printf '%s\n' "$q"
result=$("$PSQL" "$q")

Related

Raku vs. Perl5, an unexpected result

When I run this script in Raku I get the letter A with several newlines.
Why do I not get the concatenated strings as expected (and as Perl5 does) ?
EDIT BTW can I compile in commaIDE the file with the Perl5 compiler, where can I change this compiler to be that of Perl5 ?
say "A";
my $string1 = "00aabb";
my $string2 = "02babe";
say join ("A", $string1, $string2);
print "\n";
my #strings = ($string1, $string2);
say join ("A", #strings);
print "\n";
The solution I suggest is to drop the parens:
say "A";
my $string1 = "00aabb";
my $string2 = "02babe";
say join "A", $string1, $string2; # Pass THREE arguments to `join`, not ONE
print "\n";
my #strings = $string1, $string2;
say join "A", #strings;
print "\n";
(The elements in #strings in the second call to join are flattened, thus acting the same way as the first call to join.)
The above code displays:
A
00aabbA02babe
00aabbA02babe
When I run this script in Raku I get the letter A with several newlines.
Your code calls join with ONE argument, which is the joiner, and ZERO strings to concatenate. So the join call generates null strings. Hence you get blank lines.
Why do I not get the concatenated strings
The two say join... statements in your code print nothing but a newline because they're like the third and fourth say lines below:
say join( " \o/ ", "one?", "two?" ); # one? \o/ two?␤
say join " \o/ ", "one?", "two?" ; # one? \o/ two?␤
say join ( " \o/ ", "one?", "two?" ); # ␤
say join( " \o/ one? two?" ); # ␤
The first and second lines above pass three strings to join. The third passes a single List which then gets coerced to become a single string (a concatenation of the elements of the List joined using a single space character), i.e. the same result as the fourth line.
The my #strings = ($string1, $string2); incantation happens to work as you intended, because an assignment to a "plural" variable on the left hand side of the = will iterate the value, or list of values, on the right hand side.
But it's a good habit in Raku to err on the side of avoiding redundant code, in this instance only using parens if you really have to use them to express something different from code without them. This is a general principle in Raku that makes code high signal, low noise. For your code, all the parens are redundant.
The problem is the whitespace after the function name. If you leave a space there, raku will treat the expression in paranthesis as a List which is single thing. Here join will think you want to use the list as a joiner.
Observe:
>raku -e "sub foo( $arg ) { say $arg.WHAT }; foo( 'abc', )"
(Str)
>raku -e "sub foo( $arg ) { say $arg.WHAT }; foo ( 'abc', )"
(List)
So in short, if you want to use paranthesis to call a sub, don't put a whitespace between the sub name and the opening paren.
see Traps to avoid,
try:
my #s=("00aabb", "02babe");
say join "A", #s;
say join |("A",#s);
say join("A",#s);
say #s.join: "A";
#or some crazy
say join\ ("A",#s);
say "A".&join: #s;
say join #s: "A";
say "A" [&join] #s;
say #s R[&join] "A";
#…
A: "do not put whitespace between a function name and the opening paren '('" ♎️KISS

Arguments mismatch using where IN clause in query

I have column in hive table like below
testing_time
2018-12-31 14:45:55
2018-12-31 15:50:58
Now I want to get the distinct values as a variable so I can use in another query.
I have done like below
abc=`hive -e "select collect_set(testing_time)) from db.tbl";`
echo $abc
["2018-12-31 14:45:55","2018-12-31 15:50:58"]
xyz=${abc:1:-1}
when I do
hive -e "select * from db.tbl where testing_time in ($xyz)"
I get below error
Arguments for IN should be the same type! Types are {timestamp IN (string, string)
what the the mistake I am doing?
What is the correct way of achieving my result?
Note: I know I can use subquery for this scenario but I would like to use variable to achieve my result
Problem is that you're comparing timestamp (column testing_time) with string (i.e. "2018-12-31 14:45:55"), so you need to convert string to timestamp, which you can do via TIMESTAMP(string).
Here's a bash script that adds the conversion:
RES="" # here we will save the resulting SQL
IFS=","
read -ra ITEMS <<< "$xyz" # split timestamps into array
for ITEM in "${ITEMS[#]}"; do
RES="${RES}TIMESTAMP($ITEM)," # add the timestamp to RES variable,
# surrounded by TIMESTAMP(x)
done
unset IFS
RES="${RES%?}" # delete the extra comma
Then you can run the constructed SQL query:
hive -e "select * from db.tbl where testing_time in ($RES)"

checking if a variable value is null on bash script

i have this script that retrieves a parameter (a number) from a SQL query and assigns it to a variable.
there are two options - either the SQL query finds a value and then the script preforms
echo "the billcycle number is $v_bc", or it doesnt find a value and it suppose to
echo "no billcycle parameter found".
im having a problem with the if condition.
this is what i came up with:
#!/bin/bash
v_bc=`sqlplus -s /#bscsprod <<EOF
set pagesize 0
select billcycle from bc_run
where billcycle not in (50,16)
and control_group_ind is null
and billseqno=6043;
EOF`
if [ -z "$v_bc" ]; then echo no billcycle parameter found
else echo "the billcycle parameter is $v_bc"
fi
when billseqno=6043, then it means that v_bc=25, and when i run the script, the result is:
"the billcycle parameter is 25". which is what i ment it to do.
when i set billseqno=6042, according to the above SQL query, v_bc will get no value, therefore what i want it to do is echo "no billcycle parameter found".
instead i get
"the billcycle parameter is
no rows were selected".
any suggestions ?
thanks very much
Assaf.
Your code is correctly checking for an empty value -- v_bc is just not empty, even with -s.
You may either:
parse the output of sqplus when no rows are returned, so add this:
if [[ "$v_bc" == "no rows were selected" ]]; then v_bc=""; fi
This uses the bash [[ ]] command with "==" for pattern matching so we don't need to worry about leading/trailing whitespace. This is not as robust as dogbane's SET FEEDBACK OFF since it's entirely possible for "no rows were selected" to be valid data.
write a better query which always returns data, like this:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:594023455752#followup-221571500346463844
The trick being to reformulate your query and use select/union with a fallback query that conditionally provides output when your query is empty:
with data as
(select billcycle from bc_run where [...])
select * from data
union all
select 'NA', null from dual where not exists (select null from data);
(see also What is the dual table in Oracle? )
Try turning feedback off in sqlplus so that you don't get any output if no rows are selected:
v_bc=`sqlplus -s /#bscsprod <<EOF
SET FEEDBACK OFF
set pagesize 0
select billcycle from bc_run
where billcycle not in (50,16)
and control_group_ind is null
and billseqno=6043;
EOF`
Try something like
if [ "${v_bc:-SHOULDNTHAPPEN}" = "SHOULDNTHAPPEN" ]; then
echo no billcycle parameter found
else......
The idiomatic way to assign a value to a variable if that variable is empty is with an = in a parameter expansion. In other words, this:
: ${v_bc:=no billcycle paramter found}
is equivalent to:
test -z "$v_bc" && v_bc='no billcycle paramter found'
In your case, it would also be easy to do:
echo ${v_bc:-no billcycle parameter found}
but the question asked in the title is not really the problem in your case. (Since the problem is not that v_bc is the empty string, but rather that it is a value you do not expect.)

Passing psql query from perl into string

Currently I have a perl script that accesses our database, performs certain queries and prints output to the terminal. Instead, I would like to output the results into a template latex file before generating a pdf. For most of my queries I pull out numbers and store these as scalar variables (eg how often a particular operator carries out a given task). eg.
foreach $op (#operator) {
$query = "SELECT count(task_name) FROM table WHERE date <= '$date_stop' and
date >= '$date_start' and task=\'$operator[$index]\';";
#execute query
$result=$conn->exec($query);
$conres = $conn->errorMessage;
if ($result->resultStatus eq PGRES_TUPLES_OK) {
if($result->ntuples > 0) {
($task[$index]) = $result->fetchrow;
}
printf("$operator[$index] carried out task: %d\n", $task[$index]);
} else {
die "Failed.\n$conres\n\n";
exit -1;
}
$index++;
}
printf("**********************************\n\n");
In the final report I will summarise how many times each operator completed each task in a table. In addition to this there will also be some incidents which must be reported. I can print these easily to the terminal using a command such as
$query = "SELECT operator, incident_type from table_name WHERE incident_type = 'Y'
and date <= '$date_stop' and date >= '$date_start';";
$result=$conn->exec($query);
$conres = $conn->errorMessage;
if ($result->resultStatus eq PGRES_TUPLES_OK) {
if($result->ntuples > 0) {
$result->print(STDOUT, 1, 1, 0, 0, 0, 1, "\t", "", "");
}
} else {
die "Failed.\n$conres\n\n";
exit -1;
}
An example of the output of this command is
operator | incident_type
-----------------------------
AB | Incomplete due to staff shortages
-------------------------------
CD | Closed due to weather
-----------------------------
How can I make my perl script pass the operator names and incidents into a string array rather than just sending the results to the terminal?
You should consider updating your script to use DBI. This is the standard for database connectivity in Perl.
DBI has a built in facility for inserting parameters into a query string. It is safer and faster than manually creating the string yourself. Before the loop, do this once:
#dbh is a database handle that you have already opened.
my $query = $dbh->prepare(
"SELECT count(task_name) FROM table WHERE date<=? and date>=? and task=?"
);
Then within the loop, you only have to do this each time:
$query->execute($date_stop,$date_start,$op);
Note that the parameters you pass to execute automatically get inserted in place of the ?'s in your statement. It handles the quoting for you.
Also in the loop, after you execute the statement, you can get the results like this:
my $array_ref = $query->fetchall_array_ref;
Now all of the rows are stored in a two-dimensional array structure. $array_ref->[0][0] would get the first column of the first row returned.
See the DBI documentation for more information.
As others have mentioned, there are quite a few other mistakes in your code. Make sure you start with use strict; use warnings;, and ask more questions if you need further help!
Lots of good feedback to your script, but nothing about your actual question.
How can I make my perl script pass the operator names and incidents into a string array rather than just sending the results to the terminal?
Have your tried creating an array and pushing items to it?
my #array;
push (#array, "foo");
Or using nested arrays:
push (#array, ["operator", "incident"]);

How can I use the COUNT value obtained from a call to mkqlite()?

I'm using mksqlite to create and access an SQL database from matlab, and I want to get the number of rows in a table. I've tried this:
num = mksqlite('SELECT COUNT(*) FROM myTable');
, but the returned value isn't very helpful. If I put a breakpoint in my script and examine the variable, I find that it's a struct with a single field, called 'COUNT(_)', which seems to actually be an invalid name for a field, so I can't access it:
K>> class(num)
ans =
struct
K>> num
num =
COUNT(_): 0
K>> num.COUNT(_)
??? num.COUNT(_)
|
Error: The input character is not valid in MATLAB statements or expressions.
K>> num.COUNT()
??? Reference to non-existent field 'COUNT'.
K>> num.COUNT
??? Reference to non-existent field 'COUNT'.
Even the MATLAB IDE can't access it. If I try to double click the field in the variable editor, this gets spat out:
??? openvar('num.COUNT(_)', num.COUNT(_));
|
Error: The input character is not valid in MATLAB statements or expressions.
So how can I access this field?
You are correct that the problem is that mksqlite somehow manages to create an invalid field name that can't be read. The simplest solution is to add an AS clause to your SQL so that the field has a sensible name:
>> num = mksqlite('SELECT COUNT(*) AS cnt FROM myTable')
num =
cnt: 0
Then to remove the extra layer of indirection you can do:
>> num = num.cnt;
>> num
num =
0