In Hive can we pass parameter in where clause?
if yes, cloud you please explain me with one scenario?
ex:In sql
select * from mytable where col= ?
Yes, you can.
Here are several examples:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+VariableSubstitution
Here is a specific example, using a shell script, which is a very common place to use variable substitution:
#!/usr/bin/env bash
if [ "$#" -eq 1 ]; then
WHEREVAR=$1
hive -e "SELECT * FROM myDB.myTable where myFirstField=${WHEREVAR};"
else
echo "Illegal number of parameters"
Related
The case is that I have an SQL clause inside a unix script like:
sqlplus -s user/pass << END_SQL1 >> outfile.txt
set echo off feedback off heading off tab off;
select .....
from ....
where ...
and ...
and ... ;
END_SQL
If the outfile.txt is not empty, which means that I get a result from the above SQL, then I am running an update SQL that should change something at some DB elements.
Then I need to reuse the same SQL above to check if the DB elements that I wanted have changed indeed. So, is that possible to reuse this same SQL, but WITHOUT including this same SQL code again later at the script, instead to run it again and, moreover, even put the result at another output file, e.g. outfile2.txt ?
You can use RETURNING ... INTO ... clause inside the script
UPDATE myTable
SET col1 = <something1>
WHERE col2 = <something2>
RETURNING col3, col1 INTO v_col3, v_col1;
to return the results into the variables v_col3 and v_col1.
You could put your hairy SELECT query in a file, say select.sql. Then whenever you need to run the SQL, you could just do :
sqlplus -s user/pass #select.sql >> outfile.txt
You can adapt the output file as you wish :
sqlplus -s user/pass #select.sql >> outfile2.txt
NB : you said
If the outfile.txt is not empty, which means that I get a result from the above SQL
You probably want to use > when writing to outfile.txt : >> appends to the file, while > replaces it.
I've got a param which is like "This is a param", and I'm going to pass it to below hiveQL:
hive -hivevar sys_nm="This is a param" -e 'select * from rd_sys where rd_sys_nm=${hivevar:sys_nm}'
But Hive returned below error message:
Logging initialized using configuration in jar:file:/opt/mapr/hive/hive-0.13/lib/hive-common-0.13.0-mapr-1409.jar!/hive-log4j.properties
FAILED: ParseException line 1:49 missing EOF at 'is' near 'This'
g4t7491_[mgr#g4t7491 ~]$
Does anyone know how to pass it normally?
Hive var don't work like hiveconf where you need to apply "hiveconf:somthing" in the code
when declaring hivevar just add the var name like this -> ${var_name}
for example:
through command line:
hive -hivevar MONTH_VAR='11' -e "select * from table where month=${MONTH_VAR};"
you can also declair through the script:
set hivevar:MONTH_VAR=11;
-- so query would look like this (no hiveconf):
set hivevar:MONTH_VAR=11;
SELECT * from table where month=${MONTH_VAR};
You need to put the string in single quotes for it to parse correctly as a string inside the sql after interpolation.
hive -hivevar sys_nm="'This is a param'" -e 'select * from rd_sys where rd_sys_nm=${hivevar:sys_nm}'
I have two scripts. One is named sqlscript.sql and the other is named script.sh I have all of the queries needed written in my sql script. They are just a bunch of update statements. For example:
UPDATE xxDev.SYS_PARAMS SET val = 'serverName' WHERE lower(name) = 'enginebaseurl';
I'm running the .sql script IN the .sh script. When the .sh script runs, I want it to prompt the user for a server name and take that user input and replace it in serverName in the sql statements.
I'm brand new to both bash scripting and this website, so I hope I'm making sense asking this question. I'm using PuTTY if that makes a difference at all.
Suppose you use MySQL, try something like:
# TODO: prompt user for server name and store it into variable serverName
serverName="get from user"
cat <<"EOF" | mysql -u user1 -p passwd -h server1 -P 3306 -D db1
UPDATE xxDev.SYS_PARAMS SET val = '$serverName' WHERE lower(name) = 'enginebaseurl';
EOF
So in this example, you embed the sql script into the .sh so that you don't have to maintain two files.
I would probably use a variable
set #val 'serverName'
UPDATE xxDev.SYS_PARAMS SET val = #val WHERE lower(name) = 'enginebaseurl';
You can split the sqlscript.sql into
set-val.sql
set #val 'serverName'
and the actual update statements. Then you can recreate the set-val.sql from your user input:
echo -n "enter server: "
read server
echo "set #val '$server' > set-val.sql
and then you forward both files to mysql:
cat set-val.sql sqlscript.sql | mysql
You should probably use this only for internal things, it seems a little fragile.
I'm going let you figure out how to pass a shell parameter into your sql command, but here's an incredibly cool way to query the user for the server name. It might even be POSIX compliant.
#!/bin/sh
echo -n "Hit me with that server name: "; read serverName
echo "${serverName}! Outstanding! Pick up \$200 when you pass Go!"
I'm hoping someone can help with applying the output from a db2 command to a variable to use later on in a script.
So far I am at...
db2 "connect to <database> user <username> using <password>"
while read HowMany ;
do
Counter=$HowMany
echo $HowMany
done < <(db2 -x "SELECT COUNT(1) FROM SYSCAT.COLUMNS WHERE TABNAME = 'TableA' AND TABSCHEMA='SchemaA' AND GENERATED = 'A'")
When trying to reference $Counter outside of the while loop, it returns SQL1024N A database connection does not exist. SQLSTATE=08003 as does the echo $HowMany
I've tried another method using pipe, which makes the $HowMany show the correct value, but as that is a sub shell, it's lost afterwards.
I'd rather not use temp files and remove them if possible as I don't like left over files if scripts abort at any time.
The DB2 CLP on Linux and UNIX can handle command substitution without losing its database connection context, making it possible to capture query results into a local shell variable or treat it as an inlined block of text.
#!/bin/sh
# This script assumes the db2profile script has already been sourced
db2 "connect to <database> user <username> using <password>"
# Backtick command substitution is permitted
HowMany=`db2 -x "SELECT COUNT(1) FROM SYSCAT.COLUMNS WHERE TABNAME = 'TableA' AND TABSCHEMA='SchemaA' AND GENERATED = 'A'"`
# This command substitution syntax will also work
Copy2=$(db2 -x "SELECT COUNT(1) FROM SYSCAT.COLUMNS WHERE TABNAME = 'TableA' AND TABSCHEMA='SchemaA' AND GENERATED = 'A'")
# One way to get rid of leading spaces
Counter=`echo $HowMany`
# A while loop that is fed by process substitution cannot use
# the current DB2 connection context, but combining a here
# document with command substitution will work
while read HowMany ;
do
Counter=$HowMany
echo $HowMany
done <<EOT
$(db2 -x "SELECT COUNT(1) FROM SYSCAT.COLUMNS WHERE TABNAME = 'TableA' AND TABSCHEMA='SchemaA' AND GENERATED = 'A'")
EOT
As you have found, a DB2 connection in one shell is not available to sub-shells. You could use a sub-shell, but you'd have to put the CONNECT statement in that sub-shell.
So it's more of a simple rewrite, and don't use a sub-shell:
db2 "connect to <database> user <username> using <password>"
db2 -x "SELECT COUNT(1) FROM SYSCAT.COLUMNS WHERE TABNAME = 'TableA' AND TABSCHEMA='SchemaA' AND GENERATED = 'A'" | while read HowMany ; do
Counter=$HowMany
echo $HowMany
done
I'm trying to load a sql from a file in bash and execute the loaded sql. The sql file needs to be versatile, meaning it cannot be altered in order to make things easy while being run in bash (escaping special characters like * )
So I have run into some problems:
If I read my sample.sql
SELECT * FROM SAMPLETABLE
to a variable with
ab=`cat sample.sql`
and execute it
db2 `echo $ab`
I receive an sql error because by doing a cat the * has been replaced by all the files in the directory of sample.sql.
Easy solution would be to replace "" with "\" . But I cannot do this, because the file needs to stay executable in programs like DB Visualizer etc.
Could someone give me hint in the right direction?
The DB2 command line processor has options that accept a filename as input, so you shouldn't need to load statements from a text file into a shell variable.
This command will execute all SQL statements in the file, with newline treated as the statement terminator:
db2 -f sample.sql
This command will execute all SQL statements in the file, with semicolon treated as the statement terminator:
db2 -t -f sample.sql
Other useful CLP flags are:
-x : Suppress the column headings
-v : Echo the statement text immediately before execution
-z : Tee a copy of all CLP output to the filename immediately following this flag
Redirect stdin from the file.
db2 < sample.sql
In case, you have a variable used in your script and wanted to get it replaced by the shell before executed in DB2 then use this approach:
Contents of File.sql:
cat <<xEOF
insert values(1,2) into ${MY_SCHEMA}.${MY_TABLE};
select * from ${MY_SCHEMA}.${MY_TABLE};
xEOF
In command prompt do:
export MY_SCHEMA='STAR'
export MY_TAVLE='DIMENSION'
Then you are all good to get it executed in DB2:
eval File.sq |db2 +p -t
The shell will replace the global variables and then DB2 will execute it.
Hope it helps.