How to run a SQL command within a Bash script and save the output of that command to a variable - sql

I am looking to enclose some Oracle components within a Bash script that will perform a set of goals:
Log into a remote server (where my Oracle DB resides) as root.
Performs an "su - oracle".
Logs into sqlplus environment as a specific Oracle user.
Performs an SQL select command and stores the output of that command into a variable.
Displays the result of that variable in the Bash shell.
I have looked through a couple examples here on stackoverflow, many of which seem to go over executing a command but not necessarily detailing how to display the output to the user (although I am still examining a few more). For example, assuming all key exchanges are setup beforehand, a method could be to use the following:
#!/bin/sh
ssh -q root#5.6.7.8
sqlplus ABC/XYZ#core <<ENDOFSQL
select CREATE_DATE from PREPAID_SUBSCRIBER where MSISDN='12345678912';
exit;
ENDOFSQL
Instead, here is how I tried to set this up:
#!/bin/sh
datasource_name=`echo "select CREATE_DATE from PREPAID_SUBSCRIBER where MSISDN='12345678912';" | ssh -q 5.6.7.8 "su - oracle -c 'sqlplus -S ABC/XYZ#core'" | tail -2 | head -1`
Ideally, the datasource_name variable should now either take on values:
no rows selected
Or if there is an entry within the table:
CREATE_DATE
-------------------
07-06-2009 18:04:48
The tail and head commands are to get rid of the empty lines in the output, and the ssh -q and sqlplus -S options are for ignoring warnings.
However, when I run that command, and do an:
echo "${datasource_name}"
I get...
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
...instead of one of the two outputs above. If I understand correctly, this is a warning that can be caused depending on whether a specific shell is used, but most online sources indicate that this can be ignored. The nice thing about this warning is that it appears my command above is actually running and storing "something" into datasource_name, but it just isn't what I want.
Now to simplify this problem, I noticed I get the same tty warning when I simply try to su to oracle on the remote machine from the box where the bash script runs:
ssh root#5.6.7.8 "su - oracle"
Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
If I do the following, I actually get into the sqlplus environment successfully with no issues:
ssh -q root#5.6.7.8 "su - oracle -c 'sqlplus ABC/XYZ#core'"
SQL*Plus: Release 9.2.0.4.0 - Production on Tue May 29 12:35:06 2012
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to: Oracle Database 10g Enterprise Edition Release
10.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, OLAP, Data Mining and Real Application Testing options
SQL>
If I understand why the problem above is occurring, it is possible that I can figure out how to get my script to work properly. Any suggestions would be greatly appreciated! Thank you.

change the first line to:
ssh -t root#5.6.7.8 "su - oracle"
to get a tty to see if that would work for you.
another thing you can do in your script is to redirect stderr to your variable as well if you would like to see that as well in your variable, which does not appear to be the case for you, though I have done so in the past in some cases. There is an example in the comments below.

This is a sample script for MySQL, but it can be easily edited for Oracle :
#!/bin/bash
remote=oracle#5.6.7.8
ssh -q -t $remote <<EOF
bash <<EOFBASH
mysql <<ENDOFSQL>/tmp/out
show databases;
ENDOFSQL
EOFBASH
EOF
scp $remote:/tmp/out /tmp/out
ds=$(</tmp/out)
cat <<EOF
START OUTPUT
$ds
END OUTPUT
EOF
rm /tmp/out
Tested, works well. Instead of using su - oracle, try to ssh directly to oracle user ;)

Related

mysql command displays `-?` help instead of `-e` executing command when run from bash script (but not when run by hand)

I have a rather simple bash script.
#!/bin/bash
echo
echo What is the root statement for the new debate?
read body
echo
echo What is your mysql password?
read -s pass
echo
sql='INSERT
INTO `Debate` (`unblocked`, `debaterId`, `dirty` )
VALUES ('"'1'"', '"'15'"', '"'0'"' );
INSERT
INTO `Statement` (`body`, `debateId`, `debaterId` )
VALUES ('"'"${body}"'"', LAST_INSERT_ID(), '"'15'"' );'
#echo mysql -u resolution -p ${pass} -D resolution -e \"${sql//[^a-zA-Z0-9(),;\`\'_]/ }\"
mysql -u resolution -p ${pass} -D resolution -e \"${sql//[^a-zA-Z0-9(),;\`\'_]/ }\"
What this is meant to do should be pretty straight forward but it comes down to "ask user for input on terminal... form sql command with the provided input... insert into database in two tables." It's meant to be a quick hack for adding new debates into the database for testing while development is ongoing.
Instead, mysql prints the help as if I had used -?. No error... just the help text. Everything looks correct when the mysql command is echoed to the terminal. And if I copy and paste the echoed command it works just fine. I've searched google and stack overflow but found nothing about this.
U#H ~ mysql --version
mysql Ver 15.1 Distrib 10.3.28-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

How to supress the output of .sql script when placed inside bash script

Below is my simple bash script:
#!/bin/bash
cd /app/oracle/client11_2/
connect <<EOF
#/app/oracle/client11_2/testquery.sql
exit;
EOF
Note: testquery.sql contains some "SELECT" queries and also i have used spool to store the content of the queries.
But, when i execute the bash script on the terminal, it produces lot of unwanted output like below:
SQL*Plus: Release 11.2.0.3.0
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
How to avoid this output..!? I do not want this to be printed on the output screen when i execute the bash script.
You can suppress the SQL*Plus header messages by using the -s flag
sqlplus -s
That said, your snippet shows a connect but not the sqlplus invocation itself. Not sure where that is being hidden.
Make the here doc (<<) snippet as:
connect <<EOF >/dev/null
#/app/oracle/client11_2/testquery.sql
exit;
EOF
this sends the STDOUT of the here doc to /dev/null.
You can also do the output redirection at run time, sending the whole STDOUT of the script to /dev/null e.g.:
./script.sh >/dev/null
again this sends the whole STDOUT of the script to /dev/null (YMMV).

Unable to run a postgresql script from bash

I am learning the shell language. I have creating a shell script whose function is to login into the DB and run a .sql file. Following are the contents of the script -
#!/bin/bash
set -x
echo "Login to postgres user for autoqa_rpt_production"
$DB_PATH -U $POSTGRESS_USER $Auto_rpt_production$TARGET_DB -p $TARGET_PORT
echo "Running SQL Dump - auto_qa_db_sync"
\\i auto_qa_db_sync.sql
After running the above script, I get the following error
./autoqa_script.sh: 39: ./autoqa_script.sh: /i: not found
Following one article, I tried reversing the slash but it didn't worked.
I don't understand why this is happening. Because when I try manually running the sql file, it works properly. Can anyone help?
#!/bin/bash
set -x
echo "Login to postgres user for autoqa_rpt_production and run script"
$DB_PATH -U $POSTGRESS_USER $Auto_rpt_production$TARGET_DB -p $TARGET_PORT -f auto_qa_db_sync.sql
The lines you put in a shell script are (moreless, let's say so for now) equivalent to what you would put right to the Bash prompt (the one ending with '$' or '#' if you're a root). When you execute a script (a list of commands), one command will be run after the previous terminates.
What you wanted to do is to run the client and issue a "\i ./autoqa_script.sh" comand in it.
What you did was to run the client, and after the client terminated, issue that command in Bash.
You should read about Bash pipelines - these are the way to run programs and input text inside them. Following your original idea to solving the problem, you'd write something like:
echo '\i auto_qa_db_sync.sql' | $DB_PATH -U $POSTGRESS_USER $Auto_rpt_production$TARGET_DB -p $TARGET_PORT
Hope that helps to understand.

postgresql pgBench tool running user defined SQL script

Please help me clarify, if the pgbench tool can execute my own sql scenarios in parallel way?
Googling and local searching brought no positive result.
I run the script that execeutes with no errors. But after execution I see no signs, that my script was actually performed.
Does pgbench commits transaction with my sql script?
That's an output I get:
C:\Program Files\PostgreSQL\9.2\bin>pgbench.exe -n -h dbserverhost -U postgres -
T 10 -c 64 -j 8 bench_dbname -f c:\Dev\bench_script.sql
transaction type: TPC-B (sort of)
scaling factor: 1
query mode: simple
number of clients: 64
number of threads: 8
duration: 10 s
number of transactions actually processed: 1020
tps = 95.846561 (including connections establishing)
tps = 103.387127 (excluding connections establishing)
C:\Program Files\PostgreSQL\9.2\bin>
SQL script bench_script.sql is:
--comment here
begin;
insert into schm.log values ( 'pgbench test', current_timestamp );
end;
SOLUTION
pgBench Windows version is sensitive to the order of the arguements passed to the utility:
"bench_dbname" argument must be the last one parameter in a line.
This is the correct example of pgbench Windows version command line:
pgbench.exe -d -r -h 127.0.0.1 -U postgres -T 5 -f C:\Dev\bench_script.sql -c 64 -j 8 postgres
The most useful arguments for me were:
-T 60 (time in seconds to run script)
-t 100 (transaction amount per client)
-d print detailed debug info to the output
-r include in summary latency value calculated for every action of the script
-f run user defined sql script in benchmark mode
-c client amount
-j thread amount
pgBench official doc
PgBench, I love you! :)
Best wishes everybody ;)
The "transaction type: TPC-B (sort of)" means that it did not process the -f option to run your custom sql script, instead it ran the default query.
On Windows versions, getopt seems to stop parsing the options once it reaches the first one that does not start with a hyphen, i.e. "bench_dbname". So make sure -f comes before that.
I guess you also need the -n option as long as you are using your custom script?
-n
--no-vacuum
Perform no vacuuming before running the test.
This option is necessary if you are running a custom test scenario
that does not
include the standard tables pgbench_accounts, pgbench_branches,
pgbench_history, and pgbench_tellers.

How do you execute SQL from within a bash script?

I have some SQL scripts that I'm trying to automate. In the past I have used SQL*Plus, and called the sqlplus binary manually, from a bash script.
However, I'm trying to figure out if there's a way to connect to the DB, and call the script from inside of the bash script... so that I can insert date and make the queries run relative to a certain number of days in the past.
I'm slightly confused. You should be able to call sqlplus from within the bash script. This may be what you were doing with your first statement
Try Executing the following within your bash script:
#!/bin/bash
echo Start Executing SQL commands
sqlplus <user>/<password> #file-with-sql-1.sql
sqlplus <user>/<password> #file-with-sql-2.sql
If you want to be able to pass data into your scripts you can do it via SQLPlus by passing arguments into the script:
Contents of file-with-sql-1.sql
select * from users where username='&1';
Then change the bash script to call sqlplus passing in the value
#!/bin/bash
MY_USER=bob
sqlplus <user>/<password> #file-with-sql-1.sql $MY_USER
You can also use a "here document" to do the same thing:
VARIABLE=SOMEVALUE
sqlplus connectioninfo << HERE
start file1.sql
start file2.sql $VARIABLE
quit
HERE
Here is a simple way of running MySQL queries in the bash shell
mysql -u [database_username] -p [database_password] -D [database_name] -e "SELECT * FROM [table_name]"
Maybe you can pipe SQL query to sqlplus. It works for mysql:
echo "SELECT * FROM table" | mysql --user=username database
I've used the jdbcsql project on Sourceforge.
On *nix systems, this will create a csv stream of results to standard out:
java -Djava.security.egd=file///dev/urandom -jar jdbcsql.jar -d oracledb_SID -h $host -p 1521 -U some_username -m oracle -P "$PW" -f excel -s "," "$1"
Note that adding the -Djava.security.egd=file///dev/urandom increases performance greatly
Windows commands are similar: see http://jdbcsql.sourceforge.net/
If you do not want to install sqlplus on your server/machine then the following command-line tool can be your friend. It is a simple Java application, only Java 8 that you need in order to you can execute this tool.
The tool can be used to run any SQL from the Linux bash or Windows command line.
Example:
java -jar sql-runner-0.2.0-with-dependencies.jar \
-j jdbc:oracle:thin:#//oracle-db:1521/ORCLPDB1.localdomain \
-U "SYS as SYSDBA" \
-P Oradoc_db1 \
"select 1 from dual"
Documentation is here.
You can download the binary file from here.
As Bash doesn't have built in sql database connectivity... you will need to use some sort of third party tool.