How can I insert the content of the variable into single quotes inside the INSERT INTO command? - sql

I created a text file. The name of this is "test.txt" and the content is first part below. I also created script with the name insert.sh.
I run the command with ./insert.sh test.txt.
If the words / strings are in single quotes, it will insert the words into the columns. Also it will insert numbers without single quotes. The csv that I will eventually use won't have single quotes and I don't want to change the data.
How can I insert the content of the variable into single quotes inside the INSERT INTO command?
I am using psql.
Text file, test.txt
'one','ten','hundred'
'two','twenty','twohundred'
Script, insert.sh:
#!/bin/bash
while read cell
do
name=$cell
echo "$cell"
####Insert from txt into table####
sudo -u username -H -- psql -d insert_test -c "
INSERT INTO first (ten, hundred, thousend) VALUES ($cell);
"
done < $1
something like this:
INSERT INTO first (ten, hundred, thousend) VALUES (INSERT" $cell "QUOTES);
UPDATE:
I changed the code. I added the single quotes around $cell as you suggested.
#!/bin/bash
while read cell
do
name=$cell
echo "$cell"
####Insert from txt into table####
sudo -u username -H -- psql -d insert_test -c "
INSERT INTO first (ten, hundred, thousend) VALUES ('$cell');
"
done < $1
and I removed the quotes out of the text file since the csv file that I want to use later wont have any single quotes.
new text file.
one,ten,hundred
two,twenty,twohundred
and im getting the error:
one,two,three
ERROR: INSERT has more target columns than expressions
LINE 2: INSERT INTO first (ten, hundred, thousend) VALUES ('one,two,...

You need to modify the $IFS (Internal Field Separator) variable to determine the line separator used by Bash. Since you used a CSV like file, you IFS come to , character, thus this is the result $IFS=,. Note that if you need to do others stuff in you script, you need to redefine the $IFS var to the original state, so you need to store it in an temportal variable before you change it, something like $OLDIFS=$IFS.
readline read the entire line and separate the values depending on $IFS var, thus you need to write the adecauted quantity of var where readline will store the words, i.e., if you line have 3 words, you need to give 3 vars to readline, e.g.: file: foo,baz,bar, readline -r word1 word2 word3. If you don't give the correct amount of vars, readline will store the rest of word in a single var, that is your problem.
So, a solution to your problem would be:
#!/bin/bash
$OLDIFS=$IFS # If you need to do more stuff.
while IFS=, read -r word1 word2 word3
do
sudo -u username -H -- psql -d insert_test -c
"INSERT INTO first (ten, hundred, thousend) VALUES (${word1}, ${word2}, ${word3});"
done < $1
$IFS=$OLDIFS # Same of line 2.
# ...
NOTE: This is insecure because lead with easily to a SQL injection. If you use this, only use in a local database that don't have any sensetive data.

Related

Store my "Sybase" query result /output into a script variable

I need a variable to keep the results retrieved from a query (Sybase) that´s in a script.
I have built the following script, it works fine I get the desired result when I run it
Script: EXECUTE_DAILY:
isql -U database_dba -P password <<EOF!
select the_name from table_name where m_num="NUMB912" and date="17/01/2019"
go
quit
EOF!
echo "All Done"
Output:
"EXECUTE_DAILY" 97 lines, 293 characters
user#zp01$ ./EXECUTE_DAILY
the_name
-----------------------------------
NAME912
(1 row affected)
But now I would like to keep the output(the_name: NAME912) in a variable.
So far this is basically what I'm trying with no success.
variable=$(isql -U database_dba -P password -se "select the_name from table_name where m_num="NUMB912" and date="17/01/2019" ")
But, is not working. I can't save NAME912 in a variable.
You need to parse the output for the desired string/piece-of-data that you wish to store in your variable. I tend to make my life a bit easier by making sure I can easily/quickly search/parse out what I want.
Keeping a few issues in mind ...
I tend to use isql -s"|" -w10000 to ensure (most of the time) that a) the result set has all columns delimited with the pipe ('|') and b) a single row of data does not span multiple rows; the pipe delimiter makes it easier to parse out columns that may contain white space; obviously (?) use a different delimiter if a pipe may be part of your actual data
to make parsing of the isql output a bit easier I tend to add a unique, grep-able (literal) string to the rows that I'm looking to search/parse
some databases (eg, SQLAnywhere, Oracle) tend to mimic a literal value as the column header if said literal string has not been assigned an explicit alias/header; this means that if you do a simple search on your literal string then you'll get a match for the result set header as well as the actual data row
I tend to capture all isql output to a temporary file; this allows for easier follow-on processing, eg, error checking, data parsing, dumping contents to a logfile, etc
So, with the above in mind my code typically looks something like:
$ outfile=/tmp/.$$.isql.outfile
$ isql -s"|" -w10000 -U database_dba -P password <<-EOF > ${outfile} 2>&1
-- 'GREP'||'ME' ensures that 'GREPME' only shows up in the data row
select 'GREP'||'ME',the_name
from table_name
where m_num = "NUMB912"
and date = "17/01/2019"
go
EOF
$ cat ${outfile}
... snip ...
|'GREP'||'ME'|the_name | # notice the default column header = 'GREP'||'ME' which won't match my search for 'GREPME'
|------------|----------|
|GREPME |NAME912 | # this is the line I want to search/parse
... snip ...
$ read -r namevar < <(egrep GREPME ${outfile} | awk -F"|" '{print $3}')
$ echo ${namevar}
NAME912

Executing the SQL from shell scripting

I have a table called query_master table which has 4 columns and the 4th column has SQL query as values. In total there are 5 entries in the query table.
Table Structure:
S.No --> Key --> Title --> Query
1 100 EG select * from dual
Now my objective is, I have to fetch the SQL queries using shell script from the query_master and execute it. The output of that each SQL query should be written on a separate log file, and the log filename should be equal to the name of the title.
Can you please help in achieving this scenario using stored procedures or stored functions which will be more helpful for me.
I need to achieve this using shell scripting.
Try this, assuming you're using mysql:
awk -F'\t' 'NR!=1 {system("mysql -u user -p -e " $4 " database")}' file
Where file is the file containing the table, user is the user and database is the database. Alternatively set these as variables instead of hard coding them like this:
awk -F'\t' -v db="database" -v user="user" 'NR!=1 {system(""mysql -u " user " -p -e " $4 " " db)}' file
Make a shell script that accepts a SQL statement from commandline (or inputfile or stdin) and does all things for you like exporting ORACLE_HOME, tnsnames, username, password, redirecting output, calling sqlplus, output formatting, deleting column headers and other sqlplus settings.
With your magicsql.sh (after testing), aim for a solution like
magicsql.sh "select key, query from query_master order by key" | while read key query; do
magicsql.sh "${query}" > /tmp/${key}.out
done

How can I update a single field in sqlite3 with the contents of a file?

This is equivalent to my earlier question here, but for sqlite.
As before, I am trying to do the following using the sqlite3 command line client.
UPDATE my_table set my_column=CONTENT_FROM_FILE where id=1;
I have looked at the documentation on .import, but that seems to be a little heavyweight for what I am trying to do.
What is the correct way to set the value of one field from a file?
The method I seek should not impose constraints on the contents of the file.
Assuming the file content is all UTF-8 text and doesn't have any quote characters that would be misinterpreted, you could do this (assuming posix shell - on Windows try cygwin):
$ echo "UPDATE my_table set my_column='" >> temp.sql
$ cat YourContentFile >> temp.sql
$ echo "' where id=1;" >> temp.sql
$ sqlite3
SQLite version 3.7.13 2012-07-17 17:46:21
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .read temp.sql
If the content does have single quotes, escape them first with a simple find-and-replace (you'd need to do that anyway).
hth!
See: http://www.sqlite.org/cli.html#fileio
sqlite> INSERT INTO images(name,type,img)
...> VALUES('icon','jpeg',readfile('icon.jpg'));
In your case:
UPDATE my_table set my_column=readfile('yourfile') where id=1;
If you don't have readfile, you need to .load the module first.
Note
I found that the provided fileio module: http://www.sqlite.org/src/artifact?ci=trunk&filename=ext/misc/fileio.c uses sqlite3_result_blob. When I use it in my project with text columns, it results in Chinese characters being inserted into the table rather than the bytes read from file. This can be fixed by changing it to sqlite3_result_text. See http://www.sqlite.org/loadext.html for instructions on building and loading run-time extensions.

How to delete last row in output file generated by nzsql

I am trying to delete last row in the file generated by nzsql.Please find the below query.
nzsql -A -c "SELECT * FROM AM_MAS_DIVISION_DIM" > abc.out
When I execute this query the output will be generated and stored in abc.out.This will include both header columns as well as some time information at the bottom.But I don't need the bottom metadata and want to keep only my header columns. How can I do this using only nzsql.Please help me.Thanks in advance.
use -r flag in the nzsql command to avoid getting that row [assuming the metadata referred in question is the row count summary line, ex: (3 rows)]
-r Suppresses the row count that is displayed at the end of the SQL output.
reference: http://pic.dhe.ibm.com/infocenter/ntz/v7r0m3/index.jsp?topic=%2Fcom.ibm.nz.adm.doc%2Fr_sysadm_nzsql_command.html
Why don't you just pipe the output to a unix command to remove it? I think something like this will work:
nzsql -A -c "SELECT * FROM AM_MAS_DIVISION_DIM" | sed '$d' > abc.out
Seems to be a recommended solution for getting rid of the last line (although ed, gawk, and other tools can handle it).

Execute SQL from file in bash

I'm trying to load a sql from a file in bash and execute the loaded sql. The sql file needs to be versatile, meaning it cannot be altered in order to make things easy while being run in bash (escaping special characters like * )
So I have run into some problems:
If I read my sample.sql
SELECT * FROM SAMPLETABLE
to a variable with
ab=`cat sample.sql`
and execute it
db2 `echo $ab`
I receive an sql error because by doing a cat the * has been replaced by all the files in the directory of sample.sql.
Easy solution would be to replace "" with "\" . But I cannot do this, because the file needs to stay executable in programs like DB Visualizer etc.
Could someone give me hint in the right direction?
The DB2 command line processor has options that accept a filename as input, so you shouldn't need to load statements from a text file into a shell variable.
This command will execute all SQL statements in the file, with newline treated as the statement terminator:
db2 -f sample.sql
This command will execute all SQL statements in the file, with semicolon treated as the statement terminator:
db2 -t -f sample.sql
Other useful CLP flags are:
-x : Suppress the column headings
-v : Echo the statement text immediately before execution
-z : Tee a copy of all CLP output to the filename immediately following this flag
Redirect stdin from the file.
db2 < sample.sql
In case, you have a variable used in your script and wanted to get it replaced by the shell before executed in DB2 then use this approach:
Contents of File.sql:
cat <<xEOF
insert values(1,2) into ${MY_SCHEMA}.${MY_TABLE};
select * from ${MY_SCHEMA}.${MY_TABLE};
xEOF
In command prompt do:
export MY_SCHEMA='STAR'
export MY_TAVLE='DIMENSION'
Then you are all good to get it executed in DB2:
eval File.sq |db2 +p -t
The shell will replace the global variables and then DB2 will execute it.
Hope it helps.