I need to pass a sql script as an argument to a bash file. My bash file is as below
#!/bin/bash
if [ $# -lt 3 ]; then
echo "Usage: databasename, tablename, script"
exit 1
fi
export DB=$1
export PWD=absc
/research/util/Unload.py -e -d "|" -t $2 -c "MM/DD/YYYY" -s $3
I execute the bash file in this way:
./UnloadToInfx.sh "db_name" "mytable" "select * from mytable where city='LA' and date='05/22/2018'"
It prop up error regrading to $3 which is the sql script. How can I resolve this problem?
Change your script like this (encapsulate bare $1,2,3 references to "$1", "$2" etc.) The reason is that encapsulating them will present them as a unit for the program that you give the argument to.
Otherwise a string with spaces will be presented as multiple arguments and any glob characters like *? will be expanded to filenames in your current directory, which you don't want.
Even so, take care that your arguments don't start with a (-) hyphen, because then they might still be interpreted as option switches .
#!/bin/bash
if [ $# -lt 3 ]; then
echo "Usage: databasename, tablename, script"
exit 1
fi
export DB="$1"
export PWD=absc
/research/util/Unload.py -e -d "|" -t "$2" -c "MM/DD/YYYY" -s "$3"
Related
I have a .bat file that contains two commands:
SQLCMD -S . -d "databaseName" -E -i "path_to_query1.sql" -y0 -s "|" -f o:65001 > outputPath1.json
SQLCMD -S . -d "databaseName" -E -i "path_to_query2.sql" -y0 -s "|" -f o:65001 > outputPath2.json
The argument -f o:65001 is to output it to utf8 format, but only the second line outputs the query in an utf8 format.
Why is this? Why does it seem that the argument "-f o:65001" only works for the second command?
I checked it by switching the order and then again only the second command outputs the query in utf8 format.
Thanks for any tips on this.
EDIT
The solution for my specific problem was to put "chcp 65001" before the SQLCMD's. You then also don't need the argument -f 0:65001
Is there a way to use the Slick SourceCodeGenerator to generate source code from a file of SQL CREATE statements? I know there is a way to connect to a DB and read in the schema, but I want to cut out that step and just give it the file. Please advise.
Slick ready meta data via jdbc. If you find a jdbc driver that can do that from a SQL file, you may be in luck. Otherwise, why not use an H2 in-memory database? It has compatibility modes for various SQL dialects. They are limited though. Another option would be using something like this: https://github.com/bgranvea/mysql2h2-converter first to produce an H2 compatible schema file.
We used the following script to load a sql schema from a mysql database, convert it to H2 compatible format and then use it in-memory for tests. You should be able to adapt it.
#!/bin/sh
echo ''
export IP=192.168.1.123
export user=foobar
export password=secret
export database=foobar
ping -c 1 $IP &&\
echo "" &&\
echo "Server is reachable"
# dump mysql schema for debuggability (ignore in git)
# convert the mysql to h2db using the converter.
## disable foreign key check in begining and enable it in the end. Prevents foreign key errors
echo "SET FOREIGN_KEY_CHECKS=0;" > foobar-mysql.sql
## Dump the Db structure and remove the auto_increment so as to set the id column back to 1
mysqldump --compact -u $user -h $IP -d $database -p$password\
|sed 's/CONSTRAINT `_*/CONSTRAINT `/g' \
|sed 's/KEY `_*/KEY `/g' \
|sed 's/ AUTO_INCREMENT=[0-9]*//' \
>> foobar-mysql.sql
echo "SET FOREIGN_KEY_CHECKS=1;" >> foobar-mysql.sql &&\
java -jar mysql2h2-converter.jar foobar-mysql.sql \
|perl -0777 -pe 's/([^`]),/\1,\n /g' \
|perl -0777 -pe 's/\)\);/)\n);/g' \
|perl -0777 -pe 's/(CREATE TABLE [^\(]*\()/\1\n /g' \
|sed 's/UNSIGNED/unsigned/g' \
|sed 's/float/real/' \
|sed "s/\(int([0-9]*).*\) DEFAULT '\(.*\)'/\1 DEFAULT \2/" \
|sed "s/tinyint(1)/boolean/" \
> foobar-h2.sql
perl -ne 'print "$ARGV\n" if /.\z/' -- foobar-h2.sql
I need to see which files have been added or removed between two streams. The most obvious way would be "git lsfiles" in each stream. Except this is not GIT and I do not see an analogous command. So for today:
for f in $(find * -type f);do
accurev stat "$f"
done | \
fgrep -v '(external)' | \
awk '{print $1}' > .list
If there is a better way, it should be clear and easy to find here:
http://www.accurev.com/download/docs/5.7.0_books/AccuRev_5_7_User_CLI.pdf
but it is not. Help? Thank you.
If you want to see the difference between two streams, run the following command: accurev diff -a -v "Stream1" -V "Stream2"
As the command line question has been answered, here's how to do the same via the AccuRev GUI.
Select one dynamic stream, workspace or snapshot.
Right click and select "Show Diff By Files"
Select a different dynamic stream, workspace or snapshot.
You'll be presented with a list of files different between the two choices, and yes you can mix-and-match between dynamic streams, workspaces and snapshots.
You can then select any file and select "Show Difference" to see differences between the two files.
Since neither of the two answers addressed the question, I eventually worked out a script to do what is really needed. "accurev lsfiles" is sorely needed.
#! /bin/bash
declare -r progpid=$$
declare -r progdir=$(cd $(dirname $0) >/dev/null && pwd)
declare -r prog=$(basename $0)
declare -r program="$progdir/$prog"
declare -r usage_text=' [ <directory> ... ]
If no directory is specified, "." is assumed'
die() {
echo "$prog error: $*"
exec 1>/dev/null 2>&1
kill -9 $progpid
exit 1
} 1>&2
usage() {
test $# -gt 0 && {
exec 1>&2
echo "$prog usage error: $*"
}
printf "USAGE: $prog %s\n" "$usage_text"
exit $#
}
init() {
shift_ct=$#
tmpd=$(mktemp -d ${TMPDIR:-/tmp}/ls-XXXXXX)
test -d "$tmpd" || die "mktemp -d does not work"
exec 4> ${tmpd}/files
trap "rm -rf '$tmpd'" EXIT
prune_pat=
while :
do
test $# -eq 0 && break
test -f "$1" && break
[[ "$1" =~ -.* ]] || break
case "X$1" in
X-p )
prune_pat+="${2}|"
shift 2 || usage "missing arg for '-p' option"
;;
X-p* )
prune_pat+="${1#-p}"
shift
;;
X-x* )
set -x
tput reset 1>&2
PS4='>${FUNCNAME:-lsf}> '
shift
;;
X-* )
usage "unknown option: $1"
;;
* )
break
;;
esac
done
(( shift_ct -= $# ))
test ${#prune_pat} -gt 0 && \
prune_pat='(^|/)('${prune_pat%|}')$'
}
chkdir() {
declare list=$(exec 2> /dev/null
for f in "$#"
do ls -d ${f}/* ${f}/.*
done | \
grep -v -E '.*/\.\.*$' )
for f in $(accurev stat ${list} | \
grep -v -F '(external)' | \
awk '{print $1}' | \
sed 's#^/*\./##')
do
test ${#prune_pat} -gt 0 && [[ $f =~ ${prune_pat} ]] && continue
if test -d "$f"
then chkdir "$f"
elif test -f "$f" -o -L "$f"
then echo "$f" 1>&4
fi
done
}
init ${1+"$#"}
(( shift_ct > 0 )) && shift ${shift_ct}
test $# -eq 0 && set -- .
chkdir "$#"
exec 4>&-
sort -u ${tmpd}/files
It is a bit over-the-top, but I have a boilerplate I always use for my scripts.
I have a bash script to call a select in postgres. I would like to be able to pass a variable from the command line into the sql file.
sh myscript.sh 1234
#!/bin/bash
dbase_connect="psql -h server -U username dbase"
file="/tmp/$fname.csv"
sql="/home/user/sql_files/query.sql"
sudo bash -c "$dbase_connect -c -q -A -F , -f $sql -o $file"
The query can be as simple as:
select name from table where id = $1;
But I don't know how to call the $1 into the sql file. The actual query is much larger and I prefer to keep it out of the bash query itself because it is easier to maintain when called as a seperate .sql file.
you can use sed to insert parameter :
#!/bin/bash
dbase_connect="psql -h server -U username dbase"
file="/tmp/$fname.csv"
sql="/home/user/sql_files/query.sql"
tmp="home/user/sql_files/query.sql.tmp"
s="s/\$1/$1/g"
cat $sql | sed $s > $tmp
sudo bash -c "$dbase_connect -c -q -A -F , -f $tmp -o $file"
rm -f $tmp
I have a .txt file and on each line is a different file location e.g.
file1.zip
file2.zip
file3.zip
How can I open that file, loop through each line and rm -f filename on each one?
Also, will deleting it throw an error if the file doesn't exist (has already been deleted) and if so how can I avoid this?
EDIT: The file names may have spaces in them, so this needs to be catered for as well.
You can use a for loop with cat to iterate through the lines:
IFS=$'\n'; \
for file in `cat list.txt`; do \
if [ -f $file ]; then \
rm -f "$file"; \
fi; \
done
The if [ -f $file ] will check if the file exists and is a regular file (not a directory). If the check fails, it will skip it.
The IFS=$'\n' at the top will set the delimiter to be newlines-only; This will allow you to process files with whitespace.
xargs -n1 echo < test.txt
Replace 'echo' with rm -f or any other command. You can also use cat test.txt |
'man xargs' for more info.