I am using command line in Windows 7 to read rows of variables from a CSV, pass those variables into a SQL command, and execute that SQL command for each row in the CSV. This works fine, I believe. Where I am running into trouble is when my variables contain spaces, such as strings that need to be input into my database.
My code looks like this:
Command line:
C:\Users\me>(sqlcmd -S npl-sql01 -d OnBaseTEST -i "C:\Users\me\myquery.sql" -v var1="\Policy Term\Policy\" var2= 'REPORT' var3= 'Commercial' )
Where var1, var2, and var3 are variables in the SQL query that get their values from a CSV.
SQL query:
INSERT INTO testdb (column1, column2, column3)
SELECT $(var1), $(var2), $(var3), od.value
FROM otherdatabase AS od
WHERE od.identifier = 24
When I wrap the variable np in single quotes, the command line chokes up on the space in the path. When I wrap it in double quotes, SQL appears to be treating it as a column name rather than a value to be put into a column (based off of this question). I have also tried wrapping with two sets of quotes, e.g. "'a string'" and '"a string"', with no luck. How should I be writing out this string with spaces so that both command line and SQL can understand it?
Edit: For some reason, removing the trailing variable var3 allows the script to work as intended when the string is wrapped with "'two types of quotes'"
Related
I'm having an issue figuring out how to ignore signs and variables in a single quote string statement.
I am attempting to update a table with the new text with structure such as:
update xxx
set xxx =
'Our Ref. $BOOKING_NO$
.......
Kind regards'
If your $ chars are being interpreted, it isn't by Oracle ($ isn't special in Oracle anyway, and between single-quotes everything is a string), but rather by your client program or maybe shell script. If, for example, you are running this in SQL*Plus from a Unix-based shell script, you will need to use the appropriate means required by the shell you use to prevent the shell from interpreting $ and ' characters.
I am asked to help with a UNIX script that deploys database objects with or without data from 1 machine and/or environment with another. The problem is that some columns have CHAR data with single quotes. Only these single quotes need to be converted to 2 single quotes. I was thinking of using sed or awk, but I can't figure it out. Anyone got any ideas?
example:
INSERT INTO SQL_ERROR_MESSAGE (SQL_Error_Cd,Error_Text) VALUES (5492,'The argument for an INOUT/OUT parameter '%VSTR' is invalid.');
becomes
INSERT INTO SQL_ERROR_MESSAGE (SQL_Error_Cd,Error_Text) VALUES (5492,'The argument for an INOUT/OUT parameter ''%VSTR'' is invalid.');
Thanks for your support.
If you are rather certain about the format of your INSERT statement, this might work for you:
sed "s/\([^']\+'[^']\+\)\('\)\([^']\+\)'\(.*\)/\1\2\2\3\2\2\4/" INPUTFILE
But I'd recommend to check the output :-)
Basically it captures everything till the second(!) ' (in \1), then captures the 2nd ' (\2) - that's not needed, I just wrote it this way ... -, in \3 the string in between the 's which should be escaped, and finally the remaining part of the string (\4), and uses the saved entries in the replacement.
Exapmle output (Linux, GNU sed):
$ sed "s/\([^']\+'[^']\+\)\('\)\([^']\+\)'\(.*\)/\1\2\2\3\2\2\4/"
INPUT LINE #=> INSERT INTO SQL_ERROR_MESSAGE (SQL_Error_Cd,Error_Text) VALUES (5492,'The argument for an INOUT/OUT parameter '%VSTR' is invalid.');
OUTPUT LINE #=> INSERT INTO SQL_ERROR_MESSAGE (SQL_Error_Cd,Error_Text) VALUES (5492,'The argument for an INOUT/OUT parameter ''%VSTR'' is invalid.');
Here is my simplified scenario:
I have a table in SQL Server 2005 with single column of type varchar(500). Data in the column is always 350 characters in length.
When I run a select on it in SSMS query editor, copy & paste the result set in to a text file, the line length in the file is 350, which matches the actual data length.
But when I use sqlcmd with the -o parameter, the resulting file has line length 500, which matches the max length of varchar(500).
So question is, without using any string functions in select, is there a way to let sqlcmd know not to treat it like char(500) ?
You can use the sqlcmd formatting option -W to remove trailing spaces from the output file.
Read more at this MSDN article.
-W only works with default size of 256 for variable size columns. If you want more than that you got to use -y modifier which will tell you its mutually exclusive with -W. Basically you are out of luck and as in my case file grows from 0.5M to 172M. You have to use other ways to strip white space post file generation. Some PowerShell command or something.
I have a SQL script that will insert a long string into a table. The string contains a new line (and this new line is absolutely necessary), so when it is written in a text file, the query is split to multiple lines. Something like:
insert into table(id, string) values (1, 'Line1goesHere
Line2GoesHere
blablablabla
');
This runs ok in Toad, but when I save this as a .sql file and run it using sqlplus, it considers each line a separate query, meaning that each line will fail (beacuse insert into table(id, string) values (1, 'Line1goesHere, Line2GoesHere aren't well-formated scripts.
SP2-0734: unknown command beginning "Line2GoesHere" - rest of line ignored.
Is there a way to fix this?
Enable SQLBLANKLINES to allow blank lines in SQL statements. For example:
SET SQLBLANKLINES ON
insert into table(id, string) values (1, 'Line1goesHere
Line2GoesHere
blablablabla
');
The premise of this question is slightly wrong. SQL*Plus does allow multi-line strings by default. It is only blank lines that cause problems.
You can also use not-well-known feature of Oracle's SQL: Perl style quoted strings.
SQL> select q'[f dfgdfklgdfkjgd
2 sdffdslkdflkgj dglk
3 glfdglkjdgkldj ]'
4 from dual;
Q'[FDFGDFKLGDFKJGDSDFFDSLKDFLKGJDGLKGLFDGLKJDGKLDJ]'
----------------------------------------------------
f dfgdfklgdfkjgd
sdffdslkdflkgj dglk
glfdglkjdgkldj
SQL*Plus Manual
You can end a SQL command in one of three ways:
with a semicolon (;)
with a slash (/) on a line by itself
with a blank line
A blank line in a SQL statement or script tells SQL*Plus that you have
finished entering the command, but do not want to run it yet. Press
Return at the end of the last line of the command.
Turning SQLBLANKLINES on in this situation may be the answer, but even with it you still have to worry about the following SQL*Plus commands.
# ("at" sign) (Start of line)
## (double "at" sign) (Start of line)
# SQLPREFIX (Start of line)
. BLOCKTERMINATOR (Start of line and by itself)
/ (slash) (Start of line and by itself)
; SQLT[ERMINATOR] (Start of line and by itself, or at the end)
SQLPREFIX is something that you cannot turn off; it's a feature of SQL*Plus. BLOCKTERMINATOR can be activated or disabled. Slash on the other hand if it appears at the start of a new line will cause it to execute the contents in the buffer. SQL[TERMINATOR] has a similar behavior.
Another way of inserting newlines to a string is concatenating:
chr(13)||chr(10)
(on windows)
or just:
chr(10)
(otherwise)
I'm trying to query a Sybase ASA 8 database with the iSQL client and export the query results to a text file in CSV format. However the column headings are not exported to the file. There is no special option to specify that, neither in the iSQL settings nor in the OUTPUT statement.
The query and output statement looks like this:
SELECT * FROM SomeTable;
OUTPUT TO 'C:\temp\sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE ''
The result is a file like
1;Miller;Steve;1980-06-28
2;Jones;Martha;1965-11-02
3;Waters;Richard;1979-10-15
while I'd like to have
ID;LASTNAME;FIRSTNAME;DOB
1;Miller;Steve;1980-06-28
2;Jones;Martha;1965-11-02
3;Waters;Richard;1979-10-15
Any hints?
I would have suggested to start with another statement:
SELECT 'ID;LASTNAME;FIRSTNAME;DOB' FROM dummy;
OUTPUT TO 'C:\\temp\\sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE '';
and add the APPEND option on your query... but I can't get APPEND to work (but I'm using a ASA 11 engine).
Try this one
SELECT 'ID','LASTNAME','FIRSTNAME','DOB' union
SELECT string(ID),LASTNAME,FIRSTNAME,DOB FROM SomeTable;
OUTPUT TO 'C:\\temp\\sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE '';
Simply add the option
WITH COLUMN NAMES
to your statement and it adds a header line with the column names.
The complete statement is therefore:
SELECT * FROM SomeTable; OUTPUT TO 'C:\temp\sometable.csv' FORMAT ASCII DELIMITED BY ';' QUOTE '' WITH COLUMN NAMES
See sybase documentation.
I am able to use the isql command to output quoted CSV.
Example
$ isql $DATABASE $USERNAME $PASSWORD -b -d, -q -c
select username, fullname from users
gives the result:
username,fullname
"jdoe","Jane Doe"
"msmith","Mark Smith"
Command-line flags
(copied from the man page)
-b: Run isql in non-interactive batch mode. In this mode, the isql processes its standard input, expecting one SQL command per line.
-dDELIMITER: Delimits columns with delimiter.
-c: Output the names of the columns on the first row. Has any effect only with the -d or -x options.
-q: Wrap the character fields in double quotes.
Escaping Issue
You might run into problems if the query results contain double-quotes, though. The quotes aren't escaped properly, so they result in invalid CSV:
> select 'string","with"quotes' as quoted_string
quoted_string
"string","with"quotes"
You are already familiar with the OUTPUT options. There is no option that gives you what you want.
Ok, the problem is the receiving end does not accept standard CSV files, it needs semi-colons.
If you are scripting, then you are better off getting the output in the format that is closest to what you need, and then awk-ing the output file. Very fast and you can change anything you need. I think your best option is ASCII or default output format, which will provide Comma (not colon) Separated Values, in an ASCII character text file, and includes column Headers. Then use a single awk command to convert the commas to semi-colons.
Found an easier solution, Place the headers in one file say header.txt ( it will contain a single line "col_1|col_2|col_3") then to combine the header file and your output file run:
cat header.txt my_table.txt > my_table_wth_head.txt
isql -S<Server> -D<Database>-U<UserName> -s \; -P<password>\$\1 -w 10000 -iname.sql > output.csv
If you use the FORMAT EXCEL option, it will output the rows with the column name in the first row. Then once you get it into excel you can save it into another format if you need to.
SELECT * FROM SOMETABLE;
OUTPUT TO 'C:\temp\sometable.xls' FORMAT EXCEL DELIMITED BY ';' QUOTE ''
Recently I needed to solve similar issue with some prehistoric ASA7 which does not support the WITH COLUMN NAMES for .CSV output.
The solution for me was the .DBF file, which has the columns structure in it and can be processed automatically, much better than .XLS
SELECT * FROM SomeTable;
OUTPUT TO 'C:\temp\sometable.dbf' FORMAT DBASEIII;