I have an input text file where fields are tab searated. Some fields contains text with single quotes (') and some fields contains text with double quotes ("). Soem fields contains both single and double quotes. Here is an example:
Theme from Bram Stoker's Dracula (From Bram Stoker's Dracula"") Soundtrack & Theme Orchestra
Is there any way to tell OPENROWSET to not try to parse the fields?
I have found that I can set the FIELDQUOTE to either a single quote or a double quote but not to both (using FIELDQUOTE = '''"' gives error Multi-byte field quote is not supported)
Here's an example of a query I try to use:
SELECT TOP 10 *
FROM OPENROWSET
(
BULK 'files/*.txt',
DATA_SOURCE = 'files',
FORMAT = 'CSV',
PARSER_VERSION = '2.0',
FIELDTERMINATOR = '\t',
FIELDQUOTE = ''''
)
AS r
and I can also use FIELDQUOTE = '"' but not the two at the same time...
Any suggestions on how to fix this? (without changing the source files)
Related
I have that kind of file
info1;info2;info3";info4;info5
And after parsing I have that error
Error: [42636] ETL-2106: Error while parsing row=0 (starting from 0) [CSV Parser found at byte 5 (starting with 0 at the beginning of the row) of 5 a field delimiter after an quoted field (with an additional whitespace) in file '~/path'. Please check for correct enclosed fields, valid field separators and e.g. unescaped field delimiters that are contained in the data (these have to be escaped)]
I'm sure that the reason is here info3"; but how can I solve this problem I have no idea
Also I can't rid of quotes, because it should be in report
The main part of python code is
# Transform data to valid CSV format: remove BOM, remove '=' sign, remove repeating quotes in Size column
decoded_csv = r.content.decode('utf-8').replace(u'\ufeff', '').replace('=', '')
print(decoded_csv)
cr = csv.reader(decoded_csv.splitlines(), delimiter=';')
lst = list(cr)[1:]
f = csv.writer(open(base_folder + 'txt/' + shop, "w+"), delimiter=';')
for row in lst:
f.writerow(row[:-2])
After this code I get that kind of file
info1;info2;"info3""";info4;info5
And it is not what I need
But when I change code a little by adding "quoting=csv.QUOTE_NONE, quotechar='')"
# Transform data to valid CSV format: remove BOM, remove '=' sign, remove repeating quotes in Size column
decoded_csv = r.content.decode('utf-8').replace(u'\ufeff', '').replace('=', '')
print(decoded_csv)
cr = csv.reader(decoded_csv.splitlines(), delimiter=';')
lst = list(cr)[1:]
f = csv.writer(open(base_folder + 'txt/' + shop, "w+"), delimiter=';' quoting=csv.QUOTE_NONE, quotechar='')
for row in lst:
f.writerow(row[:-2])
I get what I need
info1;info2;info3";info4;info5
It is a 2nd step (exasol) and code returned the error
MERGE INTO hst AS dst
USING (
SELECT DISTINCT
ar,
ar_na,
FROM (
IMPORT INTO
(
ar VARCHAR(100) UTF8 COMMENT IS 'ar',
ar_na VARCHAR(100) UTF8 COMMENT IS 'ar na',
)
FROM CSV /*SS:R*/
AT '&1'
USER '&2'
IDENTIFIED BY '&3'
FILE '~/path'
SKIP = 0
ROW SEPARATOR = 'CRLF'
COLUMN SEPARATOR = ';'
TRIM
)
GROUP BY
ar,
ar_na,
) src ON src.ar = dst.ar
WHEN MATCHED THEN UPDATE SET
dst.ar_na = src.ar_na,
WHEN NOT MATCHED THEN
INSERT (
ar
ar_na,
)
VALUES (
src.ar,
src.ar_na,
);
If file looks like info1;info2;info3;info4;info5 everything works fine, all scripts work
By default, Exaosl treats double quotes (") as column delimiter. This enables you to specify values that contain the column separator (in your case that's the semicolon). See the entry "Special characters" in the documentation.
You have two options here:
Disable the column delimiter by passing COLUMN DELIMITER = '' to the import statement.
Duplicate all double quotes in the csv file. Exasol ignores the column delimiter if it occurs twice consecutively.
I wrote the database schema (only one table so far), and the INSERT statements for that table in one file. Then I created the database as follows:
$ sqlite3 newdatabase.db
SQLite version 3.4.0
Enter ".help" for instructions
sqlite> .read ./schema.sql
SQL error near line 16: near "s": syntax error
Line 16 of my file looks something like this:
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there\'s');
The problem is the escape character for a single quote. I also tried double escaping the single quote (using \\\' instead of \'), but that didn't work either. What am I doing wrong?
Try doubling up the single quotes (many databases expect it that way), so it would be :
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there''s');
Relevant quote from the documentation:
A string constant is formed by enclosing the string in single quotes ('). A single quote within the string can be encoded by putting two single quotes in a row - as in Pascal. C-style escapes using the backslash character are not supported because they are not standard SQL. BLOB literals are string literals containing hexadecimal data and preceded by a single "x" or "X" character. ... A literal value can also be the token "NULL".
I believe you'd want to escape by doubling the single quote:
INSERT INTO table_name (field1, field2) VALUES (123, 'Hello there''s');
for replace all (') in your string, use
.replace(/\'/g,"''")
example:
sample = "St. Mary's and St. John's";
escapedSample = sample.replace(/\'/g,"''")
Just in case if you have a loop or a json string that need to insert in the database. Try to replace the string with a single quote . here is my solution. example if you have a string that contain's a single quote.
String mystring = "Sample's";
String myfinalstring = mystring.replace("'","''");
String query = "INSERT INTO "+table name+" ("+field1+") values ('"+myfinalstring+"')";
this works for me in c# and java
In C# you can use the following to replace the single quote with a double quote:
string sample = "St. Mary's";
string escapedSample = sample.Replace("'", "''");
And the output will be:
"St. Mary''s"
And, if you are working with Sqlite directly; you can work with object instead of string and catch special things like DBNull:
private static string MySqlEscape(Object usString)
{
if (usString is DBNull)
{
return "";
}
string sample = Convert.ToString(usString);
return sample.Replace("'", "''");
}
In bash scripts, I found that escaping double quotes around the value was necessary for values that could be null or contained characters that require escaping (like hyphens).
In this example, columnA's value could be null or contain hyphens.:
sqlite3 $db_name "insert into foo values (\"$columnA\", $columnB)";
Demonstration of single quoted string behavior where complexity or double quotes are not desired.
Test:
SELECT replace('SAMY''S','''''','''');
Output:
SAMY'S
SQLite version:
SELECT sqlite_version();
Output:
3.36.0
I have a concatenated string in a Bigquery table which has double quotes enclosed while exporting data as CSV to GCS. Is there a way I can avoid double quotes in the files?
The reason the final csv has quotes is because the string contains ',' which is also the default delimiter. Specifying a different delimiter other than ',' should work.
bq extract --format none --noprint_header --field_delimiter "|" [table_name] [gcs_file_location]
If feasible, try replacing all double quotes with single quotes by using SELECT * REPLACE and REPLACE, i.e:
#standardSQL
SELECT * REPLACE (REPLACE(fieldWithIssue, '"', "'") AS fieldWithIssue)
FROM yourTable
And then attempt to extract again.
I want to export a database-tab to csv-file with quotes arround any string (column separator is ";"). This is what I want as csv file (without header):
"abc";"blabla"
When I export just with "Select * from...", no quotes surround the string:
abc;blabla
When I export with "Select concate('"',column1,'"'), concate('"',column2,'"') from...", following content is given in csv-file:
"""abc""";"""blabla"""
You can use the EXPORT options
COLUMN DELIMITER and DELIMIT to control how the data is exported - e.g.
EXPORT RETAIL.ARTICLE
INTO LOCAL CSV FILE 'C:\TEMP\testexp1.csv'
ROW SEPARATOR = 'CRLF'
COLUMN SEPARATOR = ','
COLUMN DELIMITER = '"'
DELIMIT = ALWAYS;
Column named : Body in table of Data Type - Text
Sample data contains HTML tags and hyper links. The hyperlinks use a single quote.
When I am trying to replace the single quote with a double quote using the T-SQL function below :
REPLACE(Body, '''', '"')
gives me the error indicated below--->
Argument data type text is invalid for argument 1 of replace function.
Question : Can you tell me how I can pass a HTML formatted column in T-SQL containing single quotes in hyperlinks into a REPLACE function ?
This worked : Select REPLACE(CAST(Body as varchar(max)), '''', '"') as Bdy FROM TABLE
I tried writing a function( code given below):
DECLARE #Temp VARCHAR(MAX)
Set #Temp = THIS CONTAINS THE HTML FORMATTED TEXT WITH HYPERLINK IN SINGLE QUOTE';
Set #Temp = REPLACE(#Temp , '''', '"')
Select #Temp;
And the function fails indicating
Incorrect syntax near '/'.
Edit based on your comment: have you tried casting text to varchar ? Like:
REPLACE(cast(Body as varchar(max)), '''', '"')
Old answer: in SQL, a single quote is escaped by another single quote. So you should do:
REPLACE(Body, '''', '"')
That's four single quotes, comma, single qoute, double quote, single quote.