I am using the following command to output the result of an SQL query to a text file:
$sqlite3 my_db.sqlite "SELECT text FROM message;" > out.txt
This gives me output like this:
text for entry 1
text for entry 2
Unfortunately, this breaks down when the text contains a newline:
text for entry 1
text for
entry 2
How can I specify an output delimiter (which I know doesn't exist in the text) for SQLite to use when outputting the data so I can more easily parse the result? E.g.:
text for entry 1
=%=
text for
entry 2
=%=
text for entry 3
Try -separator option for this.
$sqlite3 -separator '=%=' my_db.sqlite "SELECT text FROM message;" > out.txt
Update 1
I quess this is because of '-list' default option. In order to turn this option off you need to change current mode.
This is a list of modes
.mode MODE ?TABLE? Set output mode where MODE is one of:
csv Comma-separated values
column Left-aligned columns. (See .width)
html HTML <table> code
insert SQL insert statements for TABLE
line One value per line
list Values delimited by .separator string
tabs Tab-separated values
tcl TCL list elements
-list Query results will be displayed with the separator (|, by
default) character between each field value. The default.
-separator separator
Set output field separator. Default is '|'.
Found this info here
I had the same question and there is a simpler solution. I found this at https://sqlite.org/cli.html :
.separator COL ?ROW? Change the column and row separators
For example:
sqlite> .separator | ,
sqlite> select * from example_table;
1|3,1|4,1|15,1|21,1|33,2|13,2|16,2|32,
Or with no column separator:
sqlite> .separator '' ,
sqlite> select * from example_table;
13,14,115,121,133,213,216,232,
Or, to answer the specific question posed above, this is all that is needed:
sqlite> .separator '' \r\n=%=\r\n
sqlite> select * from message;
text for entry 1
=%=
text for
entry 2
=%=
text for entry 3
=%=
In order to seperate columns, you would have to work with group_concat and a seperator.
Query evolution:
SELECT text FROM messages;
SELECT GROUP_CONCAT(text, "=%=") FROM messages;
SELECT GROUP_CONCAT(text, "\r\n=%=\r\n") FROM messages;
// to get rid of the concat comma, use replace OR change seperator
SELECT REPLACE(GROUP_CONCAT(text, "\r\n=%="), ',', '\r\n') FROM messages;
SQLFiddle
Alternative: Sqlite to CSV export (with custom seperator), then work with that.
Related
I have a table that has some rows with normal JSON and some with escaped values in the JSON field (backslashes)
id
obj
1
{"is_from_shopping_bag":true,"products":[{"price":{"amount":"18.00","currency":"USD","offset":100,"amount_with_offset":"1800"},"product_id":"1234","quantity":1}],"source":"cart"}
2
{"is_from_shopping_bag":"","products":"[{\ "product_id\ ":\ "2345\ ",\ "price\ ":{\ "currency\ ":\ "USD\ ",\ "amount\ ":\ "140.00\ ",\ "offset\ ":100},\ "quantity\ ":1}]"}
(Note: I needed to include a space after the backslashes in the above table so that they would show up in the github generated markdown table -- my actual table does not include those spaces between the backslash and the quote character)
I am doing a sql query in Hive to get the 'currency' field.
Currently I can run
SELECT
id,
JSON_EXTRACT(obj, '$.products[0].price.currency')
FROM my_table
Which will give me the correct output for the first row, but gives me a NULL in the second row
id
obj
1
"USD"
2
NULL
What is the best way to get currency field from the second row? Is there a way to clean up the field and remove the backslashes before trying to JSON_EXTRACT the relevant data?
I could use REPLACE to swap the '\ ' for '', but is that the most efficient method?
Replace \" with " using regexp_replace like this:
regexp_replace(obj,'\\\\"','"')
i have a table:
id | detail
1 | ddsffdfdf ;df, deef,"dgfgf",/dfdf/
when I did: insert into details values(1,'ddsffdfdf ;df, deef'); => got inserted properly
When I copied that inserted value from database to a file,the file had: 1 ddsffdfdf ;df, deef
Then I loaded the whole csv file to pgsql database,with values in the format: 1 ddsffdfdf ;df, deef
ERROR: invalid input syntax for integer: "1 ddsffdfdf ;df, deef is obtained. How to solve the problem?
CSVs need a delimiter that Postgres will recognize to break the text into respective fields. Your delimiter is a space, which is insufficient. Your CSV file should look more like:
1,"ddsffdfdf df, deef"
And your SQL should look like:
COPY details FROM 'filename' WITH CSV;
The WITH CSV is important because it tells Postgres to use a comma as the delimiter and parses your values based on that. Because your second field contains a comma, you want to enclose its value in quotes so that its comma is not mistaken for a delimiter.
To look at a good example of a properly formatted CSV file, you can output your current table:
COPY details TO '/your/filename.csv' WITH CSV;
I am using command line Hive. For example hive -e "SELECT * FROM my_db.my_table;"
It is currently returning what looks like tab separated values. Is it possible to specify which delimiter it should use? For example, can I make it return pipe separated values?
what i am done in my case, i fired a query like below.
INSERT OVERWRITE LOCAL DIRECTORY '/home/Desktop/test3'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
select * from stud_02
other solution would be
hive -e 'select * from stud_01 limit 10' | sed 's/[[:space:]]\+/,/g' >> /home/Desktop/test.csv
Let's say I have a SQL Server table that looks like the following:
ID NAME DESCRIPTION
1 ANDREW COOL
2 MATT NOT COOL
All I need to do is output the data to a space delimited text file. However I want to ensure that the 'NAME' column has at maximum 10 characters. So for example with the first row 'ANDREW' is is 6 characters, then I'd want 4 spaces after it.
Same thing for second row. 'MATT' is 4 characters, so I would want 6 spaces after it. This way as you move to each column the data is lined up, worst case it gets truncated but I'm not concerned with that.
Use this select query then export this to ur text file.
select ID,cast(NAME as char(10)) as NAME,DESCRIPTION from yourtable
you can use convert function
select CONVERT(char(10),'ANDREW')
.
select ID,
CONVERT((char(10),NAME) as NAME,
DESCRIPTION
from <table>
I am running a job in MS SQL Server that outputs a text file with white space in between columns. What I'd like to do is specify a specific character sequence between each column as a delimiter.
For example, I'd like the output to look like:
Apple%%%red%%%fruit
banana%%%yellow%%%fruit
onion%%%White%%%veggie
In this example, %%% is the delimiter.
How can I do this?
Assuming that you are using the output file of the job step, and the output file is currently structured something like this:
---- --------
row1 somedata
row2 somedata
You could just concatenate the columns using '+' and fit the percent signs in as appropriate. So the job step definition would contain:
select column1 + '%%%' + column2 from table1;
And the output would look like:
---------------
row1%%%somedata
row2%%%somedata
This assumes that you are OK with concatenating each row of results into a single column. You will need to cast/convert non-character column values for this to work.
My guess is that you are looking for the T-SQL equivalent of Oracle's P/SQL command "set colsep" command. This command lets you alter the delimter of the output. TO make it semicolon, for example, you would call:
set colsep ";"
But in SQL Server... I see the way to do it.
Use the "bcp" utility and you can specify the delimiter and write to your file. He are instructions:
http://msdn.microsoft.com/en-us/library/ms162802.aspx
Look at the -t option to change the separator.