CrateDB needs tables to be dropped before a restore.
Options unavailable:
#SQL
Multiple statements copied into console/crash cli
Is there an easy way to do this?
The way I resolved this was via a bash script using the Crash CLI which pulls the tables and drops them individually.
You will need to set $HOST and $TABLE_CATALOG
crash --hosts $HOST -c "SELECT CONCAT('\"', TABLE_CATALOG, '\".\"', TABLE_NAME, '\"') FROM INFORMATION_SCHEMA.tables WHERE table_catalog = $TABLE_CATALOG --format="csv" |
tail -n +2 | head -n -1 | sed 's/"/\\"/g' |
xargs -I {} crash --hosts $HOST -c 'DROP TABLE {}'
When using PSQL's variables, I can run it as follows:
psql -d database -v var="'123'"
And I will then have access to the variable var when I type the following in the PSQL terminal:
select * from table where column = :var;
This variable feature also works when the SQL is read from a file:
psql -d database -v var="'123'" -f file.sql
But when I try to run the SQL as a single command:
psql -d database -v var="'123'" -c "select * from table where column = :var;"
I can't access the variable and get the following error:
ERROR: syntax error at or near ":"
Is it possible to pass variables to single SQL commands in PSQL?
It turns out that, as man psql explains, the -c command is limited to SQL that "contains no psql-specific features":
-c command, --command=command
Specifies that psql is to execute one command string, command, and then exit. This is useful in shell
scripts. Start-up files (psqlrc and ~/.psqlrc) are ignored with this option.
command must be either a command string that is completely parsable by the server (i.e., it contains no
psql-specific features), or a single backslash command. Thus you cannot mix SQL and psql meta-commands
with this option. To achieve that, you could pipe the string into psql, for example: echo '\x \\ SELECT
* FROM foo;' | psql. (\\ is the separator meta-command.)
It looks like I can do what I want by passing in the SQL using stdin:
echo "select * from table where column = :var;" | psql -d database -v var="'123'"
Is there a way to run the following command line statement from an SQL script in a PostgreSQL database?
raster2pgsql -s 4326 -I -C -M D:\postgresql\data\input.tif -F -t 100x100 public.demelevation > output.sql
I found something promising for Microsoft's SQL Server here, but I couldn't find something similar for PostgreSQL.
you can use COPY for that. Think if it's a good idea to call programm from psql though. Also you will need superuser permissions for that.
Example:
t=# create table c (t text);
t=# copy (select 1) to program $$echo "whatever" -v >/tmp/12$$;
COPY 1
t=# copy c from program 'echo "whatever" -v';
COPY 1
t=# select * from c;
t
-------------
whatever -v
(1 row)
Time: 0.380 ms
t=# !\ cat /tmp/12
Invalid command \. Try \? for help.
t-# \! cat /tmp/12
whatever -v
Using the following sqlcmd script:
sqlcmd -S . -d MyDb -E -s, -W -Q "select account,rptmonth, thename from theTable"
> c:\dataExport.csv
I get an csv output file containing
acctnum,rptmonth,facilname
-------,--------,---------
ALLE04,201406,Allendale Community for Senior Living-LTC APPL02,201406,Applewood Estates
ARBO02,201406,Arbors Care Center
ARIS01,201406,AristaCare at Cherry
Hill
. . .
(139 rows affected)
Is there a way to get rid of the dashed line under the column headers : -------,--------, but keep the column headers?
and also a way to get rid of the two lines used for the row count on the bottom?
I tries using parm -h-1 but that got rid of the column headers as well as the dashed line.
Solutions:
1) To remove the row count ("(139 rows affected)") you should use SET NOCOUNT ON statement. See ref.
2) To remove column headers you should use -h parameter with value -1. See ref (section Formatting Options).
Examples:
C:\Users\sqlservr.exe>sqlcmd -S(local)\SQL2012 -d Test -E -h -1 -s, -W -Q "set nocount on; select * from dbo.Account" > d:\export.txt.
or
C:\Users\sqlservr.exe>sqlcmd -S(local)\SQL2012 -d Test -E -h -1 -s, -W -Q "set nocount on; select * from dbo.Account" -o "d:\export2.txt"
The guy with the top answer didn't answer how to remove the dashed line. This is my awesome solution.
First include -h -1 which removes both the dashed line and header
Then before your select statement manually inject the header string that you need with a PRINT statement. So in your case PRINT 'acctnum,rptmonth,facilname' select..*...from...
Sorry I'm 4 years and 9 months late.
Use the following;
sqlcmd -S . -d MyDb -E -s, -h-1 -W -Q "set nocount on;select 'account','rptmonth', 'thename';select account,rptmonth, thename from theTable"
> c:\dataExport.csv
remove the header -h-1
remove row count [set nocount on;]
add header select [select 'account','rptmonth', 'thename';]
add your select [select account,rptmonth, thename from theTable;]
To remove the Row Count:
Add the below to your SQL statement
SET NOCOUNT ON;
To remove the hyphen row try the following upon successful execution:
findstr /v /c:"---" c:\dataExport.csv > c:\finalExport.csv
I use "---" as all my columns are over 3 characters and I never have that string in my data but you could also use "-,-" to reduce the risk further or any delimiter based on your data in place of the ",".
In my case worked well as :
type Temp.txt | findstr /v -- > DestFile.txt
In addition, if you want to query out all records in a table, you can code as
SET NOCOUNT ON;
SELECT SUBSTRING((SELECT ','+ COLUMN_NAME FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME=N'%table_name%' FOR XML
PATH('') ), 2, 9999);
SELECT * FROM %table_name%
Assign the above queries into a variable %query%. The the command will be looks like as below.
SQLCMD -h -1 -W -E -S %sql_server% -d %sql_dabase% -Q %query% -s"," -o output_file.csv
This is the one line solution, without doing anything inside the stored procedure to append the column headers:
sqlcmd -S . -d MyDb -E -s, -W -Q "select account,rptmonth, thename from theTable"
| findstr /v /c:"-" /b > "c:\dataExport.csv" & exit 0
What this does is it intercepts all console output and replaces the "-" char BEFORE it redirects to the output file. There is NO need to output to intermediary file. And you will need a one-liner command if you use an agent to run these commands remotely on the sql server machines, which most of the times are locked from hosting *.bat files (which you'd need for multiline commands).
I added the "exit 0" at the end to not fail the caller application overall. You may remove it starting "& exit 0" if you don't care about that.
This one liner is why I chose sqlcmd over bcp out, by the way. BCP, although optimized for speed, cannot output column headers unless doing the ugly trick within the stored proc, to append them there as a union all.
Just in case you have access to writing a bat file that contains this one liner, you MUST add #ECHO OFF before it. Otherwise the console output will also have the actual command.
Hope it helps.
With SQL Server 2017 (14.x) and later you can print header with:
SELECT string_agg(COLUMN_NAME, ', ') within group (order by ORDINAL_POSITION asc) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='YOUR_TABLE_NAME'
1.Create the file first with the header columns
2.Apprend the sqlcmd output to the file using the option -h-1
echo acctnum,rptmonth,facilname > c:\dataExport.csv
sqlcmd -S . -d MyDb -E -s, -h-1 -W -Q "select account,rptmonth, thename from theTable" >> c:\dataExport.csv
I used another solution to solve the issue of removing the dashed line below the header.
DECLARE #combinedString VARCHAR(MAX);
SELECT #combinedString = COALESCE(#combinedString + '|', '') + COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'YOUR_TABLE_NAME'
Then just use
Print #combinedString above your select statement.
I used pipe delimiter.
How do you hide the column names and row count in the output from psql?
I'm running a SQL query via psql with:
psql --user=myuser -d mydb --output=result.txt -c "SELECT * FROM mytable;"
and I'm expecting output like:
1,abc
2,def
3,xyz
but instead I get:
id,text
-------
1,abc
2,def
3,xyz
(3 rows)
Of course, it's not impossible to filter the top two rows and bottom row out after the fact, but it there a way to do it with only psql? Reading over its manpage, I see options for controlling the field delimiter, but nothing for hiding extraneous output.
You can use the -t or --tuples-only option:
psql --user=myuser -d mydb --output=result.txt -t -c "SELECT * FROM mytable;"
Edited (more than a year later) to add:
You also might want to check out the COPY command. I no longer have any PostgreSQL instances handy to test with, but I think you can write something along these lines:
psql --user=myuser -d mydb -c "COPY mytable TO 'result.txt' DELIMITER ','"
(except that result.txt will need to be an absolute path). The COPY command also supports a more-intelligent CSV format; see its documentation.
You can also redirect output from within psql and use the same option. Use \o to set the output file, and \t to output tuples only (or \pset to turn off just the rowcount "footer").
\o /home/flynn/queryout.txt
\t on
SELECT * FROM a_table;
\t off
\o
Alternatively,
\o /home/flynn/queryout.txt
\pset footer off
. . .
usually when you want to parse the psql generated output you would want to set the -A and -F ...
# generate t.col1, t.col2, t.col3 ...
while read -r c; do test -z "$c" || echo , $table_name.$c | \
perl -ne 's/\n//gm;print' ; \
done < <(cat << EOF | PGPASSWORD=${postgres_db_useradmin_pw:-} \
psql -A -F -v -q -t -X -w -U \
${postgres_db_useradmin:-} --port $postgres_db_port --host $postgres_db_host -d \
$postgres_db_name -v table_name=${table_name:-}
SELECT column_name
FROM information_schema.columns
WHERE 1=1
AND table_schema = 'public'
AND table_name =:'table_name' ;
EOF
)
echo -e "\n\n"
You could find example of the full bash call here: