Text Qualifier syntax in PDW dwloader - sql

I am using dwloader utility to load data into PDW. My dwloader syntax is given below:
dwloader.exe -M reload -b 200000 -i \d\Development\Source\FACT.TXT -T Test_DB.dbo.FACT_LOAD -R \d\Development\Bad\FACT_LOAD.bad -t "," -fh 1 -r \r\n -D "yyyy-MM-dd" -E -S 10.1.1.19 -U Devl_User -P Password1> "\d\Development\Log\FACT_LOAD.log"
Pause
I got the data loaded into the table. But all the varchar columns have data within double 'quotes'. My text file is commma delimited. Any idea on how to include the text qualifier in the above syntax?

I figured the answer myself. Including " -s 0x22" in the command above will escape the double quotes.

Related

Two of the same commands give different result (output as utf8)

I have a .bat file that contains two commands:
SQLCMD -S . -d "databaseName" -E -i "path_to_query1.sql" -y0 -s "|" -f o:65001 > outputPath1.json
SQLCMD -S . -d "databaseName" -E -i "path_to_query2.sql" -y0 -s "|" -f o:65001 > outputPath2.json
The argument -f o:65001 is to output it to utf8 format, but only the second line outputs the query in an utf8 format.
Why is this? Why does it seem that the argument "-f o:65001" only works for the second command?
I checked it by switching the order and then again only the second command outputs the query in utf8 format.
Thanks for any tips on this.
EDIT
The solution for my specific problem was to put "chcp 65001" before the SQLCMD's. You then also don't need the argument -f 0:65001

Import CSV from Linux to Azure SQL Server

I have an Azure SQL Server database and a linux box. I have a csv file on the linux machine that I want to import into SQL Server. I have a table already created where I am going to import this file.
Why does this command return an Unknown argument: -U?
bcp table in ~/test.csv -U myUsername -S databaseServerName -d dbName -q -c -t
When I rearrange the arguments passed to bcp like below, it returns an Unknown argument: -S
bcp table in ~/test.csv -S databaseServerName -d dbName -U myUsername -q -c -t
So contrary to the documentation:
https://learn.microsoft.com/en-us/sql/tools/bcp-utility?redirectedfrom=MSDN&view=sql-server-2017#U
I hit issues where bcp does not like spaces after the argument names.
https://granadacoder.wordpress.com/2009/12/22/bcp-export/
quote from the article above:
//
The other syntax sugar is that there is no space after the -S
argument. As seen below
-SMyServerName\MyInstanceName
bcp.exe "SELECT cast(LastName as char(50)) , cast(FirstName as
char(50)) , cast(MiddleName as char(50)) , cast(Suffix as char(50))
FROM MyAdventureWorksDB.Person.Person ORDER BY NEWID()" queryout
PeopleRock.txt -c -t -T -SMyServerName\MyInstanceName
also
https://www.easysoft.com/products/data_access/odbc-sql-server-driver/bulk-copy.html#importing-data-table
check your syntax sugar in linux (below example is from above easysoft link)
./bcp AdventureWorks.HumanResources.myTeam in ~/myTeam.csv \
-f ~/myTeam.Fmt -U mydomain\myuser -S mymachine\sqlexpress
Note the above has the dbname.schemaname.tablename (before the "in" word above)

How to execute postgres' sql queries from batch file?

I need to execute SQL from batch file.
I am executing following to connect to Postgres and select data from table
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
select * from test;
I am able to connect to database, however I'm getting the error
'select' is not recognized as an internal or external command,
operable program or batch file.
Has anyone faced such issue?
This is one of the query i am trying, something similar works in shell script, (please ignore syntax error in the query if there are any)
copy testdata (col1,col2,col3) from '%filepath%/%csv_file%' with csv;
You could pipe it into psql
(
echo select * from test;
) | C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
When closing parenthesis are part of the SQL query they have to be escaped with three carets.
(
echo insert into testconfig(testid,scenarioid,testname ^^^) values( 1,1,'asdf'^^^);
) | psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME%
Use the -f parameter to pass the batch file name
C:/pgsql/bin/psql -h %DB_HOST% -p 5432 -U %DB_USER% -d %DB_NAME% -f 'sql_batch_file.sql'
http://www.postgresql.org/docs/current/static/app-psql.html
-f filename
--file=filename
Use the file filename as the source of commands instead of reading commands interactively. After the file is processed, psql terminates. This is in many ways equivalent to the meta-command \i.
If filename is - (hyphen), then standard input is read until an EOF indication or \q meta-command. Note however that Readline is not used in this case (much as if -n had been specified).
if running on Linux, this is what worked for me (need to update values below with your user, db name etc)
psql "host=YOUR_HOST port=YOUR_PORT dbname=YOUR_DB_NAME user=YOUR_USER_NAME password=YOUR_PASSWORD" -f "fully_qualified_path_to_your_script.sql"
You cannot put the query on separate line, batch interpreter will assume it's another command instead of a query for psql. I believe you will need to quote it as well.
I agree with Spidey:
1] if you are passing the file with the sql use -f or --file parameter
When you want to execute several commands the best way to do that is to add parameter -f, and after that just type path to your file without any " or ' marks (relative paths works also):
psql -h %host% -p 5432 -U %user% -d %dbname% -f ..\..\folder\Data.txt
It also works in .NET Core. I need it to add basic data to my database after migrations.
Kindly refer to the documentation
1] if you are passing the file with the sql use -f or --file parameter
2] if you are passing individual command use -c or --command parameter
If you are trying the shell script
psql postgresql://$username:$password#$host/$database < /app/sql_script/script.sql

Saving psql output to csv file

I have a query written in a file located at /path/to/query. How can I save the output result to a csv file, without using COPY in the query? I tried the following command, but the output file's fields are separated by " | ".
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ','
It is not explained in the documentation, but the -F option requires the -A option (unaligned table output) to work:
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ',' -A
If you don't wish the headers in your csv, this means, without extra rows at the top and at the bottom, use the -t option too.
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ',' -A -t
From the help:
-A, --no-align unaligned table output mode
-F, --field-separator=STRING
set field separator (default: "|")
-t, --tuples-only print rows only

escape character with ssh

I'm trying to write several commands trought ssh connection bue I got problem with escape characters. Below an example of what I'd like to do:
/usr/bin/ssh mrtg#172.20.29.40 echo -e "ciao\nprova"
I got this result:
ciaonprova
instead of:
ciao
prova
if I use -e option for ssh:
/usr/bin/ssh -e mrtg#172.20.29.40 echo -e 'ciao\nprova'
I receive this error:
Bad escape character 'mrtg#172.20.29.40'.
Can someone give me a suggestion to let remote server interpret escape characters?
The -e option has nothing to do with your command (these are SSH escape characters, not shell).
You can just put your command in quotes:
/usr/bin/ssh mrtg#172.20.29.40 'echo -e "ciao\nprova"'