I have a tab delimited file which has unicode characters and has LF as line ending. When I try to import this file using bcp utility and -c switch it loads into the table, however, the characters are not correct. And when I try to use the -w switch it fails to load with below error.
"[Microsoft][ODBC Driver 17 for SQL Server]Unexpected EOF encountered in BCP data-file"
Below is text in the file
ialgcd ianvmn sourceVersion
PT Português 2021-06-W24
SP español 2021-06-W24
EG English 2021-06-W24
FR Français 2021-06-W24
GE Deutsch 2021-06-W24
RU Русский 2021-06-W24
PB Português(Brazil) 2021-06-W24
KR 한국어 2021-06-W24
CH 中文 2021-06-W24
If I use -c switch in the below manner it loads without error. But incorrect characters
bcp TableName1 in DelimitedFile1.txt -c -T -S Server01 -q -t \t -r 0x0A -F2
If I use -w switch it does not load any data and throws error
bcp TableName1 in DelimitedFile1.txt -w -T -S Server01 -q -t \t -r 0x0A -F2
I tried all different types of combinations for the line terminator and field terminators and also tried using -N switch and also tried using a format file but nothing works.
The format file I tried to use is
14.0
3
1 SQLNCHAR 2 510 "\t" 1 ialgcd SQL_Latin1_General_CP1_CI_AS
2 SQLNCHAR 2 510 "\t" 2 ianvmn SQL_Latin1_General_CP1_CI_AS
3 SQLNCHAR 2 510 "0x0A" 3 sourceVersion SQL_Latin1_General_CP1_CI_AS
I am not sure why does it work with -c switch and not with -w switch. Is it expecting a BOM character at the start of the file?
I found the solution to this issue. -w switch will work for Unicode characters but if your file is UTF-8 encoded then you have to use the -C (upper case C) and the code page number so for UT-8 it will be 65001 along with the -c switch (lower case c)
So the command will look like
bcp TableName1 in DelimitedFile1.txt -c -C 65001 -T -S Server01 -q -t \t -r 0x0A -F2
Related
I have a .bat file that contains two commands:
SQLCMD -S . -d "databaseName" -E -i "path_to_query1.sql" -y0 -s "|" -f o:65001 > outputPath1.json
SQLCMD -S . -d "databaseName" -E -i "path_to_query2.sql" -y0 -s "|" -f o:65001 > outputPath2.json
The argument -f o:65001 is to output it to utf8 format, but only the second line outputs the query in an utf8 format.
Why is this? Why does it seem that the argument "-f o:65001" only works for the second command?
I checked it by switching the order and then again only the second command outputs the query in utf8 format.
Thanks for any tips on this.
EDIT
The solution for my specific problem was to put "chcp 65001" before the SQLCMD's. You then also don't need the argument -f 0:65001
I have an Azure SQL Server database and a linux box. I have a csv file on the linux machine that I want to import into SQL Server. I have a table already created where I am going to import this file.
Why does this command return an Unknown argument: -U?
bcp table in ~/test.csv -U myUsername -S databaseServerName -d dbName -q -c -t
When I rearrange the arguments passed to bcp like below, it returns an Unknown argument: -S
bcp table in ~/test.csv -S databaseServerName -d dbName -U myUsername -q -c -t
So contrary to the documentation:
https://learn.microsoft.com/en-us/sql/tools/bcp-utility?redirectedfrom=MSDN&view=sql-server-2017#U
I hit issues where bcp does not like spaces after the argument names.
https://granadacoder.wordpress.com/2009/12/22/bcp-export/
quote from the article above:
//
The other syntax sugar is that there is no space after the -S
argument. As seen below
-SMyServerName\MyInstanceName
bcp.exe "SELECT cast(LastName as char(50)) , cast(FirstName as
char(50)) , cast(MiddleName as char(50)) , cast(Suffix as char(50))
FROM MyAdventureWorksDB.Person.Person ORDER BY NEWID()" queryout
PeopleRock.txt -c -t -T -SMyServerName\MyInstanceName
also
https://www.easysoft.com/products/data_access/odbc-sql-server-driver/bulk-copy.html#importing-data-table
check your syntax sugar in linux (below example is from above easysoft link)
./bcp AdventureWorks.HumanResources.myTeam in ~/myTeam.csv \
-f ~/myTeam.Fmt -U mydomain\myuser -S mymachine\sqlexpress
Note the above has the dbname.schemaname.tablename (before the "in" word above)
I have this bash script which connects to a postgre sql db and performs a query. I would like to be able to read line from a .txt file into the query as parameters. What is the best way to do that? Your assistance is greatly appreciated! I have my example code below however it is not working.
#!/bin/sh
query="SELECT ci.NAME_VALUE NAME_VALUE FROM certificate_identity ci WHERE ci.NAME_TYPE = 'dNSName' AND reverse(lower(ci.NAME_VALUE)) LIKE reverse(lower('%.$1'));"
(echo $1; echo $query | \
psql -t -h crt.sh -p 5432 -U guest certwatch | \
sed -e 's:^ *::g' -e 's:^*\.::g' -e '/^$/d' | \
sed -e 's:*.::g';) | sort -u
Considering that the file has only one sql query per line:
while read -r line; do echo "${line}" | "your code to run psql here"; done < file_with_query.sql
That means: while read the content of file_with_query.sql line by line, do something with each line.
I have a query written in a file located at /path/to/query. How can I save the output result to a csv file, without using COPY in the query? I tried the following command, but the output file's fields are separated by " | ".
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ','
It is not explained in the documentation, but the -F option requires the -A option (unaligned table output) to work:
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ',' -A
If you don't wish the headers in your csv, this means, without extra rows at the top and at the bottom, use the -t option too.
psql -U username -d dbname -f /path/to/query -o /path/to/output/file -F ',' -A -t
From the help:
-A, --no-align unaligned table output mode
-F, --field-separator=STRING
set field separator (default: "|")
-t, --tuples-only print rows only
I am using dwloader utility to load data into PDW. My dwloader syntax is given below:
dwloader.exe -M reload -b 200000 -i \d\Development\Source\FACT.TXT -T Test_DB.dbo.FACT_LOAD -R \d\Development\Bad\FACT_LOAD.bad -t "," -fh 1 -r \r\n -D "yyyy-MM-dd" -E -S 10.1.1.19 -U Devl_User -P Password1> "\d\Development\Log\FACT_LOAD.log"
Pause
I got the data loaded into the table. But all the varchar columns have data within double 'quotes'. My text file is commma delimited. Any idea on how to include the text qualifier in the above syntax?
I figured the answer myself. Including " -s 0x22" in the command above will escape the double quotes.