I am using the copy command to dump the data of a table in PostgreSQL to a txt file. I run the following command in PSQL:
\copy (select * from TableName) to 'C:\Database\bb.txt' with delimiter E'\t' null as '';
now in the bb.txt file, I see some special characters which are not there in the table itself. The Database has been configured with UTF8 encoding.
For example: when I run the above mentioned copy query, if the special character shows up in the column with ID=5. If I run the same copy query with (select * from tablename where ID=5), the special char is not there:
\copy (select * from TableName where ID=5) to 'C:\Database\bb.txt' with delimiter E'\t' null as '';
This happens on a Windows machine. Can someone tell me where these special characters are coming from?
Related
Currently having issues with running sql code through pypyodbc in python. The odbc is connected to netezza to run sql code through there.
db = pyodbc.connect(connection_string)
cursor = db.cursor()
sql = '''
Create external table TABLE1 (
col1,
col2,
col3
)
using (
dataobject('C:\\file.txt')
delimiter '|'
quotedvalue 'DOUBLE'
requiresquotes TRUE
nullvalue ''
escapechar '`'
y2base 2000
encoding 'internal'
remotesource 'ODBC'
logdir 'C:\\log'
);
Create table TABLE1_MAILBOX as (
select * from TABLE1
) distribute on random;
'''
cursor.execute(sql)
db.commit()
The first block of code that creates the external table works fine, it's when the second block of code that creates TABLE1_MAILBOX that things go wrong. It seems like it misses lines in TABLE1. For example, if the dataobject text file has 5000 lines, TABLE1 has 5000 lines, but TABLE1_MAILBOX has about 4750 lines.
However, if I run this code directly in netezza, it works just fine and each table has the correct amount of lines.
Not sure why it is doing through if it is run through pypyodbc in python. Could it be a glitch? Or am I doing something wrong?
I have a script with which I am trying to load data stored in a .txt file using SQL Server. The text file is stored as UTF-8 encoding. The data read from the txt file showing bom character appended to the first line of data. I am using SQL Server 2012 version.
How can I avoid the bom character while reading?
Here's what I have tried:
CREATE TABLE #Temp1(record nvarchar(512))
BULK INSERT #Temp1
FROM 'C:\Test\test.txt'
WITH (ROWTERMINATOR = '\n')
SELECT * FROM #Temp1
Temp table result is shown as:
test.txt is as below
i have problem with loading rows from a file, the point is that when im using External table like this
create table table_name
(
id VARCHAR2(60)
)
organization external
(
type ORACLE_LOADER
default directory DIRECTORY
access parameters
(
RECORDS DELIMITED BY NEWLINE CHARACTERSET EE8MSWIN1250 nobadfile nodiscardfile
FIELDS TERMINATED BY ";" OPTIONALLY ENCLOSED BY '\"' LDRTRIM
REJECT ROWS WITH ALL NULL FIELDS
(
ID VARCHAR2(60)
)
)
location ('tmp.txt')
)
reject limit 0;
my all rows have the newLine byte at the end of row, only thing that works is after loading data from file is update all rows using this
update table_name
set id = translate (id, 'x'||CHR(10)||CHR(13), 'x');
How can i make it automatically?
Check exactly what newline charcters are in your file and than define the record delimiter explicitely.
Example
records delimited by '\r\n'
The probable cause of your problem is that the newline character is not compatible with your operating system - which topic you can address as well.
while may have line delimiter as either \n or \r\n..
you can check that by opening file in notepad++ or any other supporting editor and by clicking show all characters
based no how the data is in the life you may create the external table as
RECORDS DELIMITED BY '\r\n' or
RECORDS DELIMITED BY '\n' etx
I have a few processes where I use the copy command to copy data from S3 into Redshift.
I have a new csv file where I am unable to figure out how I can bring in the "note" field- which is a free hand field a sales person writes anything into. It can have ";", ",", ".", spaces, new lines- anything.
Any common suggestions to copy this type of field? it is varchar(max) type in table_name.
Using this:
copy table_name
from 's3://location'
iam_role 'something'
delimiter as ','
ignoreheader 1
escape
removequotes
acceptinvchars
I get Delimiter not found
Using this:
copy table_name
from 's3://location'
iam_role 'something'
delimiter as ','
fillrecord
ignoreheader 1
escape
removequotes
acceptinvchars
I get String length exceeds DDL length
The second copy command command fixed your initial issue, namely of copy parsing the csv file. But now it can't be inserted because the input value exceeds the maximum column length of yr column in database. Try increasing the size of the column:
Alter column data type in Amazon Redshift
I am trying to import a txt file which has Hebrew customer names into a SQL Server Database, and for that NAME to be displayed in Hebrew for that COLUMN only.
I know how to insert it with this special insert statement below which works . If only I could grab that field from the txt file in Hebrew in SSIS .
create table #temp ( str Nvarchar(100) ) ;
insert into #temp(str)values(N'HEBREW SAMPLE IN HERE') ;
select * from #temp ;
drop table #temp;
The problem with SSIS is that all the import tasks put the Hebrew customer name in some junk format and I cant do anything then. Current collation on server, database, tables columns is ;
1. SQL_Latin1_General_CP1_CI_AS
2. SQL_Latin1_General_CP1_CI_AS
3. SQL_Latin1_General_CP1_CI_AS
Chane your database to HEBREW_CI_AS instead. You can do that at your database options for each database.
Choose property menu, then select options. First pulldown you can select sorter.