Am getting:
ERROR [HY008] Operation canceled
ERROR [08S01] Communication link failure
When trying to upload external .txt file to Netezza database. I have done this in the past (just last week), but today receive this error. Am able to connect to DB, ran truncate and create table statements for the loading of this data, can select, etc...But yet no luck loading. It's around 200K records and here is my code:
INSERT INTO PTG_ITO_ETL.FINANCE_TY15_RT_TPG
SELECT * FROM EXTERNAL 'C:\\Users\\Documents\\Data Sources\\Finance_FY15_RT\\SBTPG\\TPG_INTUIT_RT_PRODIV_20150214.TXT'
USING
(
MAXERRORS 1
DATESTYLE 'MDY'
DATEDELIM '/'
BOOLSTYLE 'Y_N'
Y2BASE 2000
ENCODING 'internal'
SKIPROWS 1
REMOTESOURCE 'ODBC'
ESCAPECHAR '\'
)
Have tried solution from only other post I could find on the subject:
ERROR [08S01] Communication link failure while inserting data in to external table in netezza
"I have found in Windows 7 and Windows Server 2008 R2 TCP Chimney Settings were the culprit.
http://blogs.dirteam.com/blogs/sanderberkouwer/archive/2008/05/15/backward-compatible-networking-with-server-core.aspx
The Follwing Commands Fixed this issue for me:
netsh interface tcp set global rss=disabled
netsh interface tcp set global chimney=disabled
netsh interface tcp set global autotuning=disabled"
But, to no avail. Don't know what is causing this issue? Am on windows 7 using Aginity, Netezza version 7.0.4 Thanks!
Thanks,
Craig
Please try the following and see if it errors
SELECT * FROM EXTERNAL 'C:\\Users\\Documents\\Data Sources\\Finance_FY15_RT\\SBTPG\\TPG_INTUIT_RT_PRODIV_20150214.TXT'
(field1 varchar(20000))
USING
(
MAXERRORS 1
Delim 199
DATESTYLE 'MDY'
DATEDELIM '/'
BOOLSTYLE 'Y_N'
Y2BASE 2000
ENCODING 'internal'
SKIPROWS 1
REMOTESOURCE 'ODBC'
ESCAPECHAR '\'
)
Appreciate the Help from #ScottMcG, after downloading the latest Netezza Driver (7.0.4.7) I was able to complete this job. Still errored out (with same error) a couple times but also worked many with extensive testing.
I read another post similar and this error seems common the larger the records go. I read one where person was using an external table with a million+ records and errored out every time. If need be I (you) may need to split the data into smaller external tables for loading to one table/DB. Seems to start having problems around 200k+.
This is also from my laptop, so maybe much higher if you were on a box inside the Data Center. I would suggest the driver upgrade and then splitting to smaller files if necessary.
Thanks All!
Related
I have a very strange 'reload.sql' file that I need to use to build a database.
It references about 200 XXX.dat files with straight-up readable data (although useless without explanations regarding the meaning of the fields).
I have tried msssql server, mysql workbench (on a server local-hosted on wamp), and directly accessing it through DBeaver and IBConsole, but I cannot manage to execute/build it.
It uses a weird syntax. There are elements like
begin
...
end
go
that hinted me towards T-SQL, but using sqlcmd on it gave me thousands upon thousands of errors regarding keywords.
Specifically, the very first batch of executable lines says
SET OPTION date_order = 'YMD'
go
SET OPTION PUBLIC.preserve_source_format = 'OFF'
go
SET TEMPORARY OPTION tsql_outer_joins = 'ON'
go
SET TEMPORARY OPTION st_geometry_describe_type = 'binary'
go
SET TEMPORARY OPTION st_geometry_on_invalid = 'Ignore'
go
SET TEMPORARY OPTION non_keywords = 'attach,compressed,detach,kerberos,nchar,nvarchar,refresh,varbit'
go
which generates about 150 errors 'Incorrect syntax near OPTION keyword' on its own, and according to google is part of a 'rexx' procedure but 'date_order' should then be 'DATFMT', right?
Another track is that of SyBase, but I cannot for the life of me get it to work (through my trials I did manage to build a .db file, that, well, is useless to me since I can't build it either..).
I've tried accessing it through ODBC pilots as well but none worked (the paradox ODBC did not crash, but said there was an error with a FROM clause, which are generated automatically...).
I need to know a way to build a database from this file or directly access the data it references, which I can't really post since it contains private medical data.
Also what madman came up with this.
The very first google link (for me anyway) against 'st-geometry-describe-option' shows this is a SAP SQL Anywhere database i.e. http://dcx.sybase.com/1200/en/dbadmin/st-geometry-describe-option.html
So I would suggest starting from the SQL Anywhere documentation and you will need to install the database software beforehand.
Issue with Delphi legacy code. Added one line of code to correct one error and created a new error.
New error is causing the same executable to yield different results on different servers(switched the pointer from dev to prod environment on the executable).
code:
sEscapedString:=stringreplace(sStringIn,'[','''+char(27)+''[',[rfReplaceAll]);
sEscapedString:=stringreplace(sEscapedString,']','''+char(27)+'']',[rfReplaceAll]);
sEscapedString:=stringreplace(sEscapedString,'''','''''',[rfReplaceAll]);// this line created new
bug
result:=' like ''' + Trim(sEscapedString) + '%'''+' escape char(27) ';
When running the code against dev this query finds objects with the characters '[' and ']' in it
Against prod the query does not find those items:
The first thing I checked was the data: Exactly identical in both cases
The second thing I checked was SQL server versions (11.0.3128 on BOTH servers)
The third thing I am checking is settings on those servers:
DBCC USEROPTIONS; -- same on both
SELECT name, collation_Name FROM sys.databases -- same on both
select ##OPTIONS -- same on both.
Quoted identifiers are 'ON' for both servers
It comes down to the fact that I know one server is treating the escape character (chr(27)) differently than the other but I do knot know why.
Does anyone have a theory(or answer) as to why the 2 similar servers are treating the escape characters differently?
The goal here is getting the prod server to return values with '[' and ']', as setting up my system to work with the legacy code will take a LOT of additional time. I do have a fix for the code
sEscapedString:=stringreplace(sStringIn,'[','[[]',[rfReplaceAll]);
But the faster option would seem to be getting the server to read the values the same.
Update: We found the root cause of the difference and it was more mundane than what we expected, turns out the query we were running was actually executed twice. The second execution was missing the key piece on the production server.
The issue was resolved by moving the new line of code so that it executed first rather than last.
I would first try to find out if this SQL only causes different behaviour when it is sent from the application: by sending the SQL from an interactive SQL client tool to both servers.
To make sure that the manually tested SQL is exactly the same as in the application, I would try to log or capture the exact SQL as sent from the application as a text file and then paste its content to the SQL client tool.
If the server is the culprit, then using the SQL from a different client tool should cause the same difference with the two servers. If the client tool shows the same (correct) result on both servers, then something is going on in the Delphi application.
p.s. upvoted, it is an interesting phenomenon
I recently updated an SSIS package that had been working fine and now I receive the following error:
Text was truncated or one or more characters had no match in the target code page.
The package effectively transferred data from tables in one database to a table in another database on another server. The update I made was to add another column to the transfer. The column is Char(10) in length and it is the same length on both the source and destination server. Before the data is transferred it Char(10) there as well. I've seen people reporting this error in blog posts as well as on Stack, none of what I have read has helped. One solution I read about involved using a data conversion to explicitly change the offending column, this did not help (or I misapplied the fix).
whihc version of SQl Server and SSIS are you usign?
I would say to take a look at the output and imput fields of your components. CHAR always ocupies all it's length (I mean, char(10) will always use 10 bytes) and since you are having a truncation error, it may be a start. try to increase the size of the field or cast as varchar on the query that loads the data (not as a permanet solution, just to try to isolate the problem)
Which connection you are using ADO.Net or OLEDB connection ??
Try deleting the source and destination if there are not much of changes you have to make ..Sometime the metadata cuases this problems. If this doesn't solve your problem post the screen shot of error.
We have a merge replication setup on SQL Server that goes like this: 1 SQL server at the office, another SQL server traveling around the world. The publisher is the SQL server at the office.
In about 1% of the cases, two of our tables with a column of XML Data type (not bound to a schema) are replicated with rows containing empty XML columns. ( This only happened when data is sent from the "traveling server" back home, but then again, data seems to be changed more often there ). We only have this in prod. environment ( WAN replication ).
Things i have verified:
The row is replicated, as the last modification date on the row is refreshed but the xml column is empty. Of course it is not empty on the other SQL Server.
No conflicts are displayed in the replication conflicts UI.
It is not caused by the size of the data inside the XML Column as some are very small.
Usually, the problem occurs in batch. ( The xml column of 8-9 consecutive rows will be empty )
The problem occurs if a row was inserted OR updated. No pattern there.
The problem seems to occur, but this is pure speculation on my part when the connection is weaker. ( We've seen this problem happen more often when the server was far away as compared to when it was close by. )
Sorry if i have confused some things, I am not really a DBA, more of a DEV with knowledge of SQL but since the application using the database keeps getting blamed for the problems ( the XML column must not be empty!! ) I have taken it at heart to try and find the problem instead of just manually patching the data each time ( Whats the use of replication if you have to do that? )
If anyone could help out with this problem, or at least suggest some ways of being able to debug / investigate this it would be greatly appreciated.
I did search alot on google and I did find this: Hot Fix . But we do have the latest service pack and the problem seems a bit different.
fyi: We have a replication setup locally here but the problem never occurs. We will be trying a WAN simulator on it as well to see if that can help.
Thanks
Edit: hot fix is now available for my issue: http://support.microsoft.com/kb/2591902
After logging this issue with Microsoft, we were able to reproduce the problem without a slow link ( Big thanks to the competent escalation engineer at Microsoft ). The repro is a bit different from our scenario, but highlights the timing issue we were getting perfectly.
Create 2 tables – One parent one child (have a PK-FK relationship)
Insert 2 rows in the parent table
Set up replication – configure merge agent to run ON DEMAND
Sync
Once all is replicated:
On the PUBLISHER: delete one row from the parent table
On the SUBSCRIBER: Insert 2 rows of data that references the parentid you deleted above
Insert 5 rows of data that references the parentid that will stay in the table
Sync, Merge agent will fail, Sync again, Merge agent will succeed
Missing XML data on the publisher on the 5 rows.
Seems it is a bug that is in SQL Server 2005/2008 and 2008R2.
It will be addressed in a hot fix in 2008 and up. ( As SQL Server 2005 is no longer being altered )
Cheers.
You may want to start out by slapping a bandaid on this perplexing situation to buy some time to fully investigate and fix (or more likely get MS to fix it). SQL Data Compare is an excellent tool that might help.
Figured i'd put an update here as this issue got me a few gray hairs and I am somewhat closer to a solution now.
I finally had some time to work on this and managed to reproduce this issue in our test environment, using a WAN simulator and slowing down the link and injecting some random packet loss. ( to best simulate the production environment where the server is overseas on a really bad line ).
After doing some SQL tracing, and some verbose logging here are my conclusions:
When replicating a row with an XML column, the process is done in 2 steps. First an insert is done of the full row but with an empty string for the XML column. Right after, an update is done this time with the XML column having data. Since the link is slow, in some situations a foreign key violation occured.
In this scenario, Table2 depends on Table1. After finishing replicating table1, and starting to replicate table2 (Enumration of insert/updates which takes time on a slow link), some entries were added to table1 and table2. Therefore some inserts on Table2 failed because Table1 entries were not in the database and were only going to be replicated next batch. The next time the replication occured, no more foreign key violations occured, however when it tried to insert the row that had previously failed in Table2 ( XML column row ), the update part of it was missing ( I could see that in the SQL profiler ) and that is why the row ended up after all was done with an empty XML.
Setting "Enforce for replication" to false on the foreign keys seems to address the problem, however I do still think that this whole process should work with the option set to true.
I logged a support call with Microsoft for this. I have sent the traces and logs to Microsoft and will see what they have to say.
I've read this article: http://msdn.microsoft.com/en-us/library/ms152529(v=SQL.90).aspx. But for me, setting this option to false is kind of a work around, no?
What do you guys think?
ps: Hope this is clear, tried to explain it the best I could. English is not my first language.
I started using the system with a pre-made file called DB2.SQL. I am using this because it is what the tutorial said to use. I then edited this file and replaced the contents with my own code:
CREATE DATABASE BANKDB13 BUFFERPOOL BP0;
When I try to execute a SQL it though, I get this error:
DSNE377A INPUT DATA SET RECFM MUST BE F OR FB WTIH LRECL 80
What does this error mean and how do I correct it on the file?
I am running it with Vista TN3270 on Windows 7 over TSO, in SPUFI mode.
What I've tried so far:
When I start editing the file, I have a screen to change the defualts, and I have changed the RECORD FORMAT to F and FB as well as setting the RECORD LENGTH to 80 with no success.
EDIT:
I resolved the problem by deleting the DB2.SQL file and recreating it, and also making sure that the sizes I gave for the files were consistent with each other.
What SQL are you trying to execute on it?
The error means that the Record Format in the input data set must be either "F IXED" or "F IXED B" LOCK with a logical record length of 80.
So this is what the error means, how to correct it depends on the SQL you're running and the desired outcome.
What Tutorial is it that you refer to, do you have a link? Is this a real world problem, homework or you expanding your knowledge into mainframe DB2?
Your SQL snippet above is creating a DB, what is the INPUT DATASET file format that you are subsequently running SQL on?