I am working on a data migration project. In current task, I have to select data from n number of tables from Oracle, join them and insert the data into a single SQL table. The number of rows are in millions.
Issue: There is data in Oracle which when we are trying to insert in SQL is giving exception. For example the datatype of the Oracle column is VARCHAR2 whereas in SQL it's int. The data is numbers. But there are few columns which have special characters like ','. This is one such example which will fail when we are trying to insert into SQL table. It's failing for many such columns.
I am using SSIS for this task. I am moving all the error id's of the rows into an error table which are throwing such error as mentioned in above example.
Question: I want the column name for which the insertion is failing for each row. Is there an option in SSIS? On error I want to store the id and the column name in an Error table.
Tried to search on internet, but didn't get anything. In SSIS, we do have option to configure the rows having Error. But didn't find that giving column name option to insert into a error table.
Edit: The data will come on daily basis i.e. the SSIS package will be executed daily.
The Error Output contains many columns providing information about it.
The list of columns includes the columns in the component input, the ErrorCode and ErrorColumn columns added by previous error outputs, and the ErrorCode and ErrorColumn columns added by this component.
If you are using OLEDB Destination, you cannot redirect the error rows while using Fast load option. And since you mentioned that
The number of rows are in millions.
Then it is not recommended to use the Row-by-Row insertion.
If there are few columns, i suggest adding a Data Conversion Transformation and use its Error output to get the error information.
References and helpful links
Configuring Error Output Columns
SSIS how to redirect the rows in OLEDB Destination when the fast load option is turned on and maximum insert commit size set to zero
Error Handling in Data
Error Handling With OLE DB Destinations
I'm trying to insert a row in the table by using below code but it is throwing an error. can anyone help me out to solve the error?
Thanks in advance!!
db2 "Insert into TARIFF_PRODUCT_ATTRIBUTES values (409499, 'ADDITION_SMS_TEMPLATE', 'IDSSMS1')";
Error is :
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0204N "DB2EAI2.TARIFF_PRODUCT_ATTRIBUTES" is an undefined name.
SQLSTATE=42704
Common causes of SQL0204N in Db2:
spelling mistake in the object name
object does not exist in the currently connected Db2 database
object exists in current database but in a different schema than your current default schema (so you must qualify the name with the correct schema-name).
mixed case table name (Db2 will always uppercase unquoted object names, so if the object is Tariff_Product_Attributes then use double-quotes around the name in the SQL to allow Db2 to find the object).
There are other less common causes , see the documentation for the complete list.
I have table in db2 (using bigsql) that is partitioned as per date on IBM BigInsights
table_name_abc
20150810
data corresponding to partition
20150811
data corresponding to partition
....
what I want is to delete particular partition say 20150810 or delete data from that partition
I tried this
db2 "truncate table test_schema.table_name_abc where partition_date = 20150810";
But it gave following error
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL0104N An unexpected token "where" was found following "test_table".
Expected tokens may include: "". SQLSTATE=42601
Can someone please instruct on how to do this?
Solved it by using the following command
db2 "ALTER TABLE test_schema.table_name_abc DROP PARTITION (partition_date = 20150515)";
Adding it as answer just in case someone needs it
I am performing a DB2 load, and I am struggling to understand the impact of using GeneratedOverride over IdentityOverride. When I run the following command:
db2 load from tab123.ixf of ixf replace into application.table_abc
All rows are rejected, with the following error being the culprit:
SQL3550W The field value in row row-number and column column-number is not NULL, but the target column has been defined as GENERATED ALWAYS.
So to try and step around this, I executed
:
db2 load from tab123.ixf of ixf modified by identityoverride replace into application.table_abc
But this immediately returned this error:
SQL3526N The modifier clause "IDENTITY OVERRIDE" is inconsistent with the current load command. Reason code: "3".
From checking the reason code I see that the issue is "Generated or identity related file type modifiers have been specified but the target table contains no such columns." .. but the SQL3550W error seems to infer that the columns are generated always!
The only way I can get these rows to commit to the table is to run..
db2 load from tab123.ixf of ixf modified by generatedoverride replace into application.table_abc
Can anyone enlighten me to why I am recieving the SQL3526N error, or what the implications of running generatedoverride are?
Thanks for sticking with me..
Generated columns are not necessarily identity columns, apparently that's the case in your situation. Check the CREATE TABLE syntax to see what are other ways to generate column values.
By using the GENERATEDOVERRIDE option during the load you are obviously replacing (overriding) the generated values with those from the input file.
I am trying to archive some of my tables into another database on the same server. However the INSERT INTO...SELECT...FROM gives me an error (SQLSTATE=42704) on build. The table exists in the second database.
Can anyone help with this?
It's not clear from your question what version of DB2 is being used. I'll presume that it's the Linux, Unix & Windows version. You look to be using federation to link the two databases.
Does the SELECT part of your query work from LS2DB001? It's worth trying to pin down which database you have the issue with.
Presuming that the problem is on LS2DB001, if the user you have defined the federated link with has permissions on the base tables in the query, check also that they have permissions on the system catalog tables. If not, they would not be able to parse and validate that you can run the query.
We've cracked it! If the following script is used then it works. The LOAD works without having to COMMIT in between batches of rows copied. ('Transaction Log full...' error problem is also solved)
CONNECT TO LS2DB001;
EXPORT TO "C:\temp\TIN_TRIGGER_OUT.IXF" OF IXF
MESSAGES "C:\temp\TIN_TRIGGER_OUT.EXM"
SELECT * FROM LS2USER.TIN_TRIGGER_OUT;
CONNECT RESET;
CONNECT TO LQIFCOLD;
LOAD FROM "C:\temp\TIN_TRIGGER_OUT.IXF" OF IXF
MESSAGES "C:\temp\TIN_TRIGGER_OUT.IMM"
INSERT INTO LS2USER.TIN_TRIGGER_OUT COPY NO INDEXING MODE AUTOSELECT;
COMMIT;
CONNECT RESET;
I found this on http://www.connx.com/products/connx/Connx%208.6%20UserGuide/CONNXCDD32D/DB2_SQL_States.htm:
42704 Undefined object or constraint name. Revise SQL syntax and retry.
For more help try to be more specific, eg paste the full sql statement, the table scheme etc.
You can do
Select 'insert into tblxxxx (blabla,blabal) values(' + fld1 + ',' + fld2 + ',' ...... + ')'
From tblxxxxxx
copy the result as a text script and execute it in the other DB.
The best way to do this would be to create a custom script. Depending on the size of the tables (how many records) you could either do a select of all of the data into memory and then roll over them inserting them into a copy of the table you create first, or you could export the data out as a csv file or some other text based file and then roll over that to insert the data into the other table.
If you do not have some sort of formal backup procedures that could do this already, this would be your best bet.
Note: some db2 databases, such as those on an iSeries do not actually have "databases", they have libraries. With the right user profile you can access two libraries at the same time, joining tables from them together or doing a
create table library/newFilename as
(select * from originallibrary/originalfilename) with data
But this only applies to the iSeries I believe.
I'm writing this response as another answer so I have more space.
I can only suggest breaking the steps down to their components, and working through to see where the error is occuring. Again, I'm assuming you're using federation:
a) In your FROM db, connecting as the user you're using for the federated link, does your select work?
b) In your TO db, using the link, does the select work?
c) In your TO db, using the link via a stored proc, does the select work?
d) In your TO db, using an INSERT...values(x,y,z), can you insert into the table?
e) In your TO db, via a stored proc, using INSERT...values(x,y,z), can you insert?
Without more information, this is the best line of attack I can suggest.